Machine Learning for Embodied Agents: From Signals to Symbols and Actions
2019
The aim of this tutorial lecture is to show the role of machine learning and some other AI-related techniques in embodied autonomous agents, and autonomous robots in particular. In this tutorial we bring to the forefront the aspects of robotics that are closely related to computer science. We believe that the progress in algorithms and data processing methods together with the rapid increase in the available computing power were the driving forces behind the successes of modern robotics in the last decade. During this period robots of various classes migrated from university laboratories to commercial companies and then to our everyday life, as now everybody can buy an autonomous vacuum cleaner or lawnmower, while self-driving cars and drones for goods delivery are waiting for proper legal regulations to enter the market. Robotics and Artificial Intelligence already went a long path of mutual inspiration and common development, starting from the symbolic AI (aka Good Old-Fashioned Artificial Intelligence) and its extensive use in early autonomous robots, such as Shakey the robot, created in SRI International by Nils Nilsson, considered one of the "fathers" of modern AI. We briefly characterize the range of the most important applications of typical AI methods in modern robotics, including motion planning algorithms [2,3], interpretation of sensory data leading to creation of a world model [4 ,5], and classical learning methods, such as reinforcement learning [6]. However, what made robotics a part of the new wave of AI applications was the recent "revolution" of machine learning, mostly grounded in the enormous success of the deep learning paradigm and its many variants that proved to outclass classic methods in a broad range of problems related to the processing of images and other types of signals. The quick adoption of the recent advances in Machine Learning (ML) in robotics seems to be motivated by the fact that ML gives the possibility to infer solutions from data, as opposed to the classic model-based paradigm that was for decades used in robotics. Whereas the modelbased solutions are mathematically elegant and theoretically provable (with respect to stability, convergence, etc.) they often fail once confronted with real-world problems and real sensory data, as their underlying mathematical models are only a very rough approximation of the real world. Therefore, a wider adoption of ML in robotics gives a chance to make robots more robust and adaptive. On the other hand, we should try to use the new techniques without discarding the knowledge and expertise we already have - machine learning methods can benefit a lot from the prior knowledge and the known structure of the problem that has to be solved by learning. This knowledge and structure can be adopted from the model-based methods that a re already well-established in robotics. In the lecture robots are understood in a broad sense, as all embodied agents that have means to physically interact with the environment. They can be either manipulators, mobile robots, aerial vehicles, self-driving cars, and various "smart" devices and sensors. In the second part of the lecture attention is paid to specific problems that appear in application of machine learning to embodied agents, such as the need to search a for solution in huge, multi-dimensional spaces ("curse of dimensionality"), and the ever-present problem of representation and incorporation of uncertainty in the processing of real-world data. Some examples of applications of autonomous robots are given, which were successful due to the use of AI - in particular the probabilistic representation of knowledge and machine learning. The most prominent examples are the DARPA competitions: "Grand Challenge", "Urban Challenge" and "Robotics Challenge" (DRC), and the "Amazon Picking Challenge", which proves the interest of large corporations in the development of AI-based robotics [7]. In the third part of the lecture new research directions offered by machine learning and the increased availability of training data are discussed. An overview of the most popular application areas of ML in robotics and other autonomous systems is presented along with the typical machine learning paradigms applied in these areas. The focus is on deep learning, mostly using convolutional neural networks to process various sensory data. We discuss three aspects of embodied agents that make machine learning in robotics quite specific with respect to other application areas, such as medical images or natural language processing. The first aspect is dealing with the "open world", in which autonomous robots usually operate. This situation breaks the assumptions underlying some popular ML methods, and creates the need to face the problem of unknown classes identification [8] incremental learning [9], and the uncertainty of sensory data [10]. We also stress out that an embodied agent has the ability to actively acquire information [11]. The second aspect is the inference about the scene seen by the agent, where in the case of robotics, semantics and geometry intermingle [12], because the robot has to work in a three-dimensional world, although it often perceives it through twodimensional images [13,14]. The third aspect of our analysis is related to the most important feature of robots that distinguishes them from all other learning agents (software-based). Robots are embodied agents, that is they have a physical "body", and are subject to physical constraints, such as the maximum speed of motion or maximum range of perception. Therefore, in ML for robots analysis of the spatio-temporal dependencies in data is very important [15]. Robots support advanced learning methods thanks to the possibility of interaction with the environment - a simple example is active vision with moving camera, a much more complex one is manipulation with active testing of the behavior of objects (repositioning, pushing) [16]. At the end of the lecture, in the context of specific needs and limitations characteristic to the applications of ML in robotics, new concepts of machine learning (e.g. deep reinforcement learning [17], interactive perception [18]) are presented. The lecture is summarized with a brief discussion of the most important challenges and open problems of ML applied to embodied agents.