Keynote Lecture - Towards Efficient Real-Time Perception in Self-Driving Cars: Methods, Challenges and Open Questions
Stefan Mathe · 2023
Arguably the first mass-produced consumer-oriented intelligent autonomous robots, self driving cars are subject to stringent and conflicting design and operating constraints. On one hand, they need to accurately sense and understand their surroundings, predict changes in a dynamic and uncertain environment, and formulate safe navigation plans. On the other hand, a self-driving care must react fast, consume little power, while at the same time being cost-effective. A viable product must meet these constraints “in the wild”, with safety argumentations extending beyond empirical validation cases, towards foreseen and, sometimes, even unforeseen scenarios. Given this seemingly daunting task, in this workshop, we tackle the more modest — but still tremendously challenging — visual sensing and scene understanding problems. As human beings, we solve this problem effortlessly, in real-time and with astonishing accuracy. The true difficulty surfaces when we try to design systems that do the same: a step-by-step procedure (algorithm) eludes us. We find ourselves in need to resort to machine learning techniques to automatically find “good” solutions. But this opens a Pandora’s box. How do we define a good solution: should we use our own perception on a (unavodably limited) set of scenarios as the “gold” standard? Will such a solution work in other scenarios? How can we argue for safety? Can we explain the behavior of the system? Does its reasoning process resemble ours in any way? Finally, how do we reduce computational costs while not compromising predictive accuracy? In this workshop, we aim to briefly revisit the currently available methods that can help answer these questions. In our journey we shall touch on the three core elements of machine learning, the task - What does the perception system need to solve? - the experience - How does the learning algorithm interact with the world in order to provide a good solution? - and the performance measure - How do we provide feedback on what a good solution is? By presenting rigorous formulations for these elements, the methods we revisit open the path towards a working practical system, and partly answer our questions. Finally, while we analyze the merits and trade-offs in state-of-the-art methods, we use the opportunity to highlight open problems and challenges, from both a theoretical and purely pragmatic perspective.