Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing [Bookshelf]
Bo Wahlberg · IEEE Control Systems · 2019
As stated in the book’s preface, the author aimed to write a textbook that gives an accessible description of POMDPs, with emphasis on filtering and controlled sensing. The four parts of this book certainly achieve this objective. A nice feature of the text is that it goes into substantial depth in several areas that have not appeared previously in other works. These include structural results for POMDPs, controlled sensing and weak derivatives for reinforcement learning of MDPs, and quickest detection with social learning that involves the interaction of local and global decision makers. The book also has a wealth of examples that motivate POMDPs, including social networks, behavioral economics (risk averse social learning), adaptive radars, and optimal search. The supplementary material contains numerous problem sets and case studies, making this book an excellent text for a graduate-level course. The book is pitched at a similar mathematical level as the classic [10]. Since the text focuses only on discrete-time filtering and stochastic control (POMDPs) where the state space is the unit simplex (therefore, compact), measure-theoretic aspects [11] are not required. Partially Observed Markov Decision Processes: From Filtering To Controlled Sensing is a valuable contribution to the literature on stochastic decision-making/control and stochastic optimization, and I highly recommend it to a diverse audience, including students and researchers in electrical engineering, applied mathematics, computer science, and operations research.