Structural Estimation of Partially Observable Markov Decision Processes
Yanling Chang, Alfredo Garcia, Zhide Wang, Lu Sun · IEEE Transactions on Automatic Control · 2022
Partially observable Markov decision processes (POMDPs) is a well-developed framework for sequential decision-making under uncertainty and partial information. This article considers the (inverse) structural estimation of the primitives of a POMDP based upon data in the form of sequences of observables and implemented actions. We analyze the structural properties of an entropy regularized POMDP and specify conditions under which the model is identifiable without knowledge of the state dynamics. We consider asoftpolicy gradient algorithm to compute a maximum likelihood estimator, and illustrate the approach with an equipment replacement problem.