Automated HPOMDP Construction through Data-mining Techniques in the Intelligent Environment Domain.
G. Michael Youngblood, Edwin O. Heierman, Diane J. Cook, Lawrence B. Holder · 2005
Markov models provide a useful representation of sys-tem behavioral actions and state observations, but they do not scale well. Utilizing a hierarchy and abstrac-tion as in hierarchical hidden Markov models (HH-MMs) improves scalability, but they are usually con-structed manually using knowledge engineering tech-niques. In this paper, we introduce a new method of automatically constructing HHMMs using the output of a sequential data-mining algorithm, Episode Discov-ery. Repetitive behavioral actions in sensor rich envi-ronments can be observed and categorized into periodic episodes through data-mining techniques utilizing the minimum description length principle. From these dis-covered episodes, we demonstrate an automated tech-nique for creating HHMMs and subsequent hierarchi-cal POMDPs (HPOMDPs) for intelligent environment research. We present the theory of this technique and frame it in a case study involving our MavHome archi-tecture and an on-campus smart apartment with a real inhabitant.