Active perception and recognition learning system based on Actor‐Q architecture
Katsunari Shibata, Tetsuo Nishino, Yoichi Okabe · Systems and Computers in Japan · 2002
Abstract This paper proposes the Actor‐Q architecture, which is a combination of Q‐Learning and Actor‐Critic architecture, as well as the active perception and recognition learning system based on that architecture. In Actor‐Q architecture, the system output is divided into the “action,” which is a discrete value as intention, and the “motion,” which is a continuous‐valued vector. As the first step, the “action” is determined from Q‐values. If the “action” is accompanied with a “motion,” the “motion” is executed according to the corresponding Actor output. Q‐value is learned by Q‐learning, and Actor is trained with the Q‐value corresponding to that “action” on behalf of the Critic output. In this study, the action is defined as the decision of the sensor motion or the recognition of the respective pattern. Q‐value is assigned to each of those. When the sensor motion is selected, the sensor is moved according to the Actor output. When recognition is selected, the recognition result that the presented pattern is the one corresponding to the selected Q‐value is output. The Q‐value is learned, using the reinforcement signal representing the true/false of the result. Both Q‐value computing module and Actor are composed of neural networks, with the visual sensor signals as input. By this architecture, the following three problems of the conventional active perception and recognition learning system are dissolved. (1) The sensor can be trapped in a local maximum of the recognition evaluation. (2) It is necessary that the recognition output should be evaluated for each time‐step, and the reinforcement signal with a continuous value should be provided. (3) The system cannot decide by itself the timing to output the recognition result. The above effect was verified by some simulations, using the visual sensor with nonuniform sensor cells. © 2002 Wiley Periodicals, Inc. Syst Comp Jpn, 33(14): 12–22, 2002; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/scj.10207