Construction and learning among neurally controlled animats
Frederick L. Crabbe, Michael G. Dyer · 2000
This dissertation describes the development of animats that perform construction. The animats are agents situated in a two-dimensional environment called DiscoTech. The agents have an action selection architecture (Layered-SON) that enables them to behave robustly, select behavior that is a compromise between multiply active goals, persist in performing a behavior for the correct length of time, prioritize multiple goals, achieve sequences of goals, interrupt sequences of goals, correct errors, and take advantage of serendipitous opportunities. Controlled by this architecture, the agents can build walls, corridors, intersections, and briar patches, while maintaining their general health. The agents learn how to navigate in their environment using the MAXSON algorithm, a layered-SON based learning algorithm that learns faster than another commonly used algorithm (Q-learning). Extensions on MAXSON enable the agents to learn from each other. They learn vicariously, by observing the behavior of other agents and the effects of that behavior to learn how to make decisions in similar circumstances. It is shown that in dangerous environments, vicarious learning can significantly increase the expected survival time of an agent. It is also shown that in environments with scarce resources, vicarious learning can improve the average behavior of the population. The agents can also learn by imitation, where one agent performs a task and another agent follows, observes, and learns how to perform that task as well. The agents learn in two stages. In the first stage they learn, by observation, the sequence of steps that make up the task. In the second stage, the agents learn how to perform each step by trial and error. The combination of the layered-SON architecture with MAXSON and the social learning results in a complete system for animat control and learning that enables them to robustly achieve goals and learn from their environment and each other to build primitive constructions.