Continuous Reinforced Snap-Drift Learning in a Neural Architecture for Proxylet Selection in Active Computer Networks
Dominic Palmer-Brown, Sin Wee Lee · UEL Research Repository (University of East London) · 2005
A new continuous learning method is used to optimise the selection of services in response to user requests in an active computer network simulation environment. The learning is an enhanced version of the ‘snap-drift’ algorithm, which employs the complementary concepts of fast, minimalist (snap) learning and slower drift (towards the input patterns) learning, in a non-stationary environment where new patterns arrive continually. Snap is based on Adaptive Resonance Theory, and drift on Learning Vector Quantisation. The new algorithm swaps its learning style between these two self-organisational modes when declining performance is detected, but maintains the same learning mode during episodes of improved performance. Performance updates occur at the end of each epoch. Reinforcement is implemented by enabling learning on any given pattern with a probability that increases linearly with declining performance. This method, which is capable of rapid re-learning, is used in the design of a modular neural network system: Performance-guided Adaptive Resonance Theory (P-ART). Simulations involving a requirement to continuously adapt to make appropirate decisions within a BT active computer network environment, demonstrate the learning is stable, and able to discover alternative solutions in rapid response to new performance requirements or significant changes in the stream of input patterns.