Learning Rapid-Temporal Adaptations
Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, Tong Wang, Adam Trischler · arXiv (Cornell University) · 2017
A hallmark of human intelligence and cognition is its flexibility. One of the long-standing goals in AI research is to replicate this flexibility in a learning machine. In this work we describe a mechanism by which artificial neural networks can learn rapid-temporal adaptation – the ability to adapt quickly to new environments or tasks – that we call adaptive neurons. Adaptive neurons modify their activations with task-specific values retrieved from a working memory. On standard metalearning and few-shot learning benchmarks in both vision and language domains, models augmented with adaptive neurons achieve state-of-the-art results.