Long-Term Memory Neural Turing Machines
笑 解 · Computer Science and Application · 2018
可读写的外部记忆模块可以在事实的记忆和基于记忆的推理上扩充神经网络的能力。神经图灵机利用注意力机制设计了一种对内存模块的读写机制，并以循环神经网络作为控制器，实现了排序、复制等算法。为了在更广泛的应用(例如自然语言处理)中缩短训练时间或加快收敛速度，我们在神经图灵机的基础上设计了一种基于全局内存的读写机制，利用卷积运算提取全局内存特征。对于一些较长的序列任务，训练速度相对于神经图灵机提高了6倍，收敛速度也有所提升，在bAbi数据集中取得了更好的推理分类结果。 The readable and writable external memory module can improve the ability of the neural network which is based on factual memory and memory-based reasoning. Neural Turing Machines use attention mechanism to design a reading and writing mechanism for the memory module. It also realized sorting, copying and other algorithms by using a recurrent neural network as a controller. To shorten the training time and speed up convergence in a wider range of applications (such as natural language processing), we design a kind of read-write mechanism based on the global memory which is applied in Turing machines. This kind of read-write mechanism does convolution operations to extract the global memory features, whose training speed is 6 times better than Neural Turing Machines; the convergence rate and reasoning results in the bAbi dataset are also better.