Learning with Artificial Neural Networks
Shangtong Zhang · ERA: Education and Research Archive (University of Alberta) · 2018
In this thesis, we make two contributions in learning with artificial neural networks. Artificial neural networks have made great success in various challenging domains. Our first contribution is a new technique named cross-propagation that does cross-validation online. In cross-validation, hold-out training data (i.e., validation set) is used to tune hyper-parameters (e.g., step size) of an algorithm. The key idea of cross-propagation is to use the newly coming training example as a hold-out validation set to update parameters (e.g., weights of a neural network) of an algorithm. We propose three cross-propagation-based algorithms to train neural networks and show the advantage of the three algorithms empirically in both online and off-line setting. Our second contribution is a systematic evaluation of experience replay, a method that is commonly used in modern deep reinforcement learning systems to stabilize the training of neural networks. Experience replay involves storing transitions into a memory and training the agent with sampled transitions from the memory. In this thesis, we rethink the utility of experience replay. It introduces a new hyper-parameter, the memory size, which is a task-dependent hyper-parameter. We further propose a simple new experience replay method which requires only little extra computation and made the reinforcement learning system more robust to the selection of the memory size compared with the original experience replay.