Deep Convolutional Networks with Recurrence for Eye-Tracking
Linus Härenstam-Nielsen · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2018
This thesis explores the use of temporally recurrent connections in convolutional neural networks for eye-tracking. We specifically investigate the impact of replacing the convolutional layers in a regular CNN with convolutional LSTMs and replacing the fully connected feature layers with regular RNNs and LSTMs. This requires us to transition from a static single-frame input model to a time-dependent multipleframe input model. Doing so naturally introduces extra complexity to the eye-tracking pipeline, so we highlight the advantages and disadvantages. Our results show that adding LSTM-cells to the convolutional layers and RNN-cells to the feature layers can increase eyetracking performance, but also that LSTM-recurrence in the featurelayers can be detrimental to performance.