Convolutional neural networks in vision neuroscience
Katja Seeliger · Radboud Repository (Radboud University) · 2021
The thesis consists of the following projects: Convolutional neural network-based encoding and decoding of visual object recognition in space and time shows that representations from a modern convolutional neural network trained for object recognition reflect the spatio-temporal visual information processing hierarchy in humans, as measured in MEG and using the encoding model framework.Generative adversarial networks for reconstructing natural images from brain activity applies recent advances in neural network-based generative models to the idea of reconstructing perceived images from brain activity.A large single-participant fMRI data set for probing brain responses to naturalistic stimuli in space and time describes the recording of a large single-participant data set of a whole brain responding to spatio-temporal visual and auditory information, with the aim of providing sufficient data for training modern neural networks directly on brain activity.Neural system identification with neural information flow introduces a new framework for training modern neural networks end-to-end on brain activity, depicting neural information processing systems between multiple areas within the architecture itself.Using the prediction of brain activity in response to experimental conditions as the objective function, the model learns the underlying hierarchy of cognitive information processing.In this way we remove biases introduced by training on external data bases and human-defined objective functions and get closer to the true optimization goals of biological information processing systems.