Superresolution via bilinear fusion of multimodal imaging data
Pulak Sarangi, Piya Pal · 2019
Advancements in neural imaging now allow simultaneous acquisition of multiple imaging modalities. When these multimodal data are combined with advanced signal processing algorithms, they can provide a better understanding of brain dynamics, which was otherwise not possible. In this paper, we specifically demonstrate a new technique for the fusion" of neural activity recorded with two-photon calcium imaging and Electrocorticography (ECoG) recordings acquired using an electrode array. Calcium signals are usually acquired at a low sampling frequency and have a high spatial resolution, but suffer in temporal resolution due to blurring of the spiking signal. ECoG, on the other hand, has high temporal resolution and is acquired at high sampling frequency, but can only detect aggregated neural population activity and suffers from poor spatial resolution. In this paper, we will develop novel signal processing techniques using bilinear fusion and sparsity-aware reconstruction to overcome these drawbacks. The data from both modalities are represented by a bilinear model which is inverted to infer the spiking activity using suitable prior assumptions such as sparsity or independence of the sources. Our approach leverages on the complementary strengths of the two modalities (in terms of temporal and spatial resolution) along with use of powerful non-convex algorithms that harness the unique structure of the dataset.