Spatiotemporal Representation Learning on Event Stream
Beibei Yang, Weiling Li, Guangyu Jiang, Zhigang Liu, Yurong Zhong, Yan Fang · 2024
The spatiotemporal correlation of events obtained by event camera contains the operational laws of moving targets. For deeper understanding events, an effective spatiotemporal representation learning-based model is desired. Note that events are dense in time and sparse in space, how to capture the global correlations of all events in the event stream simultaneously while well addressing the sparse input is a challenging task. To fulfil this task, an Elastic Net-incorporated tensor network (ENTN) model is proposed in this work with twofold ideas: a) utilizing a fully-connected 3rd-order tensor network for tensor decomposition to implement spatiotemporal representation learning on event stream, b) incorporating Elastic-net regularization to perform better feature selection. Experimental results indicate that ENTN can represent the spatiotemporal correlation of events in an event stream with high quality.