Exploring Cross Modality Feature Fusion for Activity Recognition at OpenPack Challenge 2022
Tetsuo Inoshita, Yuto Namba, Yuichi Nakatani, Kenta Ishihara, Sachio Iwasaki, Kosuke Moriwaki, Xian‐Hua Han · 2023
This report describes team vbu211's approach to the OpenPack Challenge 2022. The goal of this task is to recognize 10 work operations from a large-scale multimodal dataset of the packing process, including data from IMUs, vision sensors, and IoT-enabled devices, etc. It aims to solve the problem of “when and what kind of work operation was performed”. In this report, we explore a fusion based method for addressing the work operation recognition task. In our method, features are first extracted from five modalities and fed to the attention-based fusion module. Finally, recognition results are obtained. Our method achieved an F1-measure of 0.959 on the submission set; it placed second in the OpenPack Challenge 2022.