Interactive Event Sifting using Bayesian Graph Neural Networks
José Nascimento, Nathan Jacobs, Anderson De Rezende Rocha · 2024
Forensic analysts often use social media imagery and texts to understand important events. A primary challengeis the initial sifting of irrelevant posts. This work introduces an interactive process for training an event-centric, learningbased multimodal classification model that automates sanitization. We propose a method based on Bayesian Graph Neural Networks (BGNNs) and evaluate active learning and pseudolabeling formulations to reduce the number of posts the analyst must manually annotate. Our results indicate that BGNNs are useful for social-media data sifting for forensics investigations of events of interest, the value of active learning and pseudo-labeling varies basedon the setting, and incorporating unlabelled data from other events improves performance.