Towards a joint semantic analysis in mobile forensics environments
Jian Xi, Melanie Siegel, Dirk Labudde, Michael Spranger · Forensic Science International Digital Investigation · 2024
In recent years, mobile devices have become the dominant communication medium in our daily lives. This trend is also evident in the planning, arranging, and committing of criminal activities, particularly in organized crime. Accordingly, mobile devices have become an essential source of evidence for data analysts or investigators, especially in Law Enforcement Agencies (LEAs). However, communication via mobile devices generates vast amounts of data, rendering manual analysis impractical and resulting in growing backlogs of evidence awaiting analysis process, which can take months to years, thereby hindering investigations and trials. The automatic analysis of textual chat messages falls short because communication is not limited to the single modality, such as text, but instead spans multiple modalities, including voice messages, pictures, videos, and sometimes various messengers (channels). These modalities frequently overlap or interchange within the same communication, further complicating the analysis process. To achieve a correct and comprehensive understanding of such communication, it is essential to consider all modalities and channels through a consistent joint semantic analysis. This paper introduces a novel mobile forensics approach that enables efficient assessment of mobile data without losing semantic consistency by unifying semantic concepts across different modalities and channels. Additionally, a knowledge-guided topic modeling approach is proposed, integrating expertise into the investigation process to effectively examine large volumes of noisy mobile data. In this way, investigators can quickly identify evidentiary parts of the communication and completely facilitate reconstructing the course of events. • The joint semantic analysis considers the various communication channels involved in a conversation. • The joint semantic analysis considers the different modalities used during a conversation. • Knowledge-driven topic modeling tailored to forensic communication analysis to deal with noisy data. • More efficient understanding of data through error-tolerant mapping of mobile communication data to semantic concepts.