CRUFT: Context Recognition under Uncertainty using Fusion and Temporal Learning
Wen Ge, Emmanuel Agu · 2020
Human context recognition (HCR), which involves determining a user's current situation (or context), has long been an important task in context-aware systems. With the widespread ownership of smartphones, HCR methods that utilize signals from its built-in sensors have recently received increased attention. We propose Context Recognition under label Uncertainty using Fusion and Temporal Learning (CRUFT), a novel method to recognize a diverse set of smartphone user contexts, including long-term human activities, short-term human activities, and phone placement (pocket or bag in which the smartphone is carried). Context recognition is formulated as a multi-label classification task. CRUFT uses both handcrafted features and auto-learned deep learning features extracted from raw time-series data in two separate arms. The handcrafted arm includes a Multi-Layer Perceptron (MLP), while the raw data arm utilizes a Convolutional Neural Network (CNN) along with a Bi-Directional Long Short Term Memory (Bi-LSTM) model that exploits temporal correlations in the input stream. As smartphone sensor readings, assigned timestamps, and labels can be wrong sometimes, CRUFT integrates an uncertainty module. CRUFT outperforms the state-of-the-art baselines achieving 94.25% in overall Balanced Accuracy (BA), which improves the best performing baseline by 2.7%. Our detailed analyses demonstrate the non-trivial contributions of each component in CRUFT.