MindMatrics: A Framework for Advanced Mental Health Diagnosis Using Multimodal Data Analysis
Swapnil Joshi, Karan Jain, Urvi Joshi, Yash Jain, Sonal Balpande, Rucha Kulkarni · 2025
Many people around the world face mental health challenges, particularly students and professionals. Students often experience poor academic performance, difficulties concentrating, and memory issues, while professionals may face challenges with decision-making, communication, and confidence. Our system addresses these concerns by predicting mental health states like stress, anxiety, and depression through a standardized set of scored questions, with the cumulative score indicating the user's mental health status. The system also provides audio-based emotional state prediction using CNNs and Librosa for feature extraction and processing. Video-based emotional state prediction uses CNNs and OpenCV for video analysis. Additionally, the system includes a diary feature that tracks emotions throughout the day using BERT. At the end of each day, it summarizes the user's emotional state and maintains this data for five consecutive days. Afterward, a detailed report of emotional patterns is generated. The system also keeps a history of test predictions, enabling users to track their mental health trends over time. Personalized recommendations for psychiatrists are provided based on the results and user location. Comprehensive reports can be shared with family or friends for support and awareness, promoting early intervention and personalized mental health care.