ViT and RNN for Temporal and Spatial Analysis in Video Sequences
Asha Rani Borah, Ali Abdulhussein Hameed, H Pal Thethi, Javvadi Lakshmi Prasanna, A Sangeetha, Desh Deepak Gautam · 2025
This study proposes a novel technique to view dance routines in video clips using Vision Transformers (Vi T) and Recurrent Neural Networks (RNN). The three main algorithms in this new technique are dancing move recognition, temporal dependency modeling, and emotion guessing. Temporal Dependency Modeling employs RNNs to track the sequence of known dance moves across time, whereas Dance Move Recognition uses Vi T to identify dance motions in a video clip. Emotion Inference uses Vi T and RNN to recognize emotions in body language and facial expressions. These principles provide a firm foundation for video sequences comprehension and enjoyment. Our proposed dance routine analysis approach was compared to well-known methods. The evidence shows that the proposed method performs better in several aspects. It recognizes dancing motions 94.5% of the time, better than existing approaches. Our technique also scores 92.3% on the completeness indicator, which measures dance routine maintenance. Our 91.8% timing accuracy shows that our approach can grasp dance timing. The proposed approach scores 88.7% on the expressiveness test, demonstrating its understanding of dance creativity. Real-world apps need real-time processing performance, and our technology tops the competition at 30 FPS. Plus, it uses memory effectively, requiring only 2.5 GB RAM. Our recommended approach excels in emotion recognition. It outperforms other emotion detection systems with 92.1% accuracy. F1 scores 92.2% with 91.3% accuracy and 93.2% memory. Although it works better, the recommended solution consumes just 2.7 GB of RAM and processes at 28 frames per second in real time. Our dance routine analysis is cutting-edge. Combining Vi T with RNN improves accuracy, completeness, timeliness, and expressiveness. It also recognizes emotions swiftly and with minimal memory. This study enhances our appreciation of dancing. They may also be used to teach dancing, judge performances, and communicate emotional stories.