An Effective Approach for Violence Detection using Deep Learning and Natural Language Processing
Versha Kumari, Khuhed Memon, Burhan Aslam, Bhawani Shankar Chowdhry · 2023
An effective tool for violence detection is highly demanded to examine the rise in crime rate in today's era. Artificial Intelligence can play a significant role in violence detection and monitoring to tackle various problems of security and safety concerns. This research proposes strategies to incorporate Deep Learning and Natural Language Processing (NLP) to simultaneously detect anomalous objects and scenarios from videos using TensorFlow and aggressive, offensive, and hate speech from an audio channel of surveillance cameras. This research aims to automatically detect violence in real-time from surveillance footage by using TensorFlow custom object detection upon identification of firearms, robbery, fistfights, sexual harassment, and fire in successive images from the video feed. In addition, the audio channel of such surveillance cameras can also be significantly fruitful in detecting hate speech, verbal sexual abuse, and profanity. The proposed system includes an alert mechanism that detects any type of violence and automatically notifies the security administrator, enabling timely intervention to prevent potential damage to society. The developed models can be deployed on any existing surveillance system with next to negligible additional hardware and software resource requirements, thereby making it an efficient, fast, accurate, and economical solution. To train the model, custom datasets were designed for 6 categories in images and 2 categories in speech. The accuracy of the developed system was found to be 84%, with adequate performance under various luminance conditions, including night vision images.