Video Segmentation Pipeline For Co-Creative AI Dance Application
John Hakan Can Gunerli, Manoj Deshpande, Brian Magerko · 2024
This article introduces a method that combines human input and computation for analyzing human motion from video recordings, specifically for capturing dance movements. The central aim is to develop an innovative system for processing and analyzing videos. This system consists of four key stages: using pre-trained MediaPipe models for interactive image segmentation, organizing videos efficiently through batching, identifying and extracting keyframes, and pinpointing accurate timestamps of keyframes. This pipeline is a part of LuminAI, an interactive installation that features a virtual AI agent capable of improvising movements in collaboration with human participants. In particular, the proposed pipeline fits into the first software module of LuminAI, responsible for recognizing and segmenting continuous motion capture data into separate body actions. The proposed video segmentation pipeline is designed to fulfill the requirements for both qualitative and quantitative analyses in designing systems that classify human movements. This research advances our knowledge of human motion through video analysis and bridges the gap between technology and artistic expression.