Content-based sports video analysis and composition
Jinjun Wang · 2008
This thesis proposes solutions for content-based sports video analysis, including multimodal feature extraction, middle-level representation and semantic event detection.In addition, solutions for sports video composition and personalization are also examined.The first part of the thesis describes our methodology to detect semantic events and event boundaries from both broadcast sports video and non-broadcast sports video.Specifically, to process broadcast sports video, our approach not only uses visual/audio features, we also analyze the web-casting text information associated with the video and synchronize it with the visual/audio features to detect event, locate event boundaries and identify involved players/teams; To process non-broadcast sports video, we select the raw unedited main-camera soccer video as the input and use visual, audio and motion features extraction with multi-level modeling to detect event and event boundaries.Our proposed techniques are evaluated using large sports video data set and achieved satisfactory user acceptance score.The second part of the thesis introduces three novel applications based on our proposed sports video analysis techniques.The first application is a live sports highlight generation system which can automatically detect multiple events from a live sports game and extract a suitable video segment for each event to provide live and personalized game viewership via various mediums.The second application attempts to automatically generate broadcast soccer video composition from multiple raw video captures by detecting events from unedited soccer video and mimicking human director's practice to control the replay insertion and camera view switching operations.The third application is a personalized music sports video generation system to automatically select and align desired sports video scenes with music clips to generate music video clips.The three proposed systems are tested using objective evaluations and subjective user studies.