Data Pipelines for Real-Time, Custom Object Detection and Tracking in League of Legends
Dan Mutsvanga, Arman Petrosyants, Dmitry Nikolaev, Julia Orlova, Anton Stepanov, Roberto Passerone, Andrey G. Somov · 2024
In this work, we present a method for automatically generating a synthetic, labelled computer vision dataset of game-character positions for LoL and the application of this synthetic data for training a deep-learning, single-shot object detection model that can then be applied in pseudo labelling and generating detailed player position data from LoL video streams. Further-more, we investigate the effects of data augmentation and class balancing on the overall performance of the neural network, comparing them to the base network trained on a manually labelled dataset. Using synthetically generated datasets of 10000 images containing 21 object classes, we achieved a mean average precision (mAP@50) of 64.8% and an IoU of 67.7%.