INTEGRATION OFSEGMENTATION,TRACKING AND CLASSIFICATION MODELS TO SOLVE VIDEO ANALYTICS PROBLEMS
А.Е. Arkhipov, Ivan Fomin, V. Matveev · Известия Южного федерального университета. Технические науки · 2024
The integration of several models into one technical vision system will allow solving morecomplex tasks. In particular, for mobile robotics and unmanned aerial vehicles (UAVs), the lack ofdata sets for various conditions is an urgent problem. In the work, the integration of several modelsis proposed as a solution to this problem: segmentation, maintenance and classification. The segmentationmodel allows you to select arbitrary objects from frames, which allows it to be used in nondeterministicand dynamic environments. The classification model allows you to determine the objects necessary for navigation or other use, which are then accompanied by a third model. The paperdescribes an algorithm for model aggregation. In addition to models, the key element is the correctionof model predictions, which allows you to segment and accompany various objects reliablyenough. The procedure for correcting model predictions solves the following tasks: adding new objectsto accompany, validating segmented object masks and clarifying the associated masks. Theversatility of this solution is confirmed by working in difficult conditions, for example, underwaterphotography or images from UAVs. An experimental study of each of the models was carried out inan open area and indoors. The data sets used make it possible to assess the applicability of modelsfor mobile robotics tasks, that is, to identify possible obstacles in the robot's path, for example, acurb, as well as moving objects such as a person or a car. They demonstrated a sufficiently highquality of work. For most classes, the indicators exceeded 80% by various metrics. The main errorsare related to the size of the objects. The conducted experiments clearly demonstrate the versatility ofthis solution without additional training of models. Additionally, a study of performance on a personalcomputer with various input parameters and resolution was conducted. Increasing the number ofmodels significantly increases the computational load and does not reach real time. Therefore, one ofthe directions of further research is to increase the speed of the system