AIR: Aerial Inspection RetinaNet Maapelastustehtäviin
Pyrrö, Pasi · Aaltodoc (Aalto University) · 2022
Search and rescue (SAR) missions have been carried out for centuries to aid those who are lost or in distress, typically in some remote areas, such as wilderness. With recent advances in technology, small unmanned aerial vehicles or drones have been used during SAR missions for years in many countries. The reason is that drones enable rapid aerial photographing of large areas with potentially difficult-to-reach terrain and can even match several land search parties in efficiency. However, there remains the issue of inspecting a vast amount of aerial drone images for tiny clues about the missing person location, which is currently a manual task done by humans in most cases. It turns out this inspection process is very slow, tedious and error-prone for most people and can significantly delay the entire aerial drone search operation. In this thesis, we propose a novel deep learning based object detection approach to automate this drone footage inspection task. As such, we use a data set called HERIDAL of aerial imagery from Mediterranean landscape to train our detector, and the goal is to outperform existing object detection methods on the HERIDAL test data. Consequently, we experiment with hyperparameter tuning, model architecture selection, online data augmentation, image tiling, confidence score threshold calibration and several other tricks to improve the test performance of our method. Finally, we present Aerial Inspection RetinaNet (AIR) as the outcome of these experiments, which is our solution to this aerial person detection (APD) problem in SAR. Moreover, we demonstrate state-of-the-art results for the AIR detector on the difficult HERIDAL benchmark in terms of both precision (~21 percentage points increase) and speed. In addition, we provide a new formal definition for the APD problem in SAR missions related to the HERIDAL data set. That is, we define a novel evaluation scheme, which ranks detectors in terms of real-world SAR localization requirements, which are much looser than in typical object detection tasks. Moreover, we devise an estimator for average human detection performance via a meta-analysis study, which can be used as an initial baseline for APD method performance. Lastly, we propose a novel bounding box aggregation method for robust, approximate object localization: the merging of overlapping bounding boxes (MOB) algorithm.