Fusion: Towards a multispectral object classifier
Alexander Arcombe · 2018
Object classification has made great progress lately and a lot of it has to do with the advances in machine learning, processing power, and deep convolution neural networks. Today a camera can detect and classify objects with high precision using trained neural networks, but there is still room for improvements. Cameras using different spectrum, like visual and thermal, have their strengths and weaknesses in classifying objects. They both provide complimentary and unique data about the object if one should fail to classify an object the other one could. This thesis studies techniques to combine the two data types to improve and get a more robust classifier. A dataset was collected with synchronized visual and thermal images, with various examples, including hard examples for both data types. MobileNetV2 were trained and tested on the dataset with three different techniques to fuse the data types together in the network. The results proved the techniques to produce 15 - 17% better accuracy compared to only using one of the data types. (Less)