Exploiting Geodata to Improve Image Recognition with Deep Learning
Christian Arbinger, Martin Bullin, Andreas Henrich · Companion Proceedings of the Web Conference 2022 · 2022
Due to the widespread availability of smartphones and digital cameras with GPS functionality, the number of photos associated with geographic coordinates or geoinformation on the internet is continuously increasing. Besides the obvious benefits of geotagged images for the users, geodata can enable a better understanding of the image content and thus facilitate their classification. This work shows the added value of integrating auxiliary geodata during a multi class single label image classification task. Various ways of encoding and extracting auxiliary features from raw coordinates are compared, followed by an investigation of approaches to integrate these features into a convolutional neural network (CNN) by fusion models. We show the classification improvements of adding the raw coordinates and derived auxiliary features such as satellite photos and location-related texts (address information and tags).