Decision level fusion experiments on MWIR, VNIR, and SAR imagery
Jacob W. Ross, Shaun Stephens, Patrick Bischof, Adam R. Nolan, Matthew D. Scherreik · 2023
Decision level fusion algorithms combine separate classification scores of a test sample to make a unified class declaration. The aim of decision level fusion algorithms is to achieve better classification performance by combining decisions rather than picking the single best performing algorithm. Multi-modal fusion can be achieved by fusing scores of deep learning models trained on different sensing modalities and tested on a target imaged from co-located sensors. Given differences in phenomenology, fusing EO sensors with SAR may boost performance when extended operating conditions are detrimental to the performance of one modality over the other. The EO modalities discussed in this work (VNIR and MWIR) are susceptible to the time of day while SAR is robust to time of day. Conversely, SAR returns of a target can vary greatly when aspect angle changes while EO modalities are relatively robust. This work analyzes the effectiveness of decision level fusion algorithms on MWIR, VNIR, and SAR modalities given disparate times of day and collection aspects.