Knowledge Distillation for Memory-Efficient On-Board Image Classification of Mars Imagery
Piotr Bosowski, Nicolas Longépé, Bertrand Le Saux, Jakub Nalepa · 2023
The amount of data captured in the emerging satellite missions has been continuously growing. Thus, developing resource-efficient predictive models is of paramount importance in an array of onboard space applications, where downlinking the data for further analysis is extremely costly or impossible. In such scenarios, we should extract actionable items on board an edge device, using e.g., a machine learning model. Reducing the model’s complexity which may be significant in deep learning algorithms is not only about fitting a full-size neural net into resource-restrictive hardware, but it may result in decreasing the latency and energy consumption. We tackle this issue and exploit knowledge distillation to elaborate a simpler and memory-efficient version of the large-capacity learner, aiming to preserve the large model quality. The experimental study shows that knowledge distillation may not only improve the classification capability of the original model, but can also dramatically (up to 425×) reduce its size for the on-board classification of Mars imagery.