Ablation study: Calibrated adaptive learning ensemble methodology
Gading Aditya Perdana, Ignatius Ivan Wijaya, Kevin Ahmad Fahreza, Gabriel Asael Tarigan · Procedia Computer Science · 2025
We conduct a systematic ablation study of the Calibrated Adaptive Learning via Mutual-Ensemble Fusion (CALM) framework to determine how teacher ensemble size influences student accuracy, calibration and computational cost. Using ensembles of two to five teachers drawn from diverse convolutional architectures, we evaluate performance on CIFAR-10 under four CALM configurations. Our results show that calibration-aware training (CAL) yields the lowest expected calibration error while adaptive curriculum pacing (ACP) delivers exceptional calibration in small ensembles. The full ACP CAL setup achieves peak accuracy but introduces a trade-off with calibration quality. We observe diminishing returns beyond three teachers and identify optimal configurations that balance accuracy, uncertainty estimation and resource usage. These insights inform practical guidelines for deploying multi-teacher distillation in environments where model confidence and efficiency are critical.