SHSML: A Stochastic Approach to Hierarchically Structured Meta-Learning for Improved Inference and Confidence
Zhuoran Li, Xuefeng Chen, Liang Feng, Zhou Wu, Xin Xu · 2024
Meta-learning aims to train models that can learn from a variety of related tasks and use that learned knowledge to solve new and unseen tasks more efficiently. However, it often suffers from handling a sequence of tasks originated from different distributions. To address this issue, the hierarchically structured meta-learning (HSML) has been proposed in the literature. The HSML utilizes a hierarchically structured cluster to address task relationship and similarity, which is regarded as transferable knowledge. Despite the success enjoyed by the HSML, it is worth noting that task uncertainty is ignored by HSML in inference on new tasks by point-estimate manner, which could lead to overconfidence on inappropriate inference results. Taking this cue, in this paper, we propose a stochastic HSML (SHSML) algorithm, which extends HSML with uncertainty awareness by representing each task-specific model as a stochastic variable. By sampling multiple task-specific models and ensembling their inference results instead of point-estimation of HSML, the SHSML is able to mitigate the overconfidence problem in HSML and gives a confidence range of inference. To evaluate the performance of the proposed approach, comprehensive empirical studies are conducted on common curve regression task against state-of-the-art meta-learning algorithms. The obtained results confirmed the efficacy of the proposed approach in handling both task uncertainty and heterogeneity in meta-learning.