Self-Evaluation of Large Language Model based on Glass-box Features
Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, Tiejun Zhao, Wenpeng Lü · 2024
The proliferation of open-source Large Language Models (LLMs) underscores the pressing need for evaluation methods.Existing works primarily rely on external evaluators, focusing on training and prompting strategies.However, a crucial aspect -model-aware glass-box features -is overlooked.In this study, we explore the utility of glass-box features under the scenario of self-evaluation, namely applying an LLM to evaluate its own output.We investigate various glass-box feature groups and discovered that the softmax distribution serves as a reliable quality indicator for self-evaluation.Experimental results on public benchmarks validate the feasibility of self-evaluation of LLMs using glass-box features 1 .