Kitten: A Simple Yet Effective Baseline for Evaluating LLM-Based Compiler Testing Techniques
Yuanmin Xie, Zhenyang Xu, Yongqiang Tian, Min Qi Zhou, X. Zhou, C. P. Sun · 2025
Compiler testing is critical and indispensable to improve the correctness of compilers. Spurred by recent advancements in Large Language Models (LLMs), LLM-based compiler testing techniques such as Fuzz4All, have demonstrated their potential in uncovering real bugs in diverse compilers and reducing the required engineering efforts in designing program generators. Given the continuous evolution of LLMs and the emergence of new LLM-based approaches, establishing robust baselines is crucial for rigorous evaluation and driving future advancements in this promising research direction.