Assessment of Pre-Trained Large Language Models for Hardware Trojan Detection in RTL Designs
Gwok-Waa Wan, Sam-Zaak Wong, Dongping Liu, Xi Wang · 2024
With the successful application of large language models (LLMs) in hardware design and bug fixing, pre-trained LLMs may offer a potential solution for HT detection on the user side. For this purpose, we have carefully selected and implemented a lightweight HT benchmark for LLM at the RTL level based on Trust-Hub, which we call HTEval-mini, and includes 12 designs. We designed a unified prompt template and selected four LLMs, including the SOTA model Claude3, for multiple iterations pass@10 tests. Among all models, the highest detection accuracy and classification success rate were shown to be 66.67% and 50%, respectively. Finally, We have analyzed the results and provided insights for future development.