Can ChatGPT Pass Modern Control Theory Exam?
Itsuki Ogo, Masanobu Koga · 2024
Large language models (LLMs), such as GPT models, have been rapidly studied in recent years and are expected to be applied to academic fields such as mathematics and engineering. In this study, we examined how accurately ChatGPT (GPT-4o) can answer modern control theory questions at the undergraduate-level. A set of 98 questions on modern control theory was used to evaluate GPT-4o’s problem-solving ability on modern control theory. The results revealed that the GPT-4o showed a 49.0% correct response rate to the undergraduate-level modern control theory exercises, and that the correct response rate tended to be lower for problems involving calculations, especially those that require step-by-step thinking and complex computation. This may be attributed to the Transformer architecture of the GPT model, which generates answers based on probabilistic predictions. In this study, we proposed a method to improve response accuracy by developing a customized GPT which leverages prompt engineering methods to address these issues. In order to evaluate the proposed method, a question set consisting of 45 graduate school entrance exam questions on modern control theory was developed. The results of the evaluation showed that the correct response rate was improved by 26.6 points, yielding a 64.4% correct response rate.