Does Metacognitive Prompting Improve Causal Inference in Large Language Models?
Ryusei Ohtani, Yuko Sakurai, Satoshi Oyama · 2024
Causal inference with large language models (LLMs) is an important research topic with various applications. In this study, we examine whether metacognitive prompting, which has recently been reported to be effective for other tasks and promotes deeper insight into LLMs, improves causal inference in LLMs. We examined the effectiveness of metacog- nitive prompting in causal reasoning, focusing on the problem of determining sufficient causes of a causal relationship, which has been noted to be particularly challenging for LLMs. Our results showed that metacognitive prompting was not necessarily effective for these tasks. We found that metacognitive prompting does not necessarily make LLMs perform deep insights and that they may only pretend to perform deep insights.