Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
Liangming Pan, Alon Albalak, Xinyi Wang, William Yang Wang · 2023
Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems.This paper introduces a novel framework, LOGIC-LM, which integrates LLMs with symbolic solvers to improve logical problem-solving.Our method first utilizes LLMs to translate a natural language problem into a symbolic formulation.Afterward, a deterministic symbolic solver performs inference on the formulated problem.We also introduce a selfrefinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations.We demonstrate LOGIC-LM's effectiveness on five logical reasoning datasets: ProofWriter, PrOntoQA, FOLIO, LogicalDeduction, and AR-LSAT.On average, LOGIC-LM achieves a significant performance boost of 39.2% over using LLM alone with standard prompting and 18.4% over LLM with chain-ofthought prompting.Our findings suggest that LOGIC-LM, by combining LLMs with symbolic logic, offers a promising avenue for faithful logical reasoning.