Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning

About

Mathematical reasoning remains a significant challenge for large language models (LLMs), despite progress in prompting techniques such as Chain-of-Thought (CoT). We present **Chain of Mathematically Annotated Thought (CoMAT)**, which enhances reasoning through two stages: *Symbolic Conversion* (converting natural language queries into symbolic form) and *Reasoning Execution* (deriving answers from symbolic representations). CoMAT operates entirely with a single LLM and without external solvers. Across four LLMs, CoMAT outperforms traditional CoT on six out of seven benchmarks, achieving gains of 4.48% on MMLU-Redux (MATH) and 4.58% on GaoKao MCQ. In addition to improved performance, CoMAT ensures faithfulness and verifiability, offering a transparent reasoning process for complex mathematical tasks

Joshua Ong Jun Leang, Aryo Pradipta Gema, Shay B. Cohen• 2024

Related benchmarks

TaskDatasetResultRank
Mathematical ReasoningGSM8K
Accuracy83.9
983
Symbolic ReasoningAQUA
Accuracy72.4
26
Symbolic ReasoningOlyBench
Accuracy32.2
25
Symbolic ReasoningMMLU-Redux
Accuracy79.8
25
Showing 4 of 4 rows

Other info

Follow for update