Share your thoughts, 1 month free Claude Pro on usSee more

Mathematical Reasoning on TheoremQA (test)

48.4Accuracy

GPT-4-Turbo (24-04-09)

Updated 1mo ago

Evaluation Results

Method	Links
GPT-4-Turbo (24-04-09) 2024.06		48.4
DART-Math-DSMath-7B (Uniform) 2024.06		32.5
DART-Math-DSMath-7B (Prop2Diff) 2024.06		32.2
Llama3-70B-VRT 2024.06		28.6
DART-Math-Llama3-70B (Prop2Diff) 2024.06		28.2
DeepSeekMath-7B-Instruct 2024.06		28.1
DART-Math-Llama3-70B (Uniform) 2024.06		27.4
DeepSeekMath-7B-VRT 2024.06		27.2
Llama3-70B-ICL 2024.06		27
Llama3-70B-MMIQC 2024.06		23.5
DeepSeekMath-7B-ICL 2024.06		23.5
DeepSeekMath-7B-MMIQC 2024.06		23.4
Llama3-70B-MetaMath 2024.06		21.9
Llama3-8B-ICL 2024.06		19.8
DART-Math-Llama3-8B (Prop2Diff) 2024.06		19.4
DART-Math-Mistral-7B (Prop2Diff) 2024.06		17
Mistral-7B-WizardMath-V1.1 (RL) 2024.06		16.6
DART-Math-Mistral-7B (Uniform) 2024.06		16.4
Mistral-7B-MMIQC 2024.06		16.2
Mistral-7B-VRT 2024.06		16.2
Llama3-8B-MMIQC 2024.06		16.2
DART-Math-Llama3-8B (Uniform) 2024.06		15.4
Llama2-7B-Xwin-Math-V1.1+ 2024.06		15
Llama2-70B-Xwin-Math-V1.1+ 2024.06		14.9
Llama3-8B-VRT 2024.06		14.9
Mistral-7B-ICL 2024.06		14.2
Mistral-7B-MetaMath 2024.06		14
Llama3-8B-MetaMath 2024.06		13.8