Share your thoughts, 1 month free Claude Pro on usSee more

Reasoning on STB24

83.3Exact Match (EM)

STAR (Claude Sonnet 4.6)

Updated 22d ago

Evaluation Results

Method	Links
STAR (Claude Sonnet 4.6) 2026.05		83.3
STAR (Qwen3-8B) 2026.05		76.3
STAR 2026.05		76.3
STAR (Claude Haiku 4.5) 2026.05		76.1
STAR (GPT-OSS-20B) 2026.05		71.8
STAR (Ministral-3-8B) 2026.05		66.8
GPT-OSS-20B 2026.05		51.4
STAR (GLM-4-9B) 2026.05		48.4
STAR (Llama-3.1-8B) 2026.05		48.2
Claude Sonnet 4.6 2026.05		45.2
STAR (Llama-3.2-3B) 2026.05		43.1
Reflexion 2026.05		37.2
GLM-4-9B 2026.05		37
LLM-only (CoT) 2026.05		34.2
Ministral-3-8B 2026.05		32.1
Qwen3-8B 2026.05		31.9
LLM-only 2026.05		31.9
ReAct 2026.05		31.8
Llama-3.1-8B 2026.05		31.5
Claude Haiku 4.5 2026.05		28.7
Llama-3.2-3B 2026.05		22.3
Function-calling 2026.05		20.8
Tree-of-Thought 2026.05		11.8
Graph-of-Thought 2026.05		10.2