Share your thoughts, 1 month free Claude Pro on usSee more

Compositional Reasoning on MAT-PREF (IID)

65.2Logical Question Accuracy

Qwen3-8B + SFT + GRPO

Updated 1mo ago

Evaluation Results

Method	Links
Qwen3-8B + SFT + GRPO 2026.06		65.2
Qwen3-8B + SFT (SC@8) 2026.06		50.6
Qwen3-8B + SFT 2026.06		44.9
Qwen3-235B-Instruct 2026.06		43.8
Qwen2.5-72B 2026.06		37.1
DeepSeek-V3 2026.06		36
Llama 3.3-70B 2026.06		33.7
Random baseline 2026.06		21.9