Share your thoughts, 1 month free Claude Pro on usSee more

Compositional Reasoning on MAT-PREF OOD-property

60Logical Question Accuracy

Qwen3-8B + SFT + GRPO

Updated 1mo ago

Evaluation Results

Method	Links
Qwen3-8B + SFT + GRPO 2026.06		60
Qwen3-235B-Instruct 2026.06		53.3
Qwen3-8B + SFT (SC@8) 2026.06		51.8
DeepSeek-V3 2026.06		50.6
Qwen3-8B + SFT 2026.06		50.2
Llama 3.3-70B 2026.06		46.3
Qwen2.5-72B 2026.06		39.6
Random baseline 2026.06		23.4