Causal Reasoning on XCOPA

94.4Accuracy

PaLM 2

Updated 5d ago

Evaluation Results

Method	Links
PaLM 2 2023.05		94.4
PaLM + CoT 2023.05		89.9
BYOL-nya 2026.01		71.2
BYOL-nya 2026.01		70
Apertus-8B 2026.05		65.69
QuaSAR 2025.02		65
Apertus-v1.1-4B 2026.05		63.82
HQ seed 2026.04		62
Qwen3-4B-Base 2026.05		61.82
ML (15%) 2026.04		61.6
BYOL-nya 2026.01		61.2
Gemma-3 2026.01		60.8
ML 2026.04		60.8
ML (Q3) 2026.04		60.2
Apertus-v1.1-1.5B 2026.05		59.76
MKC-e 2026.04		59.4
HQ 2026.04		59.2
No filtering 2026.04		58.6
Qwen3-1.7B-Base 2026.05		58.35
SmolLM-3B-Base 2026.05		58.02
Gemma-3 2026.01		57.2
CoT 2025.02		56.9
baseline 2025.02		56.4
DMoE 2025.06		56
ACROS 2026.05		55.8
EuroLLM-1.7B 2026.05		55.76
DMoE 2025.06		55.7
Branch-Train-Mix 2025.06		55.6
Branch-Train-Mix 2025.06		55.5
Apertus-v1.1-0.5B 2026.05		55.49
BLOOM + Continued Pre-training 2025.06		55.3
BLOOM 2025.06		55.2
BLOOM 2025.06		55.1
BLOOM + Continued Pre-training 2025.06		55
Qwen3-0.6B-Base 2026.05		54.96
Gemma 2026.05		54.8
DMoE 2025.06		54.7
DMoE 2025.06		54.4
Branch-Train-Mix 2025.06		54.1
BLOOM 2025.06		53.9
Branch-Train-Mix 2025.06		53.9
BLOOM + Continued Pre-training 2025.06		53.8
BLOOM + Continued Pre-training 2025.06		53.6
SmolLM2-1.7B 2026.05		53.51
BLOOM 2025.06		53.4
Apertus 2026.01		53.2
Qwen2 2026.05		53.2
Frozen BP 2026.05		53
Qwen-3 2026.01		52.2
Qwen-3 2026.01		52
SmolLM2 2026.05		52
Gemma-3 2026.01		51.8
Llama-3.1 2026.01		51.2
Qwen-3 2026.01		51
Llama-3.2 2026.01		48.6