Our new X account is live! Follow @wizwand_team for updates

Question Answering on ARC Challenge (val)

89.6Accuracy

LoRA

Updated 4d ago

Evaluation Results

Method	Links
LoRA 2026.02		89.6
D2-LoRA 2026.02		89.6
DoRA 2026.02		88.3
D2-LoRA 2026.02		73.9
LoRA 2026.02		72.6
DoRA 2026.02		72.2
G-DAUG 2020.04		51.5
G-DAUG 2020.04		50.8
G-DAUG 2020.04		50.8
FP16 2024.05		50.34
KVQuant-4b-1% 2024.05		49.91
G-DAUG 2020.04		49.5
CQ-2c8b 2024.05		49.15
KVQuant-4b 2024.05		49.06
FP16 2024.05		48.29
KVQuant-4b-1% 2024.05		47.87
CQ-2c8b 2024.05		47.78
KVQuant-2b-1% 2024.05		47.53
KVQuant-4b-1% 2024.05		46.76
KVQuant-4b 2024.05		46.67
FP16 2024.05		46.42
KVQuant-4b 2024.05		45.99
CQ-2c8b 2024.05		45.99
CQ-4c8b 2024.05		45.65
KVQuant-2b-1% 2024.05		45.14
KVQuant-2b-1% 2024.05		44.97
CQ-4c8b 2024.05		44.11
CQ-4c8b 2024.05		44.03
ROBERTa 2020.04		43.5
FP16 2024.05		43.43
CQ-2c8b 2024.05		43.34
KVQuant-4b-1% 2024.05		43.17
Backtranslation 2020.04		43.1
KVQuant-4b 2024.05		42.75
KVQuant-4b 2024.05		42.32
FP16 2024.05		41.72
CQ-2c8b 2024.05		41.55
KVQuant-2b-1% 2024.05		41.47
KVQuant-4b-1% 2024.05		41.38
CQ-4c8b 2024.05		39.93
CQ-8c10b 2024.05		39.59
KVQuant-2b-1% 2024.05		38.74
CQ-8c10b 2024.05		38.74
KVQuant-2b 2024.05		38.57
CQ-4c8b 2024.05		38.48
CQ-8c10b 2024.05		37.12
KVQuant-1b-1% 2024.05		35.32
CQ-8c10b 2024.05		34.64
KVQuant-2b 2024.05		34.47
DeepSeekMoE 2024.01		34.3
CQ-8c8b 2024.05		34.3
CQ-8c8b 2024.05		33.79
CQ-8c8b 2024.05		33.79
CQ-8c10b 2024.05		33.28
KVQuant-1b-1% 2024.05		33.19
KVQuant-1b-1% 2024.05		32.59
KVQuant-2b 2024.05		32
GShard 2024.01		31.6
KVQuant-1b-1% 2024.05		31.48
Switch 2024.01		30.2
CQ-8c8b 2024.05		30.2
CQ-8c8b 2024.05		30.2
KVQuant-1b-1% 2024.05		29.69
Hash Layer 2024.01		28.2
Dense 2024.01		26
KVQuant-2b 2024.05		24.66
KVQuant-2b 2024.05		22.44
KVQuant-1b 2024.05		21.76
KVQuant-1b 2024.05		21.67
KVQuant-1b 2024.05		21.33
KVQuant-1b 2024.05		20.65
KVQuant-1b 2024.05		19.88