Share your thoughts, 1 month free Claude Pro on usSee more

Commonsense Reasoning on WinoGrande (val)

73.88Accuracy

FP16

Updated 1mo ago

Evaluation Results

Method	Links
FP16 2024.05		73.88
KVQuant-4b 2024.05		73.88
KVQuant-4b-1% 2024.05		73.72
CQ-2c8b 2024.05		73.48
KVQuant-4b-1% 2024.05		73.4
FP16 2024.05		72.69
KVQuant-4b 2024.05		72.61
CQ-2c8b 2024.05		72.53
CQ-2c8b 2024.05		72.45
KVQuant-4b-1% 2024.05		72.3
FP16 2024.05		71.98
KVQuant-4b 2024.05		71.59
KVQuant-2b-1% 2024.05		71.43
KVQuant-2b-1% 2024.05		70.8
KVQuant-4b-1% 2024.05		70.72
CQ-4c8b 2024.05		70.72
CQ-2c8b 2024.05		70.4
KVQuant-2b-1% 2024.05		70.17
FP16 2024.05		69.93
KVQuant-4b 2024.05		69.53
CQ-4c8b 2024.05		69.38
CQ-4c8b 2024.05		69.06
FP16 2024.05		68.9
Qwen-1.5 14B (Teacher) 2024.07		68.74
KVQuant-4b-1% 2024.05		68.67
CQ-2c8b 2024.05		68.27
KVQuant-2b-1% 2024.05		68.03
KVQuant-4b 2024.05		67.96
KVQuant-2b-1% 2024.05		67.64
CQ-4c8b 2024.05		67.48
CQ-4c8b 2024.05		66.45
CQ-8c10b 2024.05		65.27
D2-LoRA 2026.02		64
CQ-8c10b 2024.05		63.93
LLaMA2-13B 2024.07		63.77
LoRA 2026.02		63.6
KVQuant-2b 2024.05		63.46
D2-LoRA 2026.02		63
CQ-8c10b 2024.05		62.98
DoRA 2026.02		62.6
DoRA 2026.02		61.6
CQ-8c8b 2024.05		61.56
KVQuant-1b-1% 2024.05		61.01
CQ-8c10b 2024.05		60.46
KVQuant-2b 2024.05		59.35
LoRA 2026.02		59.2
CQ-8c10b 2024.05		59.19
Qwen-1.5 1.8B + DDK 2024.07		59.1
CQ-8c8b 2024.05		58.25
KVQuant-1b-1% 2024.05		58.17
Qwen-1.5 1.8B + KD 2024.07		58.01
Qwen-1.5 1.8B (Student) 2024.07		57.85
KVQuant-1b-1% 2024.05		57.77
TinyLLaMA-1.1B + DDK 2024.07		57.62
Qwen-1.5 1.8B + TED 2024.07		57.38
KVQuant-1b-1% 2024.05		57.3
CQ-8c8b 2024.05		57.14
Qwen-1.5 1.8B + MiniLLM 2024.07		57.14
Qwen-1.5 1.8B + CPT 2024.07		56.98
Qwen-1.5 1.8B + CPT & DoReMi 2024.07		56.75
KVQuant-1b-1% 2024.05		56.67
CQ-8c8b 2024.05		56.51
TinyLLaMA-1.1B + CPT 2024.07		56.2
TinyLLaMA-1.1B + TED 2024.07		55.72
TinyLLaMA-1.1B 2024.07		55.49
TinyLLaMA-1.1B + CPT & DoReMi 2024.07		55.25
CQ-8c8b 2024.05		55.01
TinyLLaMA-1.1B + MiniLLM 2024.07		54.46
Mistral (Full-Attention) 2024.07		54.4
TinyLLaMA-1.1B + KD 2024.07		53.91
KVQuant-2b 2024.05		53.59
BMoJo (Fading) 2024.07		53.3
Hybrid (Sliding Attention + SSM) 2024.07		52.6
BMoJo (Fading + Eidetic) 2024.07		52.1
Hybrid (Sliding Attention + SSM) 2024.07		51.9
Mamba (SSM) 2024.07		51.9
BMoJo (Fading) 2024.07		51.8
KVQuant-2b 2024.05		51.7
Mamba (SSM) 2024.07		51.7
KVQuant-2b 2024.05		51.3
KVQuant-1b 2024.05		50.91
BMoJo (Fading + Eidetic) 2024.07		50.7
KVQuant-1b 2024.05		50.51
Mistral (Full-Attention) 2024.07		50.4
KVQuant-1b 2024.05		49.8
KVQuant-1b 2024.05		49.41
KVQuant-1b 2024.05		48.46