Share your thoughts, 1 month free Claude Pro on usSee more

Physical Commonsense Reasoning on PIQA (val)

83Accuracy

Mistral-v0.1

Updated 11d ago

Evaluation Results

Method	Links
Mistral-v0.1 2024.01		83
ChatGPT + Self-consistent chain-of-thought 2024.01		81.7
KVQuant-4b-1% 2024.05		80.74
FP16 2024.05		80.58
KVQuant-4b 2024.05		80.58
CQ-2c8b 2024.05		80.52
DeBERTa-v3-L (CANDLE Distilled) 2024.01		80.3
LLAMA2 2024.01		80.2
LLM-Pruner+Finetune 2025.12		79.7
KVQuant-2b-1% 2024.05		79.65
CQ-4c8b 2024.05		79.49
LLM-Pruner 2025.12		79.4
KVQuant-4b-1% 2024.05		79.27
KVQuant-4b 2024.05		79.22
ChatGPT + Chain-of-thought 2024.01		79.2
LLM-Pruner+Finetune 2025.12		79.2
FP16 2024.05		79.16
FP16 2024.05		79.16
KVQuant-4b-1% 2024.05		79.16
CQ-2c8b 2024.05		79.11
Baseline 2025.12		79.1
DeBERTa-v3-L (MR) 2024.01		79
KVQuant-4b 2024.05		78.94
LLAMA2 2024.01		78.8
FP16 2024.05		78.67
KVQuant-4b 2024.05		78.62
CQ-2c8b 2024.05		78.62
CQ-2c8b 2024.05		78.61
CAR-DeBERTa-v3-L 2024.01		78.6
CAR-DeBERTa-v3-L 2024.01		78.6
LLRC 2025.12		78.6
KVQuant-2b-1% 2024.05		78.51
KVQuant-2b-1% 2024.05		78.51
KVQuant-4b-1% 2024.05		78.4
LLM-Pruner 2025.12		78.3
CQ-4c8b 2024.05		78.29
Baseline 2025.12		78.1
LLRC 2025.12		78.1
LLM-Pruner+Finetune 2025.12		78.1
FP16 2024.05		78.07
KVQuant-4b-1% 2024.05		78.07
DeBERTa-v3-L (MR) 2024.01		78
CQ-2c8b 2024.05		77.91
LLRC 2025.12		77.9
KVQuant-4b 2024.05		77.86
LLM-Pruner+Finetune 2025.12		77.8
KVQuant-2b-1% 2024.05		77.69
VERA-T5-xxl (CANDLE Distilled) 2024.01		77.6
SVD-LLM+Finetune 2025.12		77.6
CQ-4c8b 2024.05		77.42
CQ-8c10b 2024.05		77.31
LLM-Pruner 2025.12		77.3
VERA-T5-xxl 2024.01		77.2
KVQuant-2b-1% 2024.05		76.6
VERA-T5-xxl 2024.01		76.4
SVD-LLM+Finetune 2025.12		76.3
LLM-Pruner 2025.12		76.2
CQ-4c8b 2024.05		76.12
CQ-4c8b 2024.05		76.11
CQ-8c10b 2024.05		75.9
KVQuant-2b 2024.05		75.46
KVQuant-1b-1% 2024.05		75.46
SVD-LLM+Finetune 2025.12		75.3
CQ-8c8b 2024.05		75.24
ChatGPT 2024.01		75.1
VERA-T5-xxl 2024.01		75.1
LLRC 2025.12		75.1
KVQuant-2b 2024.05		74.81
CQ-8c10b 2024.05		74.37
CQ-8c8b 2024.05		73.99
SVD-LLM (w) 2025.12		73.9
KVQuant-1b-1% 2024.05		73.83
CQ-8c10b 2024.05		73.45
CQ-8c10b 2024.05		73.07
CQ-8c8b 2024.05		73.01
GPT-4 2024.01		73
KVQuant-2b 2024.05		72.47
SVD-LLM+Finetune 2025.12		72.4
DeepSeekMoE 2024.01		72.3
KVQuant-1b-1% 2024.05		71.38
CQ-8c8b 2024.05		71.22
CQ-8c8b 2024.05		71.16
STL-Adapter 2024.01		71.1
KVQuant-1b-1% 2024.05		70.89
GShard 2024.01		70.6
Switch 2024.01		70.5
Mistral (Full-Attention) 2024.07		70.4
Mamba (SSM) 2024.07		70.3
Self-talk 2024.01		70.2
BMoJo (Fading) 2024.07		70
KVQuant-1b-1% 2024.05		69.91
BMoJo (Fading + Eidetic) 2024.07		69.9
SVD-LLM (w) 2025.12		69.8
SVD-LLM (w) 2025.12		69.4
SmolLM Ceiling (O(L^2)) 2026.04		68.55
Hash Layer 2024.01		68.4
GPT 3.5 2024.01		67.8
ROBERTa-L 2024.01		67.6
Dense 2024.01		66.8
Hybrid (Sliding Attention + SSM) 2024.07		66.1

Showing 100 of 116 rows