Share your thoughts, 1 month free Claude Pro on usSee more

Social Commonsense Reasoning on SIQA

86.9Accuracy

Human

Updated 20d ago

Evaluation Results

Method	Links
Human 2026.03		86.9
In-Squeeze 2026.02		85.24
In-Squeeze 2026.02		84.41
Direct Fine-tuning 2026.02		84.25
Cont-Squeeze 2026.02		84.25
Direct Fine-tuning 2026.02		83.68
Cont-Squeeze 2026.02		83.68
Direct Fine-tuning 2026.02		83.16
DeBERTa-v3-L (Supervised) 2026.03		80.1
RoBERTa-L (Supervised) 2026.03		76.6
Qwen3-4B 2025.12		75.59
S-MeZO 2024.02		70.2
ChatGPT (gpt-3.5-turbo) 2026.03		69.7
Qwen2.5-3B 2025.12		69.4
IMAGINE-DeBERTa-v3-L 2026.03		69
IMAGINE-DeBERTa-v3-L (Retrieval) 2026.03		69
Qwen3-1.7B 2025.12		68.58
MeZO 2024.02		68.5
GPT-3.5 (text-davinci-003) 2026.03		68
Zero-shot Fusion 2026.03		66.6
IMAGINE-DeBERTa-v3-L 2026.03		66.3
gemma2-2B 2025.12		65.92
CANDLE-DeBERTa-v3-L 2026.03		65.9
SmolLM3-3B 2025.12		65.25
Qwen2.5-1.5B 2025.12		64.94
llama-3.2-3B 2025.12		64.33
IMAGINE-RoBERTa-L 2026.03		64.3
CAR-RoBERTa-L 2026.03		64
CAR-DeBERTa-v3-L 2026.03		64
Qwen2-1.5B 2025.12		63.46
YuLan-Mini-2.4B 2025.12		63.25
RoBERTa-L (MR) 2026.03		63.1
PCMind-2.1-Kaiyuan-2B 2025.12		62.59
DeBERTa-v3-L (MR) 2026.03		62.1
Qwen3-0.6B 2025.12		61.51
SmolLM2-1.7B 2025.12		60.18
CANDLE-VERA-T5-xxl 2026.03		59.4
VERA-T5-xxl 2026.03		58.2
VERA-T5-xxl 2026.03		58.1
GPT-4 (gpt-4) 2026.03		57
MICO 2026.03		56
OLMo-2-0425-1B 2025.12		55.53
GPT-2-L (MR) 2026.03		53.6
IMAGINE-GPT-2-L 2026.03		53
CAR-GPT-2-L 2026.03		52.3
llama-3.2-1B 2025.12		50.61
LLAMA2-13B 2026.03		50.3
COMET-DynGen 2026.03		50.1
DeBERTa-v3-L 2026.03		47.8
RoBERTa-L 2026.03		47.3
AMALIA-9B-DPO 2026.03		46.3
Self-talk 2026.03		46.2
Salamandra-7B 2026.03		44.8
Gervasio-8B 2026.03		44.8
AMALIA-9B-SFT 2026.03		44.7
GPT-2-L 2026.03		44.6
Gemma 3-12B 2026.03		43.8
Llama 3.1-8B 2026.03		43.4
Apertus-8B 2026.03		43.2
PLDRv51-SOC-110M-5 2026.03		43.09
Mistral-v0.1-7B 2026.03		42.9
Ministral-8B 2026.03		42.3
PLDRv51-SOC-110M-3 2026.03		42.17
InstructBLIP-Vicuna-7B 2026.03		42.1
GPT-Neo-125M 2026.03		42.07
PLDRv51-SOC-110M-2 2026.03		41.91
OLMo 2-7B 2026.03		41.9
PLDRv51-SOC-110M-1 2026.03		41.66
PLDRv51-SOC-110M-4 2026.03		41.66
ShapeLLM-Omni-7B 2025.12		41.5
CoRe3D 2025.12		41.5
Qwen 3-8B 2026.03		41.2
Qwen2.5-vl-7B 2025.12		41
Gemma 2-9B 2026.03		41
Mistral-7B 2026.03		40.9
EuroLLM-9B 2026.03		40.8
ABL-SOC-110M-1 2026.03		40.79
Qwen 2.5-7B 2026.03		40.7
LLaMA3.2-Vision-11B 2025.12		40.6
LLaMA-Mesh-8B 2025.12		40.3
DLM + extra token 2026.01		37.92
SUB-SOC-110M-1 2026.03		36.8
ABL-SOC-110M-2 2026.03		36.8
ABL-SOC-110M-3 2026.03		36.54
DLM 2026.01		35.98
SUB-SOC-110M-2 2026.03		35.57
LLaVA-1.5-7B 2026.03		34.8
DLM + GA 2026.01		34.44
Cont-Squeeze 2026.02		34