Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on GPQA Diamond (Pass@1)

75.7Pass@1

OpenAI-o1-1217

Updated 1mo ago

Evaluation Results

Method	Links
OpenAI-o1-1217 2025.01		75.7
DeepSeek-R1 2025.01		71.5
DeepSeek-R1-Distill-Llama-70B 2025.01		65.2
Claude-3.5-Sonnet-1022 2025.01		65
Claude-3.5-Sonnet-1022 2025.01		65
DeepSeek-R1-Distill-Qwen-32B 2025.01		62.1
OpenAI-o1-mini 2025.01		60
DeepSeek-V3 2025.01		59.1
DeepSeek-R1-Distill-Qwen-14B 2025.01		59.1
GPT-4o-0513 2025.01		49.9
GPT-4o-0513 2025.01		49.9
BF16 2025.12		49.49
DeepSeek-R1-Distill-Qwen-7B 2025.01		49.1
DeepSeek-R1-Distill-Llama-8B 2025.01		49
MixKVQ 2025.12		45.45
KIVI 2025.12		44.94
Low-temperature 2026.01		43
RotateKV 2025.12		42.93
KVQuant 2025.12		41.41
Power Sampling 2026.01		40.9
GRPO (MATH) 2026.01		39.9
KVTuner 2025.12		39.9
MCMC Power Sampling 2026.01		38.9
Power Sampling 2026.01		36.4
GRPO (MATH) 2026.01		35.4
KIVI 2025.12		35.35
Low-temperature 2026.01		35.3
Power Sampling 2026.01		34.9
MCDIFFUSE 2026.02		34.78
MCMC Power Sampling 2026.01		34.5
Best-of-N 2026.01		34.3
DeepSeek-R1-Distill-Qwen-1.5B 2025.01		33.8
Best-of-N 2026.01		33.8
ReFusion 7B 2026.02		33.43
Base 2026.01		33.3
GRPO (MATH) 2026.01		33.3
MCMC Power Sampling 2026.01		31.8
Llama-3.1 8B 2026.02		31.14
ReFusion 7B 2026.02		30.45
Low-temperature 2026.01		30.3
Best-of-N 2026.01		28.2
Base 2026.01		27.8
Base 2026.01		27.8
ReFusion 7B 2026.02		27.63
KVQuant 2025.12		25.76
Dream 7B 2026.02		25.42
Qwen3 8B 2026.02		23.13
LLaDA 8B 2026.02		21.39
Qwen2.5 7B 2026.02		16.94