Share your thoughts, 1 month free Claude Pro on usSee more

Language Understanding on MMLU (test)

88MMLU Average Accuracy

GPT 4o

Updated 4d ago

Evaluation Results

Method
GPT 4o 2024.10	88	-	-	-	-	-	-	-
adv-ICL 2023.12	74	61.3	76.7	82.3	75.8	-	-	-
Original 2026.04	74	70	-	-	-	-	-	-
FPFT 2026.04	74	70	-	-	-	-	-	-
H-CoT 2026.04	74	70	-	-	-	-	-	-
AutoRAN 2026.04	74	70	-	-	-	-	-	-
Chain-of-Lure 2026.04	74	70	-	-	-	-	-	-
FLAN-PALM 540B 2022.05	73.5	-	-	-	-	-	-	-
LRFT 2026.04	73	70	-	-	-	-	-	-
CoT Hijacking 2026.04	73	70	-	-	-	-	-	-
TSBH (Pass@1) 2026.04	73	69	-	-	-	-	-	-
TSBH (Pass@5) 2026.04	73	69	-	-	-	-	-	-
APO 2023.12	72.1	58.3	75.6	80.7	73.9	-	-	-
Vanilla 2025.05	72.06	-	-	-	-	-	-	-
CAA 2025.05	71.88	-	-	-	-	-	-	-
SAEAXBENCH 2025.05	71.77	-	-	-	-	-	-	-
STA 2025.05	71.74	-	-	-	-	-	-	-
GPS 2023.12	71.7	57.9	74.6	80	74.3	-	-	-
Prompt auto 2025.05	71.59	-	-	-	-	-	-	-
Few-shot 2023.12	71	57.5	73.9	79.2	73.5	-	-	-
VITA 2024.10	71	-	-	-	-	-	-	-
ROUGE-L 2023.12	70.8	56.7	74.2	78.4	73.9	-	-	-
Perplexity 2023.12	70.5	56.3	74.8	79.6	71.2	-	-	-
Qwen2-7B 2024.07	70.3	-	-	-	-	-	-	-
Llama3-Instruct 2024.10	67.1	-	-	-	-	-	-	-
Llama-3-8B 2024.07	66.6	-	-	-	-	-	-	-
Baichuan-omni 2024.10	65.3	-	-	-	-	-	-	-
Gemma-7B 2024.07	64.6	-	-	-	-	-	-	-
Mistral-7B 2024.07	64.2	-	-	-	-	-	-	-
Mistral 2024.02	62.4	-	-	-	-	-	-	-
GCAI 2026.01	62.38	53.22	56	73.45	70.39	-	-	-
ICAI 2026.01	62.05	53.28	55.07	73.38	70.29	-	-	-
Qwen1.5-Chat 2024.10	61.5	-	-	-	-	-	-	-
Qwen1.5-7B 2024.07	61	-	-	-	-	-	-	-
Qwen3-1.7B-Base 2026.02	60.4	-	-	-	-	-	0	-
Locas-GLU 2026.02	60.3	-	-	-	-	-	-0.1	-
Locas-GLU 2026.02	60.2	-	-	-	-	-	-0.2	-
Original 2026.04	60	49	-	-	-	-	-	-
TempLoRA 2026.02	59.8	-	-	-	-	-	-0.6	-
FLAN-PALM 62B 2022.05	59.6	-	-	-	-	-	-	-
TempLoRA 2026.02	59.2	-	-	-	-	-	-1.2	-
FPFT 2026.04	59	49	-	-	-	-	-	-
LRFT 2026.04	59	50	-	-	-	-	-	-
H-CoT 2026.04	59	50	-	-	-	-	-	-
AutoRAN 2026.04	59	50	-	-	-	-	-	-
Chain-of-Lure 2026.04	59	49	-	-	-	-	-	-
CoT Hijacking 2026.04	59	49	-	-	-	-	-	-
TSBH (Pass@1) 2026.04	59	50	-	-	-	-	-	-
TSBH (Pass@5) 2026.04	59	50	-	-	-	-	-	-
LLM-QAT 2023.05	58.5	47.7	56.5	66.9	64.2	-	-	-
SmoothQuant 2023.05	58.2	46.7	56	67.5	63.8	-	-	-
MAP-Neo 2024.10	58.2	-	-	-	-	-	-	-
FLAN-UL2 20B 2022.05	58.1	-	-	-	-	-	-	-
adv-ICL 2023.12	58.1	44.1	58.9	64.8	64.5	-	-	-
SmoothQuant 2023.05	58	46	56	67.3	64.1	-	-	-
LLaMA-30B 2023.05	57.8	46	55.8	66.7	63.4	-	-	-
RTN 2023.05	57.8	45.6	56.3	66.8	63.7	-	-	-
RTN 2023.05	57.5	45.8	55.6	66.3	63.4	-	-	-
LLM-QAT 2023.05	57.4	45.9	54.9	66.7	63.6	-	-	-
FLAN-UL2 20B 2022.05	56.2	-	-	-	-	-	-	-
GPS 2023.12	55.9	40.4	56.9	64.1	62.3	-	-	-
APO 2023.12	55.9	40	57.2	63.7	62.7	-	-	-
Perplexity 2023.12	55.2	39.5	55.2	64.1	61.9	-	-	-
LLM-QAT 2023.05	55.1	44.4	52.8	63.6	61.2	-	-	-
ROUGE-L 2023.12	55	39.5	55.5	63.7	61.1	-	-	-
FP16 2026.04	55	-	-	-	-	-	-	-
DeepSeekMath-Base 2024.02	54.9	-	-	-	-	-	-	-
Few-shot 2023.12	54.6	38.7	55.8	63.3	61.5	-	-	-
COVERCAL 2026.04	54.6	-	-	-	-	-	-	-
LLM-QAT 2023.05	54.4	44.3	52.2	61.4	61	-	-	-
RTN 2023.05	54.2	44	50.9	62.8	61.3	-	-	-
Max-ActVar 2026.04	54	-	-	-	-	-	-	-
RTN 2023.05	53.9	43.6	51	62.2	60.6	-	-	-
FLAN T5 XXL 2022.05	53.7	-	-	-	-	-	-	-
Random 2026.04	53.5	-	-	-	-	-	-	-
LLM-QAT 2023.05	52.7	42.2	49.7	60.8	59.7	-	-	-
LLM-QAT 2023.05	51	41.7	47.7	58.9	57.5	-	-	-
Original 2026.04	51	40	-	-	-	-	-	-
FPFT 2026.04	51	40	-	-	-	-	-	-
H-CoT 2026.04	51	39	-	-	-	-	-	-
AutoRAN 2026.04	51	39	-	-	-	-	-	-
Chain-of-Lure 2026.04	51	40	-	-	-	-	-	-
CoT Hijacking 2026.04	51	40	-	-	-	-	-	-
LRFT 2026.04	50	39	-	-	-	-	-	-
TSBH (Pass@1) 2026.04	50	40	-	-	-	-	-	-
TSBH (Pass@5) 2026.04	50	40	-	-	-	-	-	-
LLM-QAT 2023.05	49.3	40.4	47.6	55.9	54.5	-	-	-
DeepSeek-Coder-Base-v1.5 2024.02	49.1	-	-	-	-	-	-	-
AdaGC 2025.02	48.7	-	-	-	-	-	-	-
GlobalGC 2025.02	48.61	-	-	-	-	-	-	-
GlobalGC 2025.02	48.48	-	-	-	-	-	-	-
METRO-T0++LARGE++ 2023.05	48	-	-	-	-	-	-	-
GlobalGC 2025.02	47.75	-	-	-	-	-	-	-
OPT IML-Max 175B 2022.05	47.1	-	-	-	-	-	-	-
LLaMA-13B 2023.05	46.7	36.2	44.4	54.3	53.3	-	-	-
SmoothQuant 2023.05	46.7	36.4	44.5	53.7	53.4	-	-	-
SmoothQuant 2023.05	46.6	36.1	44.5	53.5	53.3	-	-	-
LLM-QAT 2023.05	46.3	36.1	43.6	53.8	53.2	-	-	-
RTN 2023.05	45.9	35.6	44.2	52.2	52.5	-	-	-
LLM-QAT 2023.05	45.8	36.1	43.5	52.6	52.5	-	-	-

Showing 100 of 226 rows