Share your thoughts, 1 month free Claude Pro on usSee more

Commonsense Reasoning on ARC Challenge

93.8Accuracy

Self-Debias Offline

Updated 8d ago

Evaluation Results

Method
Self-Debias Offline 2026.04	93.8	-	-
SGE 2024.05	93.28	-	-
Self-Debias Offline + Self-Correction 2026.04	93.2	-	-
Self-Debias Iter2 2026.04	93.1	-	-
Self-Debias Iter2 + Self-Correction 2026.04	93	-	-
Self-Debias SFT 2026.04	92.9	-	-
Self-Debias Iter1 + Self-Correction 2026.04	92.8	-	-
Self-Debias Iter1 2026.04	92.5	-	-
Self-Debias SFT + Self-Correction 2026.04	92.1	-	-
Decomp Prompting 2024.05	91.76	-	-
Refine Prompting 2024.05	91.09	-	-
CoT Prompting 2024.05	90.8	-	-
Qwen2.5-7B-Instruct 2026.04	88.9	-	-
PaLM 2023.11	88.7	-	-
Self-consistency 2022.03	88.7	-	-
Self-Agreement 2023.11	87.8	-	-
Self-Agreement 2023.11	87.7	-	-
IO Prompting 2024.05	87.52	-	-
Self-consistency 2022.03	87.5	-	-
GPT-3.5-turbo 2023.11	87	-	-
Self-Agreement 2023.11	86.8	-	-
Mixed-Few-Shot CoT 2023.11	86	-	-
GPT-3.5-turbo 2023.11	85.4	-	-
PaLM 2023.11	85.2	-	-
CoT-prompting 2022.03	85.2	-	-
Qwen2.5-7B-Instruct + Self-Correction 2026.04	85	-	-
DeepSeek-R1-Distill-Qwen-7B 2026.04	83.8	-	-
Qwen1.5-8B 2026.04	83.7	-	-
CoT-prompting 2022.03	83.6	-	-
Self-Consistency 2023.11	83.1	-	-
DeepSeek-R1-Distill-Qwen-7B + Self-Correction 2026.04	81.9	-	-
Qwen1.5-8B + Self-Correction 2026.04	81.7	-	-
Zero-Shot CoT 2023.11	81.5	-	-
Llama-3.1-8B-Instruct 2026.04	78.6	-	-
Fine-tuned SOTA 2020.05	78.5	-	-
Multi-Agents (Debate) 2023.11	78.4	-	-
UnifiedQA-FT 2023.11	75	-	-
Previous SoTA 2022.03	75	-	-
GPT-3.5-turbo 2023.11	73.9	-	-
Llama-3.1-8B-Instruct + Self-Correction 2026.04	71.9	-	-
Self-Agreement 2023.11	70.6	-	-
Bayesian-LoRA (S=4) 2026.01	68.9	9.2	0.77
BLoB (N=10) 2026.01	68.81	9.59	0.78
Self-Agreement 2023.11	68.5	-	-
BBB 2026.01	68.13	26.23	2.23
Self-Consistency 2023.11	68.1	-	-
Mixed-Few-Shot CoT 2023.11	68	-	-
LA (post-hoc) 2026.01	66.9	7.5	0.86
Zero-Shot CoT 2023.11	66.5	-	-
LLLA (post-hoc) 2026.01	66.1	5.6	0.94
MAP 2026.01	64.9	26.1	1.64
Dropout 2026.01	64.9	25.6	1.55
Ckpt Ens 2026.01	64.9	26.1	1.64
Temp 2026.01	64.9	4.6	0.9
LaMDA 2023.11	59.8	-	-
Self-consistency 2022.03	59.8	-	-
LLaMA 2023.02	57.8	-	-
Multi-Agents (Debate) 2023.11	56.5	-	-
LLaMA 2023.02	56	-	-
LaMDA 2023.11	55.1	-	-
CoT-prompting 2022.03	55.1	-	-
Self-consistency 2022.03	53.7	-	-
KEEL 2026.01	53.6	-	-
GPT-3 2020.05	53.2	-	-
PaLM 2023.02	53	-	-
LLaMA 2023.02	52.7	-	-
PaLM 2023.02	52.5	-	-
Pre-LN 2026.01	51.6	-	-
GPT-3 2020.05	51.5	-	-
GPT-3 2020.05	51.4	-	-
GPT-3 2023.02	51.4	-	-
Baseline (LLaMA3-8B) 2024.10	50.42	-	-
UL2 20B 2022.05	49.5	-	-
Self-consistency 2022.03	49.5	-	-
DRAGON 2022.10	48.6	-	-
32-bit AdamW 2023.09	48.3	-	-
4-bit AdamW 2023.09	48	-	-
LLaMA 2023.02	47.6	-	-
32-bit AdamW 2023.09	47.1	-	-
4-bit AdamW 2023.09	46.6	-	-
FP-16 2024.02	46.25	-	-
Baseline (LLaMA2-13B) 2024.10	45.56	-	-
FP-16 2024.02	45.39	-	-
32-bit AdamW 2023.09	45.1	-	-
Original LLaMA 2023.09	45.1	-	-
phi-1.5-web (1.3B) 2023.09	44.9	-	-
GreaseLM 2022.10	44.7	-	-
FP-16 2024.02	44.62	-	-
Original LLaMA 2023.09	44.5	-	-
QAGNN 2022.10	44.4	-	-
phi-1.5 (1.3B) 2023.09	44.4	-	-
4-bit AdamW 2023.09	44.4	-	-
Llama2-7B 2023.09	43.4	-	-
Vicuna-13B (v1.1) 2023.09	43.2	-	-
CoT-prompting 2022.03	43.1	-	-
RoBERTa 2022.10	43	-	-
UL2 20B 2022.05	42.9	-	-
CoT-prompting 2022.03	42.9	-	-
DB-LLM 2024.02	42.58	-	-
FP-16 2024.02	41.38	-	-

Showing 100 of 190 rows