Share your thoughts, 1 month free Claude Pro on usSee more

Distractor Generation on ARC Challenge

21.08P@1

GPT-3(few-shot-random)

Updated 1mo ago

Evaluation Results

Method	Links
GPT-3(few-shot-random) 2026.04		21.08	7.07	14.96	24.82	27.75	0.26	0.17
GPT-3(few-shot-k-NN) 2026.04		20.48	6.87	15.05	24.06	26.8	0.17	0.09
GPT-3(few-shot-COT-random) 2026.04		20.31	6.81	15.29	24.23	27.23	0.34	0.09
GPT-3(few-shot-COT-k-NN) 2026.04		19.88	6.67	14.6	23.25	25.73	0.17	0.09
GPT-3(zero-shot) 2026.04		17.41	5.85	11.96	20.31	22.53	0.09	0.09
TinyLlama 2026.04		12.37	4.15	8.6	15.15	17.55	1.37	1.11
T5(contrast) 2026.04		9.13	3.09	5.47	11.26	13.06	9.64	3.07
GPT-3-CSG-DS(clustering) 2026.04		8.7	2.92	4	10.75	12.68	0	0
T5-CSG-DS(beam) 2026.04		7.85	2.67	6.5	11.46	14.71	31.83	0.26
T5(base) 2026.04		7.17	2.45	3.68	8.79	10.14	19.2	2.9
T5-CSG-DS(clustering) 2026.04		5.46	1.86	2.23	7.35	9.18	12.97	0
T5(multi-task) 2026.04		5.38	1.85	3.63	7.21	8.81	18.43	2.39