Share your thoughts, 1 month free Claude Pro on usSee more

Autonomous Machine Learning Engineering on MLE-bench (held-in and held-out)

76.53CIFAR-10 Performance

AIDE (GPT-5)

Updated 1mo ago

Evaluation Results

Method	Links
AIDE (GPT-5) 2025.05		76.53	22.15	8.77	77.38	31.5	9.18	26.42	4.51	0.13	4.25	0.14	29.69	0.11	20.95
ML-Master (DeepSeek-R1) 2025.05		73.43	18.25	12.07	14.56	14.75	2.78	33.39	3.03	0	4.01	0.04	29.27	0.22	13.13
AIDE (DeepSeek-R1) 2025.05		72.55	5.35	13.07	33.23	10.25	4.54	30.77	1.38	0.31	0.36	0.01	5.78	0.14	11.4
ML-Master (GPT-5) 2025.05		71.64	22.3	10.54	10.96	23.88	2.48	67.07	6.38	0.79	10.41	0.35	26.49	0.25	19.12
MLAB (GPT-5) 2025.05		61.46	12.15	12.74	66	45.63	6.43	89.59	4.36	11.2	6.79	0	23.38	0.23	18.14
MLAB (Qwen3-235B) 2025.05		57.61	3.01	6.7	62.6	2.12	16.36	26.68	0.3	1.02	0.8	0.01	1.96	0.07	10.93
AIDE (Gemini-2.5-Pro) 2025.05		53.59	11.13	9.44	62.72	84.25	6.08	56.45	7.35	0.74	4.34	0.04	31.92	0.13	11.35
ML-Agent 2025.05		33.8	6.77	13.47	52.38	13.87	1.41	72.89	1.91	1.74	1.76	0.01	12.96	0.2	16.4
MLAB (DeepSeek-R1) 2025.05		28.96	3.45	5.53	8.83	4.85	0.04	33.44	0.05	0.25	0.89	0	2.67	0.13	6.83
MLAB (Gemini-2.5-Pro) 2025.05		16.78	1.16	0.1	37.85	4.38	4.26	22.38	0	0.13	0.04	0	0.13	0	5.38
AIDE (Qwen2.5-7B-Instruct) 2025.05		11.36	2.42	7.52	7.33	4.75	4.33	0.52	9.78	0.38	0.07	0.01	0	0.08	0.77
MLAB (Qwen2.5-7B-Instruct) 2025.05		1.37	0.23	1.39	2.1	2.52	6.32	12.25	1.23	0.51	0.46	0.06	3.75	0.04	1.43
ML-Master (Qwen2.5-7B-Instruct) 2025.05		1.03	0	0.1	2.44	1.38	3.99	1.12	1.79	0.26	0.04	0.02	0.02	0	0.1
AIDE (Qwen3-235B) 2025.05		0.1	2.04	11.1	41.65	4.75	2.89	8.26	2.37	0.43	0.96	12.15	0.51	0	4.38