Share your thoughts, 1 month free Claude Pro on usSee more

Autonomous Machine Learning Engineering on MLE-Bench Lite

81.82Any Medal Rate

AiScientist

Updated 12d ago

Evaluation Results

Method
AiScientist 2026.04	81.82	100	86.36	18.18	31.82	31.82	-	-
AiScientist 2026.04	81.82	100	90.91	9.09	31.82	40.91	-	-
AIBuildAI 2026.04	77.27	100	81.82	13.64	25.76	37.88	-	-
LoongFlow 2026.04	77.27	77.27	77.27	12.12	25.76	39.39	-	-
ML-Master 2.0 2026.04	75.76	100	84.85	13.64	31.82	30.3	-	-
Famou-Agent 2.0 2026.04	75.76	100	86.36	15.15	19.7	40.91	-	-
EvoMaster 2026.04	75.76	100	84.85	13.64	31.82	30.3	-	-
MARS 2026.04	74.24	100	89.39	6.06	15.15	53.03	-	-
Leeroo 2026.03	68.2	-	-	-	-	-	-	-
GOME 2026.03	68.2	-	-	-	-	-	-	-
R&D-Agent 2026.04	68.18	77.27	74.24	12.12	22.73	33.33	-	-
Leeroo 2026.04	68.18	68.18	68.18	18.18	19.7	30.3	-	-
MLE-STAR 2026.04	68.18	100	75.76	16.67	16.67	34.85	-	-
R&D-Agent 2026.04	68.18	77.27	74.24	12.12	22.73	33.33	-	-
Thesis 2026.03	65.2	-	-	-	-	-	-	-
ML-Master 2.0 2026.04	63.64	100	81.82	18.18	13.64	31.82	-	-
InternAgent 2026.04	62.12	100	78.79	10.61	16.67	34.85	-	-
InternAgent-MLE 2026.03	62.1	-	-	-	-	-	-	-
FM Agent 2026.03	62.1	-	-	-	-	-	-	-
CLAUDE-4.5-SONNET 2026.02	60.61	100	80.3	7.58	16.67	36.36	75.86	-
GEMINI-3-PRO 2026.02	59.09	100	84.85	4.55	15.15	39.39	79.18	-
GPT-5.2 2026.02	56.06	95.45	71.21	10.61	16.67	28.79	71.05	-
AIRA 2026.03	55	-	-	-	-	-	-	-
AIRA-dojo 2026.04	55	100	70.45	7.95	12.73	34.32	-	-
ACE-30B 2026.02	51.52	100	78.79	12.12	7.58	31.82	71.14	-
KompeteAI 2026.03	51.5	-	-	-	-	-	-	-
ML-Master 2026.03	48.5	-	-	-	-	18.1	-	-
ML-Master 2026.04	48.48	100	74.24	4.55	13.64	30.3	-	-
AIRA 2026.03	47.73	-	-	-	-	28.64	-	-
+ IdleSpec (Ours) 2026.05	45.5	95.5	59.1	13.6	13.6	18.2	-	86.4
SciDER 2026.03	45.45	-	-	-	-	36.4	-	-
AIDE 2026.04	45.45	77.27	54.55	4.55	9.09	31.82	-	-
AIDE 2026.04	40.91	77.27	50	4.55	13.64	22.73	-	-
+Sequential Revision 2026.05	40.9	90.1	50	13.6	9.1	18.2	-	81.8
DEEPSEEK-V3.2 2026.02	39.39	98.48	71.21	7.58	4.55	27.27	65.92	-
QWEN3-235B-2507 2026.02	37.88	92.42	69.7	6.06	7.58	24.24	69.7	-
Vanilla 2026.05	36.4	86.4	40.1	9.1	9.1	18.2	-	77.3
+Sleep-time Compute 2026.05	36.4	90.1	59.1	13.6	9.1	13.6	-	81.8
Claude-4.5-Sonnet 2026.04	31.8	95.5	31.8	0	4.5	27.3	-	-
GLM-4.6 2026.02	30.3	90.91	45.45	6.06	3.03	21.21	45.45	-
Qwen3-14B-SFT-SandMLE 2026.04	27.3	95.5	31.8	4.5	0	22.7	-	-
Qwen3-30B-SandMLE 2026.04	27.3	100	36.4	0	13.6	13.6	-	-
Qwen3-30B-SFT-SandMLE 2026.04	27.3	90.9	45.5	4.5	9.1	13.6	-	-
QWEN3-30B-2507 2026.02	27.27	84.85	50	1.52	7.58	18.18	48.75	-
Deepseek-V3.1 2026.04	22.7	90.9	36.4	0	4.5	18.2	-	-
Gemini-2.5-flash 2026.04	22.7	81.8	45.5	4.5	4.5	13.6	-	-
Qwen3-8B-SandMLE 2026.04	22.7	63.6	27.3	0	4.5	18.2	-	-
Qwen3-8B-SFT-SandMLE 2026.04	22.7	90.9	31.8	4.5	0	18.2	-	-
Qwen3-14B-SandMLE 2026.04	22.7	77.3	27.3	4.5	4.5	13.6	-	-
Qwen3-30B-Seed-SFT 2026.04	22.7	77.3	31.8	4.5	4.5	13.6	-	-
Qwen3-14B-Base 2026.04	18.2	72.7	18.2	4.5	4.5	18.2	-	-
Qwen3-14B-Seed-SFT 2026.04	18.2	72.7	22.7	4.5	0	13.6	-	-
OpenClaw 2026.04	18.18	81.82	31.82	4.55	13.64	0	-	-
AIDE 2026.03	16.9	-	-	-	-	9.4	-	-
Qwen3-8B-Base 2026.04	13.6	68.2	18.2	0	4.5	9.1	-	-
Qwen3-8B-Seed-SFT 2026.04	13.6	72.7	18.2	4.5	0	9.1	-	-
Qwen3-30B-Base 2026.04	13.6	68.2	18.2	4.5	0	9.1	-	-