Share your thoughts, 1 month free Claude Pro on usSee more

Software Engineering on SWE-bench Verified (Accuracy)

62.6Accuracy

JoyAI-LLM Flash

Updated 16d ago

Evaluation Results

Method	Links
JoyAI-LLM Flash 2026.04		62.6
PivotRL 2026.03		61.33
GLM-4.7-Flash-T 2026.04		59.4
Qwen3.5-35B-A3B 2026.04		57.4
AdaptOrch 2026.02		52.6
Self-MoA (matched) 2026.02		51.5
MoA-3L 2026.02		48.1
Static-Parallel 2026.02		47.3
Static-Sequential 2026.02		45.6
LLM-Blender 2026.02		44.9
Single Best 2026.02		42.8
SWE-Agent 2025.10		40.3
SWE-Agent 2025.10		40.2
OT-Agent-SFT-8B (100K) 2026.06		38.9
SWE-Lego-Qwen3-8B 2026.06		37.6
OpenHands CodeActAgent 2025.10		36.8
SWE-Agent 2025.10		34.4
SWE-Agent 2025.10		33.6
OT-Agent-ColdSFT+RL-8B 2026.06		31.9
Qwen3-Next-80B-A3B 2026.04		31.2
OpenHands CodeActAgent 2025.10		30.6
Qwen3-30B-A3B + SAO 2026.07		29.8
Qwen3-30B-A3B 2026.04		29
Qwen3-30B-A3B + GRPO (w/ DIS) 2026.07		27
Qwen3-30B-A3B 2026.07		23
OT-Agent-SFT-8B (10K) 2026.06		22.7
Nemotron-Terminal-8B 2026.06		22.1
OpenHands CodeActAgent 2025.10		20.6
OpenHands CodeActAgent 2025.10		20.4
SWE-Agent 2025.10		20.2
OpenHands CodeActAgent 2025.10		16.4
SWE-Agent 2025.10		15.8
SWE-Agent 2025.10		15.2
Qwen3-8B 2026.06		13.2
Nemotron-3-Super 2026.03		12.87
Endless Terminals 2026.06		12.5
OpenHands CodeActAgent 2025.10		10.6
OpenHands CodeActAgent 2025.10		10.6
OpenHands CodeActAgent 2025.10		5.8
OpenHands CodeActAgent 2025.10		2.8
SWE-Agent 2025.10		2.2
SWE-Agent 2025.10		2
SWE-Agent 2025.10		0.4