Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Feature Implementation on FEA-Bench (test)
Loading...
5.92
Pass Rate
SWE-SPOT
-0.2368
1.3616
2.96
4.5584
Jan 29, 2026
Pass Rate
Average Execution Metric
Updated 1mo ago
Evaluation Results
Method
Method
Links
Pass Rate
Average Execution Metric
SWE-SPOT
Size=4B, SWE Training...
2026.01
5.92
17.12
GPT-4.1-mini
Size=[10, 100]B, SWE T...
2026.01
5.7
17.85
Mini-Coder-4B
Size=4B, SWE Training...
2026.01
4.61
8.7
CWM (Meta)
Size=32B, SWE Training...
2026.01
4.17
15.88
Qwen3-Coder-30B
Size=32B, SWE Training...
2026.01
3.29
11.56
GPT-5-nano
Size=[0.1, 10]B, SWE T...
2026.01
1.75
2.97
Gemini-2.5-Flash-Lite
Size=[0.1, 10]B, SWE T...
2026.01
0.88
7.46
Gemma-3-27b-it
Size=27B, SWE Training...
2026.01
0
1.34
Qwen3-4B-Instruct-2507
Size=4B, SWE Training...
2026.01
0
3.2
Feedback
Search any
task
Search any
task