Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Code Localization on SWE-Bench Lite
Loading...
84.33
File-Level Recall
SHERLOC
75.438
77.7465
80.055
82.3635
Jun 23, 2026
File-Level Recall
File-Level Precision
File-Level F1 Score
File-Level Exact Match
Chunk-Level Coverage Recall
Chunk-Level Tightness
Chunk-Level Precision
Interaction Turns
Zero-Shot Performance
Updated 1mo ago
Evaluation Results
Method
Method
Links
File-Level Recall
File-Level Precision
File-Level F1 Score
File-Level Exact Match
Chunk-Level Coverage Recall
Chunk-Level Tightness
Chunk-Level Precision
Interaction Turns
Zero-Shot Performance
SHERLOC
Backbone=Qwen3-235B-A2...
2026.06
84.33
81.1
82.14
78.08
39.14
27.52
44.54
5
0
SHERLOC
Backbone=DeepSeek-V3-0324
2026.06
79.67
68.2
71.81
58
49.26
10.63
46.67
4.4
3.3
SHERLOC
Backbone=Qwen3-30B-A3B...
2026.06
76.33
74.59
75.15
73
25.53
22.96
31.13
7.4
0.1
SHERLOC
Backbone=DeepSeek-R1-0528
2026.06
75.78
71.45
72.72
68
36.49
20.72
39.44
3.4
12
Feedback
Search any
task
Search any
task
