Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Code Localization on SWE-Bench Verified
Loading...
81.27
File-Level Recall
SHERLOC
73.3244
75.3872
77.45
79.5128
Jun 23, 2026
File-Level Recall
File-Level Precision
File-Level F1 Score
File-Level Exact Match
Chunk-Level Coverage Recall
Chunk-Level Tightness
Chunk-Level Precision
Turn Count
Zero-Tool Usage
Updated 1mo ago
Evaluation Results
Method
Method
Links
File-Level Recall
File-Level Precision
File-Level F1 Score
File-Level Exact Match
Chunk-Level Coverage Recall
Chunk-Level Tightness
Chunk-Level Precision
Turn Count
Zero-Tool Usage
SHERLOC
Backbone=Qwen3-235B-A2...
2026.06
81.27
87.55
83.53
75.2
41.28
36.36
53.47
4.7
0
SHERLOC
Backbone=DeepSeek-V3-0324
2026.06
79.53
75.47
75.46
60.2
54.4
14.5
56.4
4.7
0
SHERLOC
Backbone=Qwen3-30B-A3B...
2026.06
75.07
78.5
74.96
67.8
29.8
29.4
39.4
7.2
0
SHERLOC
Backbone=DeepSeek-R1-0528
2026.06
73.63
76.12
73.39
65.2
37.7
25.7
45.9
3.2
10
Feedback
Search any
task
Search any
task
