Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Code Editing Copy-as-Decode Efficiency Analysis on HEvalFix Python
Loading...
164
Number of Cases (N)
Copy-as-Decode (Oracle)
155.8
159.9
164
168.1
Apr 20, 2026
Number of Cases (N)
Total Gold Tokens (T)
Copied Tokens (T_copy)
Generated Tokens (T_gen)
Copy Fraction (f_line)
Total Copy Operations (K)
Mean Copy Span Length (N_mean)
Median Copy Span Length (p50)
95th Percentile Copy Span Length (p95)
Kernel Speedup (m=8)
Conservative Speedup Bound (m=8)
Aspirational Speedup Bound (N_mean based)
Tightest Speedup Bound
Updated 1mo ago
Evaluation Results
Method
Method
Links
Number of Cases (N)
Total Gold Tokens (T)
Copied Tokens (T_copy)
Generated Tokens (T_gen)
Copy Fraction (f_line)
Total Copy Operations (K)
Mean Copy Span Length (N_mean)
Median Copy Span Length (p50)
95th Percentile Copy Span Length (p95)
Kernel Speedup (m=8)
Conservative Speedup Bound (m=8)
Aspirational Speedup Bound (N_mean based)
Tightest Speedup Bound
Copy-as-Decode (Oracle)
Alignment=difflib.Sequ...
2026.04
164
8,872
6,577
2,295
74.1
295
22.3
19
56
6.79
2.72
3.3
3.45
Feedback
Search any
task
Search any
task