Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Function-level synthesis on LCB hard (held-out 25% eval)
Loading...
33.2
Utility Difference (Δπ)
SR
-1.016
7.867
16.75
25.633
Jun 23, 2026
Utility Difference (Δπ)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Utility Difference (Δπ)
SR
Generator=gpt-5-mini
2026.06
33.2
BoN
Generator=gpt-5-mini
2026.06
20
bayesian_greedy
Generator=gpt-5-mini
2026.06
7.6
fixed_pipeline
Generator=gpt-5-mini
2026.06
7.5
tCllm
Generator=gpt-5-mini
2026.06
6
bayesian_DP
Generator=gpt-oss-20b
2026.06
4.6
Rfx
Generator=gpt-5-mini
2026.06
4.4
tCtest
Generator=gpt-5-mini
2026.06
4.3
tCsyn
Generator=gpt-5-mini
2026.06
3.1
bayesian_DP
Generator=claude-sonne...
2026.06
2.5
bayesian_DP
Generator=qwen3-coder
2026.06
2.4
bayesian_DP
Generator=claude-haiku...
2026.06
2.1
bayesian_DP
Generator=gpt-5-mini
2026.06
0.3
Feedback
Search any
task
Search any
task