Our new X account is live! Follow @wizwand_team for updates
Search any
task
Feedback
Search any
task
SOTA Program synthesis benchmarks and papers with code | Wizwand
Our new X account is live! Follow @wizwand_team for updates
Home
/
Tasks
Program synthesis
Benchmarks
Dataset Name
SOTA Method
Dataset Name
SOTA Method
Metric
Trend
Results
Last Updated
C-RASP Synthesis Benchmark Suite Regular, Counting, and Context-free Languages
C-RASP
Synthesis Result Score
27
34
4d ago
VizDoom (test)
EVAPS
Exact Match
62.22
18
4d ago
HumanEval (test)
SC-MAS
Accuracy
92.37
16
4d ago
PSB 1
HOTGP
Compare String Length
100
15
4d ago
APPS 1.0 (test)
CodeRL+CodeT5
Pass@5 (Introductory)
25.61
11
4d ago
PSB1 1 (val)
DSLS
Last Index of Zero
62
10
4d ago
SPoC (TestW)
DrRepair w/ pseudocode
Success Rate
57
10
4d ago
SPoC (TestP)
DrRepair w/ pseudocode
Success Rate
0.385
10
4d ago
C dataset (test)
LaSynth
Accuracy
55.2
7
4d ago
PSB1 (train)
HOTGP
Compare String Lengths
100
5
4d ago
APPS
CodeRL+CodeT5
Pass@5 (Introductory)
25.61
5
4d ago
PSB1
Copilot
Checksum Correctness
89
4
4d ago
openai_humaneval (test)
BLOOM-176B
Pass@1
15.52
4
4d ago
Karel (test)
Exec
Generalization Accuracy
86.04
4
4d ago
PSB2
Origami AC/DC
Basement
40
3
4d ago
HumanEval Standard Relaxed (test)
QualityFlow
pass@1
0.988
3
4d ago
PolyPSB
Origami AC/DC
Area of Rectangle
100
2
4d ago
PSB2 (test)
Copilot
Basement
95
2
4d ago
HumanEval-EvalPlus Standard (test)
QualityFlow
pass@1
89.6
2
4d ago
MBPP-EvalPlus Standard (test)
QualityFlow
Pass@1
79.9
2
4d ago
INV-BV (Invariant Bit-Vector)
ASAP
Count: INV NE BVUDIV1 (4bit)
0
2
4d ago
SLIA (String Linear Integer Arithmetic)
ASAP
Phone-3 Long Score
0
2
4d ago
Assembly and Algorithmic Tasks
-
-
0
4d ago
Showing 23 of 23 rows
25 / page
50 / page
100 / page
1
Search any
task
Search any
task
Terms of Service
FAQs
