Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Code execution on CodeNetMut (test)
Loading...
48.06
Output Accuracy
CodeExecutor
16.2256
24.4903
32.755
41.0197
May 8, 2023
Output Accuracy
Trace Accuracy
Line Precision
Line Recall
Line F1
Identifier Precision
Identifier Recall
Identifier F1
Updated 4d ago
Evaluation Results
Method
Method
Links
Output Accuracy
Trace Accuracy
Line Precision
Line Recall
Line F1
Identifier Precision
Identifier Recall
Identifier F1
CodeExecutor
curriculum_learning=true
2023.05
48.06
33.38
58.7
43.48
49.96
67.81
45.29
54.31
CodeExecutor w/o CL
curriculum_learning=false
2023.05
45.93
30.98
60.21
42.45
49.79
68.55
41.58
51.76
CEL-S3
training_stage=Stage 3
2023.05
43.8
29.44
59.32
41.76
49.01
68.3
41.69
51.78
Codex
shots=3
2023.05
17.45
-
-
-
-
-
-
-
Feedback
Search any
task
Search any
task