Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Code Generation on CoderEval-Python file-runnable
Loading...
30.29
Pass@1
COCOGEN
14.3884
18.5167
22.645
26.7733
Mar 25, 2024
Pass@1
Pass@5
Pass@10
Updated 4d ago
Evaluation Results
Method
Method
Links
Pass@1
Pass@5
Pass@10
COCOGEN
Base LLM=GPT-3.5-Turbo
2024.03
30.29
43.58
47.06
RepoCoder
Base LLM=GPT-3.5-Turbo
2024.03
29.41
34.61
36.76
Direct
Base LLM=GPT-3.5-Turbo
2024.03
19.85
27.62
30.88
Direct
Base LLM=Code Llama (13B)
2024.03
18.53
27.82
29.41
ReACC
Base LLM=GPT-3.5-Turbo
2024.03
17.65
28.92
33.82
ReACC
Base LLM=Code Llama (13B)
2024.03
17.65
27.61
33.82
COCOGEN
Base LLM=Code Llama (13B)
2024.03
17.06
29.39
33.82
RepoCoder
Base LLM=Code Llama (13B)
2024.03
15
28.31
32.35
Feedback
Search any
task
Search any
task
