VerilogEval
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Verilog Code Generation | VerilogEval v1 (Human) | Pass@198.1 | 54 | |
| HDL generation | VerilogEval 2.0 | Pass@184.97 | 40 | |
| Verilog Code Generation | VerilogEval Machine | Pass@197.9 | 37 | |
| Verilog Code Generation | VerilogEval SR v2 | Pass@183.3 | 34 | |
| Verilog Code Generation | VerilogEval CC v2 | Pass@182.7 | 33 | |
| RTL generation | VerilogEval 156 cases (test) | Pass@10.78 | 32 | |
| Verilog Code Generation | VerilogEval Machine v1 | Pass@182.9 | 17 | |
| Verilog Generation | VerilogEval v2 | Pass@186.9 | 14 | |
| Code Completion | VerilogEval CC v2 (test) | Pass@179.1 | 11 | |
| Specification-to-RTL | VerilogEval SR v2 (test) | Pass@177.5 | 11 | |
| Python reference-model generation | VerilogEval v2 | Pass@182.4 | 10 | |
| Verilog Code Generation | VerilogEval 2.0 (test) | Pass@182.47 | 10 | |
| Verilog Generation | VerilogEval v2 (41 filtered problems) | Solved Rate0.8049 | 8 | |
| Hardware Generation | VerilogEval 156 problems | Compilation Rate99.4 | 5 |
Showing 14 of 14 rows