RTL code generation on RTLLM v2.0 (test)
76.2Syntax Pass@1
o1-preview
Updated 1mo ago
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
2025.05 | 76.2 | 83.71 | 84 | 50 | 60.86 | 62.52 | |
2025.05 | 72 | 77.31 | 78.53 | 49.7 | 56.8 | 58.84 | |
2025.05 | 71.6 | 80.58 | 81.75 | 38.5 | 52.62 | 55.99 | |
2025.05 | 68.3 | 81.31 | 83.36 | 33.7 | 49.57 | 52.9 | |
2025.05 | 67.9 | 77.53 | 79.52 | 43.7 | 49.98 | 50 | |
2025.05 | 63.79 | 74.82 | 80.05 | 38.91 | 47.24 | 51.72 | |
2025.05 | 63.5 | 76.74 | 80.1 | 39.7 | 51.96 | 54.7 | |
2025.05 | 60.69 | 78.81 | 80.88 | 28.79 | 45.86 | 49.66 | |
2025.05 | 60.6 | 73.12 | 75.7 | 32.5 | 44.42 | 47.96 | |
2025.05 | 59.3 | 72.38 | 74.67 | 31.4 | 39.59 | 42.57 | |
2025.05 | 57.6 | 80.23 | 83.25 | 30 | 49.63 | 54.74 | |
2025.05 | 56.5 | 69.72 | 71.75 | 30.1 | 39.59 | 41.4 | |
2025.05 | 54.48 | 63.52 | 67.99 | 16.28 | 28.78 | 32.76 | |
2025.05 | 51.5 | 77.71 | 82.17 | 23.1 | 44.54 | 52.22 | |
2025.05 | 47.7 | 74.96 | 82.2 | 22 | 39.49 | 45.74 | |
2025.05 | 32.4 | 57.01 | 62.76 | 14.6 | 26.04 | 30.16 |