Share your thoughts, 1 month free Claude Pro on usSee more

Verilog Generation on VerilogEval v2

89.6Pass@1

GPT-OSS-120B

Updated 1mo ago

Evaluation Results

Method
GPT-OSS-120B 2026.06	89.6	-	-	-	-	-	96.7	97.6
Claude Opus 4.7 2026.05	86.9	-	-	-	-	-	90.4	-
GPT-5.5 2026.05	84.7	-	-	-	-	-	90.4	-
SiliconMind-V1 2026.06	82	-	-	-	-	-	89.6	91
SiliconMind-V1 2026.06	81	-	-	-	-	-	89.8	92.4
STG 2026.06	80.2	-	-	-	-	-	89.9	92
ChipMATE-Agents-9B 2026.05	80.1	-	-	-	-	-	82.4	-
STG 2026.06	80	-	-	-	-	-	90.2	91.5
DeepSeek R1 2026.05	77.5	-	-	-	-	-	84.7	-
ChipMATE-Verilog-9B 2026.05	75.3	-	-	-	-	-	77.6	-
ChipMATE-Agents-4B 2026.05	75	-	-	-	-	-	76.3	-
SiliconMind-V1 2026.06	73.9	-	-	-	-	-	83.6	85.8
CodeV-R1 2026.06	73.2	-	-	-	-	-	83.6	86.6
STG 2026.06	70.5	-	-	-	-	-	84.9	89.4
CodeV-R1 2026.05	68.8	-	-	-	-	-	78.2	-
DeepSeek Coder 2026.05	68.5	-	-	-	-	-	80.8	-
ChipMATE-Verilog-4B 2026.05	67.4	-	-	-	-	-	71.8	-
DeepSeek V4 2026.05	67.3	-	-	-	-	-	80.1	-
VeriRL (paper) 2026.06	67.2	-	-	-	-	-	76.1	-
CodeV-R1 (distill) 2026.05	65.2	-	-	-	-	-	75.2	-
GPT-4o 2026.05	64.1	-	-	-	-	-	73.7	-
VeriRL (reproduced) 2026.06	60.8	-	-	-	-	-	74.8	78.8
Qwen3-8B 2026.06	52.5	-	-	-	-	-	65.4	69.1
Qwen3.5-9B 2026.05	48.5	-	-	-	-	-	66.6	-
Qwen3.5-4B 2026.05	41.7	-	-	-	-	-	60.9	-
Qwen2.5-C-7B-Instruct 2026.06	33.6	-	-	-	-	-	53.7	60.1
Qwen3-4B-Thinking 2026.06	21.4	-	-	-	-	-	30.4	33.4
Gemini-2.0-Flash 2026.01	-	72.4	74.4	77.6	78.8	80.1	-	-
Siliconmind-7B 2026.01	-	82.1	87.8	89.7	91.7	91.7	-	-
GPT-OSS-120B 2026.01	-	92.9	93.6	94.2	94.2	94.2	-	-
DeepSeek-R1-FP4 2026.01	-	91	94.2	94.9	94.9	95.5	-	-
REvolution (Deepseek-V3) 2026.01	-	-	-	-	95.5	-	-	-
REvolution (Llama-3.3-70B) 2026.01	-	-	-	-	88.5	-	-	-
VerilogCoder (Llama-3-70B) 2026.01	-	-	-	67.3	-	-	-	-
VerilogCoder (GPT-4-Turbo) 2026.01	-	-	-	94.2	-	-	-	-
Gemini-2.0-Flash-IGR 2026.01	-	76.3	85.3	86.5	86.5	86.5	-	-
Gemini-2.0-Flash-MCTS 2026.01	-	77.6	81.4	84.6	86.5	87.8	-	-
Siliconmind-7B-MCTS 2026.01	-	84	85.3	86.5	90.4	92.3	-	-
GPT-OSS-120B-IGR 2026.01	-	89.1	95.5	96.2	96.8	96.8	-	-
GPT-OSS-120B-MCTS 2026.01	-	89.7	93.6	94.9	95.5	96.2	-	-
DeepSeek-R1-FP4-MCTS 2026.01	-	94.2	96.8	96.8	97.4	98.1	-	-