RTL generation on VerilogEval 156 cases (test)

0.78Pass@1

Claude-4.5-Sonnet

Updated 4d ago

Evaluation Results

Method
Claude-4.5-Sonnet 2025.11	0.78	0.84	0.86
GPT-5 2025.11	0.7	0.76	0.79
VeriDispatcher (Average+OpenAI Emb.-w) 2025.11	0.69	0.76	0.8
VeriDispatcher (OpenAI Emb.-w) 2025.11	0.69	0.78	0.79
VeriDispatcher (Qwen Emb.-w) 2025.11	0.65	0.74	0.79
CodeV-R1-Distill-Qwen-7B 2025.11	0.52	0.65	0.7
HaVen 2025.11	0.48	0.56	0.6
CodeV-DS-6.7B 2025.11	0.46	0.64	0.71
CodeV-QW-7B 2025.11	0.45	0.65	0.69
VeriDispatcher (Average Emb.-wo) 2025.11	0.43	0.62	0.69
VeriDispatcher (Qwen Emb.-wo) 2025.11	0.43	0.61	0.65
VeriDispatcher (OpenAI Emb.-wo) 2025.11	0.38	0.6	0.65
DeepSeek-7B 2025.12	0.38	0.44	-
DeepSeek-7B 2025.12	0.37	0.45	-
RTLCoder-Mistral 2025.11	0.36	0.5	0.56
Qwen-7B 2025.12	0.36	0.49	-
Qwen-7B 2025.12	0.36	0.48	-
Llama-8B 2025.12	0.35	0.46	-
DeepSeek-7B 2025.12	0.35	0.44	-
Qwen-7B 2025.12	0.34	0.48	-
OriGen 2025.11	0.33	0.56	0.63
DeepSeek-7B 2025.12	0.33	0.42	-
Qwen-7B 2025.12	0.33	0.44	-
CodeV-CL-7B 2025.11	0.31	0.56	0.62
Llama-8B 2025.12	0.31	0.45	-
Llama-8B 2025.12	0.31	0.43	-
Llama-7B 2025.12	0.3	0.39	-
Llama-7B 2025.12	0.3	0.38	-
Llama-7B 2025.12	0.29	0.39	-
Llama-7B 2025.12	0.28	0.37	-
Llama-8B 2025.12	0.28	0.38	-
RTLCoder-DeepSeek 2025.11	0.26	0.51	0.58