Share your thoughts, 1 month free Claude Pro on usSee more

Agent Task on PinchBench (PB)

100Accuracy

LLM-guided spec search

Updated 15d ago

Evaluation Results

Method	Links
LLM-guided spec search 2026.05		100	3.2
Claude Opus 4.6 2026.05		100	-
GPT 5.4 2026.05		100	-
Gemini 3.1 Pro 2026.05		100	-
Qwen3.5-27B 2026.05		100	-
Qwen3.5-35B 2026.05		100	-
Qwen3.5-122B 2026.05		100	-
Best Local 2026.05		100	-
Qwen3.5-9B 2026.05		96.8	-
Qwen3.5-9B 2026.05		96.8	-
LLM-guided spec search 2026.05		96.5	11.5
Gemma4-26B 2026.05		95.1	-
LLM-guided spec search 2026.05		94	2.7
Qwen3.5-4B 2026.05		91.3	-
Nemotron-Super-120B 2026.05		91.1	-
Gemma4-E4B 2026.05		85	-
Gemma4-E4B 2026.05		85	-
Granite 4.0 H-Small 2026.05		84	-
LLM-guided spec search 2026.05		83	74.3
Granite 3.3 8B 2026.05		28	-
Nemotron-Nano-4B 2026.05		8.7	-