Share your thoughts, 1 month free Claude Pro on usSee more

Algorithmic Program Execution on FP Algorithmic Task n=100 (test)

100Accuracy

DeepSeek-Coder-33B-Instruct

Updated 23d ago

Evaluation Results

Method	Links
DeepSeek-Coder-33B-Instruct 2025.10		100
OLMo-3-32B-Instruct 2025.10		100
Gemini-3.1-Pro 2025.10		100
GPT-5-Thinking 2025.10		100
GPT-5-Thinking 2025.10		100
GPT-5-Thinking 2025.10		99
GPT-5-Thinking 2025.10		82
MLAgentBench (GPT-5-Thinking) 2025.10		53.6
Qwen3-30B-Instruct 2025.10		53
DeepSeek-Coder-1.3B 2025.10		52.2
Llama-3.2-1B 2025.10		52.2
Qwen3-1.7B 2025.10		51.2
DeepSeek-Coder-1.3B 2025.10		50.5
Llama-3.2-1B 2025.10		50.4
Qwen3-30B-Instruct 2025.10		50.4
TabPFN 2025.10		50.1
Qwen3-Coder-30B-Instruct 2025.10		50
XGBoost 2025.10		49.9
SVM 2025.10		49.9
Random Forest 2025.10		49.8
Genetic Algorithm 2025.10		49.8
AIDE-ML (GPT-5-Thinking) 2025.10		49.5
Qwen3-1.7B 2025.10		49.3
Gemini-3.1-Pro 2025.10		48
DeepSeek-Coder-33B-Instruct 2025.10		43