Share your thoughts, 1 month free Claude Pro on usSee more

Operating System Interaction Agent Task on OS

53.6Success Rate

DeepSeek-R1

Updated 17d ago

Evaluation Results

Method	Links
DeepSeek-R1 2026.07		53.6
DeepSeek-V3 2026.07		53
Claude-Sonnet-4 Thinking 2026.07		51
Claude-Sonnet-4 2026.07		45.3
o4-mini 2026.07		41.8
Qwen2.5-32B-Instruct 2026.07		37
o3-mini 2026.07		35.1
GPT-5 2026.07		34.5
EPPO 2026.07		32.8
AgentRL 2026.07		31.3
BAPO 2026.07		30
DCPO 2026.07		27.2
Qwen2.5-14B-Instruct 2026.07		26
Qwen2.5-3B-Instruct 2026.07		23.9
AgentLM-70B 2026.07		21.5
Hephaestus-8B-Base 2026.07		20.8
Hephaestus-8B-IFT 2026.07		20.8
AgentLM-13B 2026.07		18.1
AgentLM-7B 2026.07		17.4