Our new X account is live! Follow @wizwand_team for updates

Android Device Automation on AndroidLab (test)

0.3623SR

DeepSeek-V3.1

Updated 4d ago

Evaluation Results

Method	Links
DeepSeek-V3.1 2025.12		0.3623	0.4095	81.01	0.9463
Qwen3-8B-Instruct 2025.12		0.3623	0.4196	88.04	0.9449
Qwen3-235B-A22B 2025.12		0.3478	0.3876	83.35	0.8948
Qwen3-32B-Instruct 2025.12		0.3478	0.4026	89.47	0.9367
GPT-4-1106-Preview 2025.12		0.3116	0.3821	66.34	0.8624
LLaMA3.1-8B-Instruct 2025.12		0.3115	0.3803	81.28	0.958
Qwen2.5-7B-Instruct 2025.12		0.3043	0.352	102.3	0.9636
Qwen2.5-7B-Instruct 2025.12		0.3015	0.3659	49.19	0.7328
Qwen3-32B-Instruct 2025.12		0.2898	0.3592	97.79	0.9733
GLM4-Plus 2025.12		0.2754	0.3208	92.35	0.8341
Qwen3-8B-Instruct 2025.12		0.2681	0.3109	72.16	0.6985
GPT-4o 2025.12		0.2536	0.3056	107.45	0.8656
LLaMA3.1-8B-Instruct 2025.12		0.2391	0.3031	75.58	0.9246
LLaMA3.1-8B-Instruct 2025.12		0.2391	0.3036	37.96	0.8323
Qwen3-32B-Instruct 2025.12		0.2246	0.282	39.28	0.655
LLaMA3.1-8B-Instruct 2025.12		0.2028	0.2613	69.44	0.9043
Qwen2.5-7B-Instruct 2025.12		0.2028	0.2705	35.52	0.6246
Qwen3-8B-Instruct 2025.12		0.1956	0.256	38.69	0.6518
Gemini-1.5-Pro 2025.12		0.1884	0.224	57.72	0.8399
Qwen3-32B-Instruct 2025.12		0.1812	0.218	91.99	0.8757
Qwen2.5-7B-Instruct 2025.12		0.1232	0.1498	67.56	0.7852
Qwen3-8B-Instruct 2025.12		0.1014	0.1238	66.21	0.6715
Gemini-1.00 2025.12		0.087	0.1075	51.8	0.7108
LLaMA3.1-8B-Instruct 2025.12		0.0507	0.0628	52.77	0.5182
LLaMA3.1-8B-Instruct 2025.12		0.0217	0.0362	-	0.5277