Share your thoughts, 1 month free Claude Pro on usSee more

Interactive Desktop Task Success on OSWorld

50.11Overall Success Rate

Gemini 1.5 Pro + MMSkills

Updated 1mo ago

Evaluation Results

Method
Gemini 1.5 Pro + MMSkills 2026.05	50.11	59.91	50	53.19	53.19	60.86	24.11	70.83	66.67	70.59	65.22
Gemini 1.5 Flash + MMSkills 2026.05	47.97	55.37	42.31	53.19	40.34	56.52	30.98	75	66.67	52.94	60.87
Kimi-v1.5 + MMSkills 2026.05	46.59	57.69	42.31	40.43	48.92	60.86	23.4	79.17	73.33	41.18	69.57
Gemini 1.5 Pro (No skill) 2026.05	44.08	53.47	34.62	57.45	40.43	47.82	31.97	54.17	40	35.29	56.52
RL + corrected reward estimator (rˆt) 2026.06	43.2	-	-	-	-	-	-	-	-	-	-
RL + raw evaluator reward (r˜t) 2026.06	41.8	-	-	-	-	-	-	-	-	-	-
Gemini 1.5 Pro (Text-only) 2026.05	40.76	44.35	34.62	38.3	40.34	56.52	22.38	70.83	66.67	41.18	56.52
Gemini 1.5 Flash (Text-only) 2026.05	40.27	51.02	23.08	38.3	34	56.52	19.16	54.17	60	58.82	52.17
RL + corrected reward estimator (rˆt) 2026.06	39.9	-	-	-	-	-	-	-	-	-	-
Kimi-v1.5 (Text-only) 2026.05	39.66	57.69	40	40.43	36.14	17.38	22.38	62.5	53.33	58.82	43.48
Qwen2-VL-72B + MMSkills 2026.05	39.17	59.91	69.23	23.4	32.01	47.82	19.35	41.67	73.33	41.18	56.52
GLM-4V + MMSkills 2026.05	38.51	51.02	53.85	31.91	31.83	43.47	22.26	66.67	40	23.53	65.22
RL + raw evaluator reward (r˜t) 2026.06	38.5	-	-	-	-	-	-	-	-	-	-
Gemini 1.5 Flash (No skill) 2026.05	36.65	37.78	50	38.3	29.73	52.17	21.51	54.17	66.67	52.39	47.83
GLM-4V (Text-only) 2026.05	36.61	53.24	53.85	31.91	31.98	52.17	20.24	20.83	46.67	35.29	65.22
Kimi-v1.5 (No skill) 2026.05	34.98	51.02	34.62	34.04	35.32	30.43	14.86	54.17	66.67	32.6	52.17
GLM-4V (No skill) 2026.05	28.71	37.78	19.23	21.28	29.7	26.08	18.7	54.17	53.33	11.76	47.83
Qwen2-VL-72B (Text-only) 2026.05	28.57	42.22	50	10.64	21.31	34.78	14.86	33.33	60	35.29	47.83
UI-TARS 2026.06	28.3	-	-	-	-	-	-	-	-	-	-
Qwen2-VL-7B-Instruct + MMSkills 2026.05	25.4	39.91	42.31	8.51	23.37	17.39	13.43	25	60	29.41	47.83
Qwen2-VL-72B (No skill) 2026.05	21.34	15.56	38.46	17.02	25.53	43.48	9.48	25	26.67	17.65	34.78
Qwen2-VL-7B-Instruct (Text-only) 2026.05	14.93	19.91	11.54	6.38	16.99	17.39	7.33	16.67	33.33	17.65	34.78
Qwen2-VL-7B-Instruct (No skill) 2026.05	10.78	15.47	7.69	2.13	8.59	4.34	7.33	25	13.33	29.41	17.39