Share your thoughts, 1 month free Claude Pro on usSee more

Role-Play Evaluation on CoSER

62.43Avg Score

Claude-4.5-Opus

Updated 1mo ago

Evaluation Results

Method
Claude-4.5-Opus 2026.01	62.43	63.74	64.28	58.45	63.24	-
Claude 4.5-Opus 2026.06	62.43	63.74	64.28	58.45	63.24	-
Gemini-3-Pro 2026.01	61.8	65.95	60.42	58.34	62.49	-
Gemini 3-Pro 2026.06	61.8	65.95	60.42	58.34	62.49	-
GPT-5.1 2026.01	61.1	64.95	53.99	60.13	65.35	-
Gemini-2.5-Pro 2026.01	60.68	61.05	60.8	57.48	63.4	-
DeepSeek-v3.2 2026.01	58.68	55.85	57.07	57.44	64.35	-
MiniMax-M2-her 2026.01	57.3	60.03	50.11	49.3	69.77	-
DeepSeek-v3.1 2026.01	53.5	50.15	53.18	53.93	56.72	-
HER-RL 2026.01	53.12	54.33	47.26	52.78	58.12	-
HER-SFT 2026.01	50.92	50.52	45.99	49.78	57.37	-
Qwen3.5 397B-Agentopia 2026.06	49.16	41.02	49.67	46.93	59.01	-
Qwen2.5-7B-Instruct-CoSER 2026.06	49.03	54.8	37.07	45.17	59.08	-
Qwen3-4B-Instruct + Psy-CoT + RAPO 2026.06	47.87	55.41	40.8	39.32	55.94	-
Qwen2.5-7B-Instruct + Psy-CoT + RAPO 2026.06	47.74	55.77	44.67	37.5	53.02	-
Llama-3.1-8B-CoSER 2026.06	47.59	49.9	37.27	42.82	60.37	-
Grok-4.1-Fast 2026.01	47.4	49.21	47.57	42.64	50.17	-
Qwen2.5-14B-Instruct-AdaMARP 2026.06	47	48.56	36.02	45.03	57.38	-
Llama-3.1-8B-Crab 2026.06	46.64	52.12	37.13	40.99	56.3	-
Llama-3.3-70B-Instruct 2026.06	46.2	49.71	36.17	40.94	57.97	-
Llama-3.1-70B-CoSER 2026.06	45.89	48.17	36.26	40.95	58.2	-
GPT-5-Chat 2026.06	45.88	46.78	35.1	42.03	59.61	-
Qwen2.5-7B-Instruct + Psy-CoT + GRPO 2026.06	45.82	53.27	42.52	34.45	52.04	-
DeepSeek-V4-Flash 2026.06	45.49	47.07	37.3	39.77	57.84	-
Qwen3-4B-Instruct + Psy-CoT + GRPO 2026.06	45.42	55.6	41	34.64	50.44	-
Claude-4.5-Sonnet 2026.01	45.21	47.18	36.02	47.55	50.09	-
Claude 4.5-Sonnet 2026.06	45.21	47.18	36.02	47.55	50.09	-
GPT-4o-mini 2026.06	45.09	47.32	40.18	40.51	52.47	-
Qwen2.5-7B-Instruct-Crab 2026.06	44.75	55.31	32.18	40.44	51.06	-
Llama-3.1-8B-Instruct + Psy-CoT + RAPO 2026.06	44.55	53.94	41.34	33.99	48.95	-
Qwen2.5-14B-Instruct 2026.06	43.91	47.73	34.97	39.65	53.29	-
Doubao-1.5-Pro-Character 2026.06	43.52	48.13	32.74	40.93	52.28	-
Llama-3.1-8B-Instruct + Psy-CoT + GRPO 2026.06	43.32	52.99	37.57	35.08	47.64	-
Qwen2.5-7B-Instruct + Psy-CoT + DAPO 2026.06	43.1	52.09	39.08	37.19	44.03	-
Qwen2.5-7B-Instruct + Psy-CoT + GSPO 2026.06	42.61	51.84	37.66	32.18	48.78	-
Qwen3.5 397B 2026.06	42.51	39.6	40.16	40.32	49.97	-
Qwen3-4B-Instruct 2026.06	42.11	47.01	35.56	35.72	50.17	-
Qwen2.5-7B-Instruct + Psy-CoT + CPO 2026.06	41.88	52.31	36.85	28.73	49.62	-
Qwen2.5-7B-Instruct 2026.06	41.31	48.09	33.15	37.42	46.58	-
Claude-3.7-Think 2026.01	39.73	44.84	31	42.45	40.65	-
CoSER-70B 2026.01	35.95	35.05	31.16	32.28	45.33	-
CoSER 70B 2026.06	35.95	35.05	31.16	32.28	45.33	-
GPT-5-Mini 2026.01	32.97	38.1	24.6	27.2	42	-
GPT-5 Mini 2026.06	32.97	38.1	24.6	27.2	42	-
Llama-3.1-8B-Instruct 2026.06	31.69	37.39	25.78	30.22	33.36	-
GPT-4o-240806 2026.01	27.69	34	14.9	22.9	38.9	-
GPT-OSS-120B 2026.01	26.12	32.8	14.8	21.5	35.4	-
Qwen3-32B 2026.01	22.86	30.56	19.61	15.52	30.56	-
GPT-5.4 2026.05	-	-	-	-	-	65.9
GPT-5-nano 2026.05	-	-	-	-	-	35.2
HER-32B 2026.05	-	-	-	-	-	41.9
Qwen3-VL-8B-Instruct 2026.05	-	-	-	-	-	43.5
GRPO 2026.05	-	-	-	-	-	54.1
DITTO 2026.05	-	-	-	-	-	51.2
GPT 5.5 2026.06	-	-	-	-	-	66.2
Gemini 3.1 Pro 2026.06	-	-	-	-	-	62.1
Claude Opus 4.7 2026.06	-	-	-	-	-	66.5
Qwen 3.6 Plus 2026.06	-	-	-	-	-	55.9
Others * 2026.06	-	-	-	-	-	30.3
Qwen3 8B Inst 2026.06	-	-	-	-	-	43.5
Base 8B 2026.06	-	-	-	-	-	6.1
OSIM 8B-Mid 2026.06	-	-	-	-	-	24.8
OSIM 8B 2026.06	-	-	-	-	-	62.6