Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Question Answering on LongBench V2

51.43SingleDoc Accuracy

Gemini-2.5-Flash-Thinking

Updated 1mo ago

Evaluation Results

Method	Links
Gemini-2.5-Flash-Thinking 2026.01		51.43	55.2	58	66.67	56.77
RLVR 2026.01		50.29	50.4	42	56.41	47.91
LONGPAS 2026.01		47.43	54.4	60	76.92	54.27
LONGPAS 2026.01		46.29	42.4	56	71.79	49.11
GPT-OSS-120B 2026.01		44.57	43.2	53.06	61.54	47.01
GPT5-Nano 2026.01		44	39.2	50	46.15	43.74
Qwen3-30B-A3B-Thinking 2026.01		44	44	46	64.1	48.31
Qwen3-30B-A3B-Instruct 2026.01		42.43	38.4	52	62.82	44.43
LONGPAS 2026.01		40.29	40	43	62.82	42.3
RLVR 2026.01		40	46.4	50	51.28	44.53
LONGPAS 2026.01		39.57	28.4	29.5	31.41	33.7
RLVR 2026.01		39.14	37	49.5	60.9	42.1
GPT-OSS-20B 2026.01		38.51	40.8	56	61.54	43.37
LONGPAS 2026.01		38.14	40.6	48	63.46	42.94
RLVR 2026.01		37.14	40.8	46	66.67	41.75
Qwen3-4B-Thinking 2026.01		37	35.6	41.5	60.26	40.46
Qwen2.5-7B-Instruct 2026.01		36.71	28.2	32	35.9	33.6
Qwen3-4B-Instruct 2026.01		36	30.6	36.5	60.26	37.28
RLVR 2026.01		34.43	29	30	40.38	31.01
RLVR 2026.01		31.86	29.2	29	32.69	28.93
LONGPAS 2026.01		30.14	29.8	32	34.62	29.62
LLaMA3.1-8B-Instruct 2026.01		28.86	26.2	27	32.05	27.93