Share your thoughts, 1 month free Claude Pro on usSee more

Tabular Question Answering on ReasonTabQA 1.0 (Overall)

67.58Overall Accuracy

Gemini-3-Pro-Preview

Updated 1mo ago

Evaluation Results

Method
Gemini-3-Pro-Preview 2026.01	67.58	68.44	66.96	74.64	69.27	64.8	70.15	65.85	62.17
Claude-Opus-4.5 2026.01	66.2	66.84	65.74	72.67	69.18	62.95	68.36	64.89	61.36
Doubao-1.5-thinking-pro 2026.01	64.48	59.5	68.06	72.79	71.84	58.57	65.33	63.79	62.93
Claude-4.0-Sonnet 2026.01	62.53	60	64.35	70.69	61.35	60.86	64.14	61.56	58.91
Qwen3-8B-Think-SFT-TabCodeRL 2026.01	61.89	57.92	64.74	68.49	61.99	60.02	63.89	61.55	55.63
Kimi-K2-Instruct 2026.01	59.57	54.19	63.43	62.43	60.09	58.52	63.13	56.17	54.11
GPT-5.2 2026.01	59.46	61.29	58.14	64.79	61.25	57.11	66.29	53.81	47.23
Qwen3-32B (thinking) 2026.01	58.76	56.77	60.19	63.79	60.4	56.57	63.13	54.78	51.68
Qwen3-8B-NoThink-SFT-TabCodeRL 2026.01	58.01	54.22	60.73	61.59	59.4	56.34	60.69	56.92	50.92
GPT-4o 2026.01	56.9	54.25	58.8	60.9	59.01	54.76	63.15	51.69	45.79
Deepseek-R1 2026.01	55.8	55.11	56.3	58.9	58.11	53.81	58.51	52.59	52.91
OpenAI o1-mini 2026.01	55.8	47.74	61.57	61.72	55.63	54.25	61.62	50.5	46.39
QWQ-32B 2026.01	54.1	55.48	53.11	61.9	56.99	50.53	57.7	53.71	42.39
DeepSeek-R1-Dist-Qwen-32B 2026.01	53.64	54.84	52.78	58.45	55.09	51.6	58.59	48.4	47.12
Qwen3-30B-A3B (thinking) 2026.01	53.61	52	54.77	59.81	56.32	50.57	54.49	53.79	50.18
Qwen3-32B (no-thinking) 2026.01	53.13	49.03	56.07	57.21	52.79	52.17	56.85	51.12	44.3
Deepseek-V3 2026.01	52.15	54.81	50.24	57.12	53.66	50.04	55.07	51.12	44.1
Qwen2.5-72B-Instruct 2026.01	51.92	53.55	50.75	61.42	52.32	49.1	55.1	50.98	42.79
Telechat3-36B 2026.01	51.13	53.51	49.42	58.49	50.98	49.17	54.12	51.99	38.99
Qwen3-8B (thinking) 2026.01	49.87	43.23	54.63	62.07	49.69	46.59	56.06	44.58	39.12
Mistral-Large-Instruct-2407 2026.01	47.17	47.1	47.22	62.07	46.6	43.33	53.03	42.24	36.84
Qwen2.5-Coder-32B-Instruct 2026.01	45.28	47.1	43.98	58.62	37.86	45.24	50	36.21	47.37
TableGPT2-7B 2026.01	42.05	45.16	39.81	53.45	40.86	39.48	45.96	37.07	38.6
Qwen3-8B (no-thinking) 2026.01	40.97	43.23	39.35	52.07	41.55	37.62	44.95	40.48	28.14
DeepSeek-R1-Dist-Qwen-14B 2026.01	39.89	36.13	42.59	44.83	39.83	38.55	43.43	37.05	33.35
Llama-3.3-70B-Instruct 2026.01	37.74	38.1	37.48	46.55	39.81	34.29	38.89	40.52	28.07
Qwen2-72B-Instruct 2026.01	36.66	40	34.26	56.9	32.04	33.33	39.39	31.03	38.6
DS-R1-Dist-Qwen-7B-SFT-TabCodeRL 2026.01	34.46	35.57	33.66	36.52	34.44	33.9	37.42	31.97	29.25
Llama-3.1-8B-Instruct 2026.01	33.24	36.23	31.09	39.12	32.41	32.02	37.19	32.12	21.8
TableLlama-7B-SFT-TabCodeRL 2026.01	20.43	23.83	17.99	21.92	21.26	19.61	22.71	19.15	15.11
TableLLM-7B 2026.01	13.5	15.51	12.06	17.49	14.15	12.08	14.12	12.93	12.51
TableLlama-7B 2026.01	13.23	17.7	10.02	15.58	14.12	12.14	15.12	13.54	6.03
DeepSeek-R1-Dist-Qwen-7B 2026.01	11.86	16.13	8.8	13.9	11.68	11.39	17.68	6.03	3.51