Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images

About

Visual reasoning over structured data such as tables is a critical capability for modern vision-language models (VLMs), yet current benchmarks remain limited in scale, diversity, or reasoning depth, especially when it comes to rendered table images. Addressing this gap, we introduce Visual-TableQA, a large-scale, open-domain multimodal dataset specifically designed to evaluate and enhance visual reasoning over complex tabular data. Our generation pipeline is modular, scalable, and fully autonomous, involving multiple reasoning LLMs collaborating across distinct roles: generation, validation, and inspiration. Visual-TableQA comprises 2.5k richly structured LaTeX-rendered tables and 6k reasoning-intensive QA pairs, all produced at a cost of under USD 100. To promote diversity and creativity, our pipeline performs multi-model collaborative data generation via cross-model prompting ('inspiration') and LLM-jury filtering. Stronger models seed layouts and topics that weaker models elaborate, collectively distilling diverse reasoning patterns and visual structures into the dataset. Empirical results show that models fine-tuned on Visual-TableQA generalize robustly to external benchmarks, outperforming several proprietary models despite the dataset's synthetic nature. The full pipeline and resources are publicly available at https://github.com/AI-4-Everyone/Visual-TableQA.

Boammani Aser Lompo, Marc Haraoui• 2025

Related benchmarks

TaskDatasetResultRank
Chart Question AnsweringChartQA (test)--
196
Mathematical Visual Question AnsweringMATH-Vision Full (test)
Relaxed Accuracy49.77
12
Table Question AnsweringReachQA (test)
Relaxed Accuracy60.95
12
Visual Table Question AnsweringVisual-TableQA (test)
Relaxed Accuracy82.98
11
Table Question AnsweringVisual-TableQA-CIT (test)--
8
Visual Question AnsweringTableVQA-Bench v1 (test)
VTabFact82
6
Visual Question AnsweringTable-VQA v1 (test)
Accuracy79.03
4
Showing 7 of 7 rows

Other info

GitHub

Follow for update