Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

About

Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations. We present ChartCynics, an agentic dual-path framework designed to unmask visual deception via a "skeptical" reasoning paradigm. Unlike holistic models, ChartCynics decouples perception from verification: a Diagnostic Vision Path captures structural anomalies (e.g., inverted axes) through strategic ROI cropping, while an OCR-Driven Data Path ensures numerical grounding. To resolve cross-modal conflicts, we introduce an Agentic Summarizer optimized via a two-stage protocol: Oracle-Informed SFT for reasoning distillation and Deception-Aware GRPO for adversarial alignment. This pipeline effectively penalizes visual traps and enforces logical consistency. Evaluations on two benchmarks show that ChartCynics achieves 74.43% and 64.55% accuracy, providing an absolute performance boost of ~29% over the Qwen3-VL-8B backbone, outperforming state-of-the-art proprietary models. Our results demonstrate that specialized agentic workflows can grant smaller open-source models superior robustness, establishing a new foundation for trustworthy chart interpretation.

Yanjie Zhang, Yafei Li, Rui Sheng, Zixin Chen, Yanna Lin, Huamin Qu, Lei Chen, Yushi Sun• 2026

Related benchmarks

TaskDatasetResultRank
Chart Question AnsweringMC v1 (test)
Accuracy74.43
11
Chart Question AnsweringCDCC v1 (test)
Accuracy79.09
11
Chart Question AnsweringMSMB Standard N = 122
Accuracy94.26
2
Chart Question AnsweringMSMB Misleading N = 122
Accuracy68.03
2
Chart Question AnsweringMSMB N = 244 (Overall)
Accuracy81.15
2
Showing 5 of 5 rows

Other info

Follow for update