Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization

About

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in parsing textual data and generating code. However, their performance in tasks involving tabular data, especially those requiring symbolic reasoning, faces challenges due to the structural variance and inconsistency in table cell values often found in web tables. In this paper, we introduce NormTab, a novel framework aimed at enhancing the symbolic reasoning performance of LLMs by normalizing web tables. We study table normalization as a stand-alone, one-time preprocessing step using LLMs to support symbolic reasoning on tabular data. Our experimental evaluation, conducted on challenging web table datasets such as WikiTableQuestion and TabFact, demonstrates that leveraging NormTab significantly improves symbolic reasoning performance, showcasing the importance and effectiveness of web table normalization for enhancing LLM-based symbolic reasoning tasks.

Md Mahadi Hasan Nahid, Davood Rafiei• 2024

Related benchmarks

TaskDatasetResultRank
Table Question AnsweringWikiTQ (test)
Accuracy61.2
92
Table Question AnsweringWikiTQ
F1 Score65.91
50
Table Question AnsweringSequentialQA
F1 Score50
50
Table Question AnsweringNQ-Table
F1 Score55.4
50
Table Question AnsweringHiTab
F1 Score56.81
50
Table-based Fact VerificationTabFact
Accuracy68.9
33
Table Question AnsweringNQ-Table (Challenge-set)
F1 Score66.84
10
Table Question AnsweringWikiTQ (challenge-set)
F1 Score62.28
10
Showing 8 of 8 rows

Other info

Code

Follow for update