Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checking

About

Fact-checking systems built on LLMs achieve high verdict accuracy on standard benchmarks, yet routinely output Supports labels whose cited evidence does not license the claim. Structured decomposition is the natural way to inspect those warrants, but rigid extraction protocols strip the full-claim context that facets need. We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim -- paired with WSP (Warranted Supports Proportion), an automatic NLI check that the cited warrant entails the claim. We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones. SIFT recovers accuracy on cells where naive decomposition costs up to 27.6 points, while raising WSP above direct prompting; WSP itself calibrates against human gold evidence at AUC 0.92 and precision 0.98.

Arka Ujjal Dey, John Collomosse• 2026

Related benchmarks

TaskDatasetResultRank
Fact Verification5PILS (test)
Accuracy79.6
21
Fact VerificationDP (test)
Accuracy69
21
Fact VerificationSciFact (test)
Accuracy84
21
Fact VerificationFEVER held-out (test)
Accuracy94.7
21
Showing 4 of 4 rows

Other info

Follow for update