Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

StatefulDiscovery: Evidence-Calibrated Claim Formation in Open-Ended Scientific Discovery

About

Open-ended scientific discovery asks agents to move beyond executing analyses for predefined questions. Across multiple rounds of exploration, a discovery agent must decide which phenomena warrant investigation while avoiding overinterpretation, where emerging claims exceed the evidential scope of the analyses supporting them. This creates an evidence-calibration problem: the exploration trajectory must be coupled with claim status so that evidence can guide both what to investigate next and what can be claimed. We introduce StatefulDiscovery, a discovery framework that externalizes investigation state and uses it to coordinate frontier selection, evidence acquisition, and claim adjudication. We evaluate StatefulDiscovery across 40 real-data discovery tasks. Compared with several baselines, StatefulDiscovery produces more claims overall judged to be both well-supported and high-value. Ablations indicate that structured hypotheses, local adjudication, and frontier control contribute to performance. Together, these results suggest that explicit discovery state can couple exploration with evidence-calibrated claim formation.

Jiayao Chen, Shi Liu, Linyi Yang• 2026

Related benchmarks

TaskDatasetResultRank
Open-ended scientific discoveryBixBench open-ended discovery (evaluation set)
ES Score3.75
5
Open-ended scientific discoveryBLADE open-ended discovery (evaluation set)
Expert Score3.95
5
Open-ended scientific discoveryDiscoveryBench open-ended discovery (evaluation set)
ES3.41
5
Open-ended scientific discovery40-task evaluation set Overall 2025
Expert Score (ES)3.76
5
Query-guided evaluationBixBench V50
V50 Accuracy80
4
Scientific Discovery Pairwise PreferenceShared 40-task (evaluation)
Win Count40
4
Query-guided evaluationHeurekaBench
G-Eval Score3.41
4
Showing 7 of 7 rows

Other info

Follow for update