StatefulDiscovery: Evidence-Calibrated Claim Formation in Open-Ended Scientific Discovery
About
Open-ended scientific discovery asks agents to move beyond executing analyses for predefined questions. Across multiple rounds of exploration, a discovery agent must decide which phenomena warrant investigation while avoiding overinterpretation, where emerging claims exceed the evidential scope of the analyses supporting them. This creates an evidence-calibration problem: the exploration trajectory must be coupled with claim status so that evidence can guide both what to investigate next and what can be claimed. We introduce StatefulDiscovery, a discovery framework that externalizes investigation state and uses it to coordinate frontier selection, evidence acquisition, and claim adjudication. We evaluate StatefulDiscovery across 40 real-data discovery tasks. Compared with several baselines, StatefulDiscovery produces more claims overall judged to be both well-supported and high-value. Ablations indicate that structured hypotheses, local adjudication, and frontier control contribute to performance. Together, these results suggest that explicit discovery state can couple exploration with evidence-calibrated claim formation.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Open-ended scientific discovery | BixBench open-ended discovery (evaluation set) | ES Score3.75 | 5 | |
| Open-ended scientific discovery | BLADE open-ended discovery (evaluation set) | Expert Score3.95 | 5 | |
| Open-ended scientific discovery | DiscoveryBench open-ended discovery (evaluation set) | ES3.41 | 5 | |
| Open-ended scientific discovery | 40-task evaluation set Overall 2025 | Expert Score (ES)3.76 | 5 | |
| Query-guided evaluation | BixBench V50 | V50 Accuracy80 | 4 | |
| Scientific Discovery Pairwise Preference | Shared 40-task (evaluation) | Win Count40 | 4 | |
| Query-guided evaluation | HeurekaBench | G-Eval Score3.41 | 4 |