Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents

About

Training deep search agents requires verifiable questions whose answers remain unavailable until sufficient evidence has been acquired through search. Existing synthesis methods often increase apparent difficulty by enriching graph structures, but structural complexity alone does not guarantee realized search difficulty: the intended search process can collapse through a cheaper identifying route. We formalize this gap with a shortcut-aware difficulty framework and identify four actionable shortcut risks: evidence co-coverage, single-clue selectivity, exposed constants, and prior-knowledge binding. To diagnose their realized effects, we use trajectory signatures including solving cost, answer hit time, and prior-shortcut rate. Guided by this framework, we introduce FORT, a Framework of Shortcut-Resistant Training-Data Synthesis. FORT constructs shortcut-resistant training data by controlling shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. Experiments show that FORT induces longer pre-answer search and fewer shortcut patterns than existing open-source deep search datasets. Using the resulting trajectories, we train FORT-Searcher with supervised fine-tuning (SFT) only, and it achieves the best overall performance among comparable-size open-source search agents on challenging deep search benchmarks. Relevant resources will be made available at https://github.com/RUCAIBox/FORT-Searcher.

Jia Deng, Yimeng Chen, Xiaoqing Xiang, Ziyang Zeng, Shuo Tang, Wayne Xin Zhao, Feng Chang, Chuan Hao, Yuan Wei, Ran Tao, Bryan Dai, Ji-Rong Wen• 2026

Related benchmarks

TaskDatasetResultRank
Web SearchBrowsecomp
Score72.2
18
Web SearchBrowseComp-ZH
Score75
13
Web SearchSEAL 0
Score46
7
Deep Search and Information Retrievalxbench DeepSearch 2510--
7
Web SearchXbench DeepSearch 2505
Score80.8
6
Open-domain web search trajectory diagnosisFORT
Omega141
1
Open-domain web search trajectory diagnosisInfoSeek--
1
Open-domain web search trajectory diagnosisMiroVerse Voyager--
1
Open-domain web search trajectory diagnosisDeepDive--
1
Open-domain web search trajectory diagnosisDeepResearch-9K--
1
Showing 10 of 12 rows

Other info

GitHub

Follow for update