An Answer is just the Start: Related Insight Generation for Open-Ended Document-Grounded QA
About
Answering open-ended questions remains challenging for AI systems because it requires synthesis, judgment, and exploration beyond factual retrieval, and users often refine answers through multiple iterations rather than accepting a single response. Existing QA benchmarks do not explicitly support this refinement process. To address this gap, we introduce a new task, document-grounded related insight generation, where the goal is to generate additional insights from a document collection that help improve, extend, or rethink an initial answer to an open-ended question, ultimately supporting richer user interaction and a better overall question answering experience. We curate and release SCOpE-QA (Scientific Collections for Open-Ended QA), a dataset of 3,000 open-ended questions across 20 research collections. We present InsightGen, a two-stage approach that first constructs a thematic representation of the document collection using clustering, and then selects related context based on neighborhood selection from the thematic graph to generate diverse and relevant insights using LLMs. Extensive evaluation on 3,000 questions using two generation models and two evaluation settings shows that InsightGen consistently produces useful, relevant, and actionable insights, establishing a strong baseline for this new task.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Insight Generation | Internal document collections non-scientific domain Legal Business Analysis 1.0 (test) | Insight Score4.71 | 20 | |
| Insight Generation | Internal document collections Instagram Marketing | Insight Level Score4.76 | 20 | |
| Insight Generation | Internal document collections (non-scientific domain) - Climate Change Awareness | Insight Score4.72 | 20 | |
| Insight Generation | Internal Document Collections Climate Change Policy | Insight-level Score4.34 | 20 | |
| Insight Generation | Internal document collections - Gut Health Insights non-scientific domain | Insight Level Score4.5 | 20 | |
| Insight Generation | Internal document collections (non-scientific domain) Finance | Insight Level Score4.67 | 20 | |
| Insight Generation | Internal document collections non-scientific domain Finance Investment 2 | Insight Score4.67 | 20 | |
| Insight Generation | Internal document collections Legal & Regulatory Compliance (non-scientific domain) | Insight Score4.45 | 20 | |
| Insight Generation | Internal document collections non-scientific domain Finance Investment 3 | Insight Score4.73 | 20 | |
| Insight Generation | Internal document collections non-scientific domain Hotel Sales Strategies | Insight Level Score4.52 | 20 |