Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Freshness and the Limits of Heuristic Trend Detection in Temporal RAG

About

We present a lightweight, model-agnostic temporal layer for RAG and use cybersecurity data to separate two problems that are usually conflated. For freshness, a half-life recency prior surfaces the newest relevant item where a cosine-only baseline scores 0.00; on a hard NVD CVE test, where the freshest item is not the most similar, it reaches Latest@10 of 0.60 versus 0.20 for a semantic-then-newest baseline, but stays partial and parameter-sensitive. For topic evolution, a heuristic tracker's low 0.08 macro-F1 is driven by the labeling rule, not the clusterer (HDBSCAN: 0.10; fixing the rule alone reaches 0.49, and 0.96 without clustering noise). We contribute a reproducible decoupling of the two, with honest real-data scope and a reference implementation.

Matthew Grofsky• 2025

Related benchmarks

TaskDatasetResultRank
Document RankingNVD CVE N=5 queries
Latest@1060
4
Freshness RetrievalCERT 849k (test)
Latest-Set@10100
4
Freshness RetrievalNVD CVE (test)
Latest@1060
4
Freshness RetrievalSynthetic
Latest@10100
2
Freshness RetrievalCERT
Latest-Set@101
2
Freshness RetrievalNVD CVE
Latest@1060
2
Temporal CorrectnessSynthetic
As-of Correctness100
1
Trend DetectionSynthetic
Trend F1 (per-week-delta rule)8
1
Showing 8 of 8 rows

Other info

Follow for update