Share your thoughts, 1 month free Claude Pro on usSee more

Evidence Retrieval on Synthetic Benchmark 1.0 (test)

22.5Hit@2

OSU-Mem + GF

Updated 25d ago

Evaluation Results

Method
OSU-Mem + GF 2026.06	22.5	36.7
OSU-Mem 2026.06	22.2	36.4
OSU-Mem (no novelty) 2026.06	20.9	36.1
Flat + MMR 2026.06	15.2	28.8
Flat Vector 2026.06	14.7	28.4
Coverage-Aware 2026.06	14.7	28.9
Disjoint Hierarchy 2026.06	14.5	28.2
OSU-Mem (disjoint) 2026.06	13.4	27.7
Tree Abstraction 2026.06	12.3	27.5
OSU-Mem + GF 2026.06	8.8	22.2
OSU-Mem 2026.06	8.4	22.1
OSU-Mem (sim only) 2026.06	8.3	21.4
OSU-Mem (no novelty) 2026.06	8.3	21.7
Periodic Summary 2026.06	7.3	22.1
Flat + MMR 2026.06	5.2	15.8
Coverage-Aware 2026.06	4.9	15.7
Flat Vector 2026.06	4.5	15.6
Disjoint Hierarchy 2026.06	4.4	15
Tree Abstraction 2026.06	4.4	15.7
OSU-Mem (disjoint) 2026.06	3.4	13.8
OSU-Mem + GF 2026.06	2.9	13
OSU-Mem 2026.06	2.7	13
OSU-Mem (no novelty) 2026.06	2.7	13
Flat + MMR 2026.06	2.2	8.5
Flat Vector 2026.06	1.6	8.8
Coverage-Aware 2026.06	1.4	8.6
Disjoint Hierarchy 2026.06	1.4	8.4
OSU-Mem (sim only) 2026.06	1.4	11
Periodic Summary 2026.06	1.3	9.3
Tree Abstraction 2026.06	1.2	9.3
OSU-Mem 2026.06	1	7.6
OSU-Mem + GF 2026.06	1	7.7
OSU-Mem (no novelty) 2026.06	1	7.6
Budget-Trunc Replay 2026.06	0.8	32.1
OSU-Mem (disjoint) 2026.06	0.8	7.5
OSU-Mem (sim only) 2026.06	0.8	6.5
Coverage-Aware 2026.06	0.6	4.4
Sliding Window 2026.06	0.5	31.4
Flat Vector 2026.06	0.5	5.1
Flat + MMR 2026.06	0.5	4.5
Disjoint Hierarchy 2026.06	0.4	4.7
OSU-Mem (disjoint) 2026.06	0.3	4.7
Tree Abstraction 2026.06	0.2	5.5
OSU-Mem (sim only) 2026.06	0.2	3.7
No LT Memory 2026.06	0	0
No LT Memory 2026.06	0	0
No LT Memory 2026.06	0	0
No LT Memory 2026.06	0	0
Budget-Trunc Replay 2026.06	0	0.4
Budget-Trunc Replay 2026.06	0	0.4
Budget-Trunc Replay 2026.06	0	14.9
Sliding Window 2026.06	0	0
Sliding Window 2026.06	0	0
Sliding Window 2026.06	0	13.6
Periodic Summary 2026.06	0	0.3
Periodic Summary 2026.06	0	4.1