epbench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Episodic Memory | EpBench 200-Chapters Book (test) | Average Cost ($)0.009 | 6 | |
| Episodic Memory Retrieval | Epbench 200-Chapters Book (Overall) | Precision86.5 | 6 | |
| Episodic Memory Retrieval | Epbench 6+ Cues 200-Chapters Book | Precision94 | 6 | |
| Episodic Memory Retrieval | Epbench 3-5 Cues 200-Chapters Book | Precision87.8 | 6 | |
| Episodic Memory Retrieval | Epbench 2 Cues 200-Chapters Book | Precision81.7 | 6 | |
| Episodic Memory Retrieval | Epbench Chapters Book 200 (1 Cue) | Precision75.5 | 6 | |
| Episodic Memory Retrieval | Epbench 0 Cues 200-Chapters Book | Precision97.8 | 6 | |
| Long-context Question Answering | epbench ep_news | F1 Score51.23 | 6 | |
| Long-context Question Answering | epbench ep_scifi | F1 Score52.04 | 6 | |
| Long-context Question Answering | epbench ep_default | F1 Score53.25 | 6 | |
| Episodic Memory Recall | Epbench Book 2000-Chapters (Overall) | Precision83 | 5 | |
| Episodic Memory Recall | Epbench 6+ Cues 2000-Chapters | Precision91.1 | 5 | |
| Episodic Memory Recall | Epbench 3-5 Cues 2000-Chapters | Precision84.1 | 5 | |
| Episodic Memory Recall | Epbench 2 Cues 2000-Chapters | Precision84.5 | 5 | |
| Episodic Memory Recall | Epbench 2000-Chapters Book 1 Cue | Precision76.1 | 5 | |
| Episodic Memory Recall | Epbench 0 Cues 2000-Chapters Book | Precision94.3 | 5 | |
| Episodic Memory | Epbench 2000-Chapters Book (test) | Precision83 | 5 |
Showing 17 of 17 rows