Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on NarrativeQA (EM, F1, Rate)

15EM

Non-compression

Updated 1mo ago

Evaluation Results

Method	Links
Non-compression 2025.10		15	35.2	1
ProLong-8B 2025.10		14	31.63	1
Non-compression 2025.10		10.5	28.25	1
FILM-7B 2025.10		10	28.47	1
BRIEF-PRO-AUTO 2025.10		8.5	19.53	32
BRIEF-PRO-LOW 2025.10		8.5	20.85	40
EXIT 2025.10		8	19.7	40
LongLLMLingua 2025.10		8	21.09	17
BRIEF-PRO-AUTO 2025.10		8	22.83	32
Non-compression 2025.10		7.5	29.43	1
Rerank Top-3 2025.10		7.5	21.62	3
GPT-4.1-nano 2025.10		7.5	22.91	91
BRIEF-PRO-LOW 2025.10		7.5	20.71	40
EXIT 2025.10		7.5	18.45	40
Rerank Top-3 2025.10		7.5	19.1	3
GPT-4.1-nano 2025.10		7.5	24.5	91
BRIEF-PRO-HIGH 2025.10		7.5	18.88	97
BRIEF-PRO-MEDIUM 2025.10		7.5	19.25	66
BRIEF-PRO-LOW 2025.10		7	21.84	40
Llama-3.2-3B-Instruct 2025.10		7	19.95	45
BRIEF-PRO-HIGH 2025.10		7	18.65	97
GPT-4.1-nano 2025.10		6.5	24.46	91
BRIEF-PRO-HIGH 2025.10		6.5	19.1	97
BRIEF-PRO-MEDIUM 2025.10		6.5	20.4	66
BRIEF-PRO-MEDIUM 2025.10		6.5	20.02	66
RECOMP (Extractive) 2025.10		6	16.62	94
EXIT 2025.10		5.5	20.11	40
Llama-3.2-3B-Instruct 2025.10		5.5	19.81	45
BRIEF-PRO-AUTO 2025.10		5.5	21.73	32
RECOMP (Abstractive) 2025.10		5.5	17.16	17
RECOMP (Abstractive) 2025.10		5.5	14.01	17
Llama-3.2-3B-Instruct 2025.10		5.5	17.15	45
Rerank Top-3 2025.10		5	20.06	3
RECOMP (Extractive) 2025.10		5	15.62	94
RECOMP (Extractive) 2025.10		4.5	16.19	94
LongLLMLingua 2025.10		4.5	21.56	17
LongLLMLingua 2025.10		4	18.24	17
RECOMP (Abstractive) 2025.10		3.5	13.69	17