Share your thoughts, 1 month free Claude Pro on usSee more

Multi-hop Question Answering on HotpotQA (EM, F1, Rate)

55Exact Match (EM)

BRIEF-PRO-HIGH

Updated 1mo ago

Evaluation Results

Method
BRIEF-PRO-HIGH 2025.10	55	67.42	72
BRIEF-PRO-MEDIUM 2025.10	54.5	68.02	42
BRIEF-PRO-LOW 2025.10	54.5	68.35	26
BRIEF-PRO-AUTO 2025.10	53	67.29	34
BRIEF-PRO-MEDIUM 2025.10	52	65.38	42
Non-compression 2025.10	51.5	65.14	1
BRIEF-PRO-AUTO 2025.10	51.5	65.52	34
BRIEF-PRO-MEDIUM 2025.10	51	63.41	42
BRIEF-PRO-HIGH 2025.10	51	64.13	72
BRIEF-PRO-LOW 2025.10	51	64.86	26
BRIEF-PRO-LOW 2025.10	50.5	64.36	26
Rerank Top-5 2025.10	50	64.45	1.5
BRIEF-PRO-AUTO 2025.10	49	63.05	34
FILM-7B 2025.10	47.5	61.43	1
BRIEF-PRO-AUTO_L7C 2025.10	47.5	60.97	13
BRIEF-PRO-HIGH 2025.10	47	58.85	72
EXIT 2025.10	46	59.8	32
LongLLMLingua 2025.10	46	61.32	8
BRIEF-PRO-AUTO_L7C 2025.10	46	56.06	13
Rerank Top-3 2025.10	43	55.78	2.2
ProLong-8B 2025.10	42.5	54.43	1
BRIEF-PRO-AUTO_L7C 2025.10	42.5	56.17	13
Non-compression 2025.10	42.5	56.56	1
Rerank Top-5 2025.10	42	55.16	1.5
GPT-4.1-nano 2025.10	42	56.59	90
EXIT 2025.10	42	54.41	32
RECOMP (Extractive) 2025.10	41	52.44	36
Rerank Top-5 2025.10	41	54.87	1.5
LongLLMLingua 2025.10	40.5	54.9	8
GPT-4.1-nano 2025.10	40.5	55.87	90
Non-compression 2025.10	40	54.29	1
RECOMP (Extractive) 2025.10	40	53.27	36
Rerank Top-3 2025.10	40	52.85	2.2
LongLLMLingua 2025.10	38.5	53.63	8
GPT-4.1-nano 2025.10	38	53.67	90
Rerank Top-3 2025.10	37.5	50.07	2.2
RECOMP (Abstractive) 2025.10	37.5	50.47	10
EXIT 2025.10	36.5	48.62	32
RECOMP (Extractive) 2025.10	35	48.48	36
Llama-3.2-3B-Instruct 2025.10	34.5	47.26	41
Llama-3.2-3B-Instruct 2025.10	33	44.72	41
Rerank Top-1 2025.10	32	44.42	8
Llama-3.2-3B-Instruct 2025.10	30	41.67	41
Rerank Top-1 2025.10	30	40.85	8
RECOMP (Abstractive) 2025.10	28.5	40.59	10
RECOMP (Abstractive) 2025.10	27	39.21	10
BRIEF 2025.10	25.5	34.75	59
BRIEF 2025.10	25	36.03	59
Rerank Top-1 2025.10	23.5	35.66	8
BRIEF 2025.10	18	27.7	59