Share your thoughts, 1 month free Claude Pro on usSee more

Multi-hop Question Answering on MuSiQue (EM, F1, Rate)

40EM

BRIEF-PRO-LOW

Updated 1mo ago

Evaluation Results

Method
BRIEF-PRO-LOW 2025.10	40	51.95	26
BRIEF-PRO-AUTO 2025.10	38.5	48.84	35
BRIEF-PRO-HIGH 2025.10	38	45.55	84
BRIEF-PRO-MEDIUM 2025.10	38	49.17	47
Non-compression 2025.10	36	45.47	1
BRIEF-PRO-AUTO_L7C 2025.10	36	46.4	13
BRIEF-PRO-LOW 2025.10	33	42.96	26
GPT-4.1-nano 2025.10	32	41.46	128
LongLLMLingua 2025.10	32	42.35	10
GPT-4.1-nano 2025.10	31.5	41.24	128
BRIEF-PRO-HIGH 2025.10	31.5	38.7	84
BRIEF-PRO-LOW 2025.10	30.5	39.68	26
BRIEF-PRO-MEDIUM 2025.10	30.5	41.15	47
Rerank Top-5 2025.10	30	39.06	1.7
RECOMP (Extractive) 2025.10	29.5	37.65	41
BRIEF-PRO-AUTO 2025.10	29.5	41.1	35
GPT-4.1-nano 2025.10	28.5	40.33	128
BRIEF-PRO-AUTO 2025.10	27.5	36.64	35
BRIEF-PRO-MEDIUM 2025.10	27.5	35.55	47
FILM-7B 2025.10	26.5	36.45	1
Rerank Top-3 2025.10	26	35.32	2.5
RECOMP (Abstractive) 2025.10	25	33.8	10
Llama-3.2-3B-Instruct 2025.10	24.5	30.66	48
BRIEF-PRO-HIGH 2025.10	24	31.7	84
Non-compression 2025.10	24	33.08	1
EXIT 2025.10	23	30.85	35
BRIEF-PRO-AUTO_L7C 2025.10	22.5	32.44	13
Rerank Top-3 2025.10	22.5	30.73	2.5
Rerank Top-5 2025.10	21	29.02	1.7
Non-compression 2025.10	20.5	28.88	1
Rerank Top-5 2025.10	20.5	28.03	1.7
LongLLMLingua 2025.10	20.5	28.75	10
LongLLMLingua 2025.10	20.5	28.55	10
BRIEF-PRO-AUTO_L7C 2025.10	20.5	31.58	13
ProLong-8B 2025.10	19.5	27.42	1
RECOMP (Extractive) 2025.10	19.5	27.32	41
Rerank Top-1 2025.10	19	26.5	7
RECOMP (Abstractive) 2025.10	17.5	25.92	10
Llama-3.2-3B-Instruct 2025.10	17.5	23.6	48
EXIT 2025.10	16.5	24.33	35
RECOMP (Extractive) 2025.10	16	23.39	41
Rerank Top-3 2025.10	15.5	23.4	2.5
Llama-3.2-3B-Instruct 2025.10	13	18.48	48
RECOMP (Abstractive) 2025.10	13	19.13	10
BRIEF 2025.10	11.5	18.74	22
EXIT 2025.10	11	17.19	35
Rerank Top-1 2025.10	11	17.99	7
Rerank Top-1 2025.10	9.5	14.58	7
BRIEF 2025.10	6	13.55	22
BRIEF 2025.10	5	12.39	22