Retrieval-Augmented Generation on LOFT

100NQ Score

MiniLM

Updated 4d ago

Evaluation Results

Method
MiniLM 2025.01	100	98	82	-
MiniLM 2025.01	100	100	90	-
MiniLM 2025.01	100	100	92	-
TinyBERT 2025.01	99	98	80	-
TinyBERT 2025.01	99	99	81	-
TinyBERT 2025.01	99	99	82	-
SFT-RTA w/ RAP 2025.01	95	99	100	-
Phi-3-7B-128K 2025.01	90	85	63	79
Mistral-2-7B-32K 2025.01	89	81	41	70
Oracle RAG 2025.01	89	81	41	-
Qwen-2-7B-32K 2025.01	88	81	56	75
GPT-4-turbo 2025.01	88	87	72	82
GPT-4 Oracle 2025.01	88	87	72	-
Mistral-7B Re-ranking (TinyBERT) 2025.01	88	78	29	-
Mistral-7B Re-ranking (TinyBERT) 2025.01	87	87	33	-
GPT-4-turbo 2025.01	85	78	51	71
GPT-4 Vanilla RAG 2025.01	85	78	51	-
SFT-RTA w/ RAP 2025.01	85	79	39	-
Mistral-7B Re-ranking (MiniLM) 2025.01	84	84	39	-
Mistral-7B Re-ranking (MiniLM) 2025.01	83	77	27	-
LLaMA-3-instruct-8B 2025.01	81	77	60	73
Qwen-2-1.5B-32K 2025.01	80	76	44	67
Qwen-2-7B-32K 2025.01	79	61	29	56
SFT-DA w/ RAP 2025.01	78	76	47	-
Phi-3-7B-128K 2025.01	76	68	41	62
SFT-Cite-Context-ID 2025.01	76	54	35	-
SFT-Retrieve-then-Answer 2025.01	74	69	33	-
SFT-Direct Answer 2025.01	70	65	25	-
Mistral-2-7B-32K 2025.01	64	62	27	51
Mistral-7B Vanilla RAG 2025.01	64	62	27	-
Qwen-2-1.5B-32K 2025.01	61	43	12	39
Mistral-7B RAG w/ RTA Prompting 2025.01	60	70	27	-
GPT-4-turbo 2025.01	58	57	28	48
GPT-4 Close-book 2025.01	58	57	28	-
Mistral-2-7B-32K 2025.01	46	38	5	30
Mistral-7B Close-book 2025.01	46	38	5	-
LLaMA-3-instruct-8B 2025.01	45	56	16	39
LLaMA-3-instruct-8B 2025.01	41	19	1	20
Phi-3-7B-128K 2025.01	39	35	9	28
Qwen-2-7B-32K 2025.01	31	30	7	23
Qwen-2-1.5B-32K 2025.01	21	22	3	15
RetHead w/ Lgen + Lret 2025.01	15	13	7	-