Retrieval-Augmented Generation on ICR2

87NQ Score

Phi-3-7B-128K

Updated 4d ago

Evaluation Results

Method
Phi-3-7B-128K 2025.01	87	81	91	14	68
Qwen-2-7B-32K 2025.01	83	80	91	17	68
Mistral-2-7B-32K 2025.01	83	81	94	18	69
Oracle RAG 2025.01	83	81	94	18	-
LLaMA-3-instruct-8B 2025.01	80	74	86	19	65
GPT-4-turbo 2025.01	79	82	94	18	68
GPT-4 Oracle 2025.01	79	82	94	18	-
Qwen-2-1.5B-32K 2025.01	77	65	86	15	61
GPT-4-turbo 2025.01	67	64	92	15	59
GPT-4 Vanilla RAG 2025.01	67	64	92	15	-
SFT-DA w/ RAP 2025.01	64	67	89	21	-
SFT-Cite-Context-ID 2025.01	63	63	89	21	-
SFT-RTA w/ RAP 2025.01	63	71	92	23	-
Mistral-7B Re-ranking (TinyBERT) 2025.01	62	51	91	13	-
GPT-4-turbo 2025.01	61	50	85	16	53
GPT-4 Close-book 2025.01	61	50	85	16	-
Mistral-7B Re-ranking (MiniLM) 2025.01	61	47	92	12	-
Phi-3-7B-128K 2025.01	60	50	90	14	53
SFT-Retrieve-then-Answer 2025.01	60	67	91	22	-
SFT-Direct Answer 2025.01	59	70	90	22	-
Qwen-2-7B-32K 2025.01	57	47	88	13	51
Mistral-7B RAG w/ RTA Prompting 2025.01	54	51	83	15	-
Mistral-2-7B-32K 2025.01	53	50	81	13	49
LLaMA-3-instruct-8B 2025.01	53	45	83	15	49
Mistral-7B Vanilla RAG 2025.01	53	50	81	13	-
Mistral-7B Re-ranking (TinyBERT) 2025.01	52	47	88	12	-
Mistral-7B Re-ranking (MiniLM) 2025.01	51	46	86	13	-
Mistral-2-7B-32K 2025.01	49	38	73	14	43
Mistral-7B Close-book 2025.01	49	38	73	14	-
RetHead w/ Lgen + Lret 2025.01	48	54	90	21	-
Phi-3-7B-128K 2025.01	46	32	73	13	41
LLaMA-3-instruct-8B 2025.01	44	19	53	14	32
RetHead w/ Lret 2025.01	39	44	82	13	-
Qwen-2-7B-32K 2025.01	35	30	71	15	38
Qwen-2-1.5B-32K 2025.01	35	32	78	12	39
Qwen-2-1.5B-32K 2025.01	29	24	62	15	32
RetHead w/ Lgen 2025.01	28	25	82	18	-