Our new X account is live! Follow @wizwand_team for updates

Conversational Question Answering on CHATRAG BENCH 1.0 (test)

57.14Average Score (w/o HDial)

Llama3-ChatQA-1.5-70B

Updated 4d ago

Evaluation Results

Method	Links
Llama3-ChatQA-1.5-70B 2024.01		57.14	58.25	41.26	38.82	51.4	78.44	50.76	81.88	83.82	55.63	68.27	32.31
GPT-4-Turbo-2024-04-09 2024.01		54.72	54.03	35.35	40.1	51.46	77.73	41.6	84.16	79.98	48.32	47.86	33.75
GPT-4-0613 2024.01		54.35	53.9	34.16	40.29	52.01	77.42	43.39	81.28	79.21	45.09	49.81	36.34
Llama3-ChatQA-1.5-8B 2024.01		53.99	55.17	39.33	39.73	49.03	76.46	49.6	78.46	73.28	49.96	65.76	30.1
ChatQA-1.0-70B 2024.01		53.89	54.14	38.9	41.82	48.05	78.57	51.94	73.69	69.14	50.98	56.44	31.9
Llama3-Instruct-70B 2024.01		52.95	52.52	37.88	36.96	51.34	76.98	41.24	76.6	69.61	49.72	48.59	36.23
Command R+ 2024.01		51.4	50.93	33.51	34.16	49.77	69.71	40.67	71.21	74.07	53.77	46.7	35.76
GPT-3.5-Turbo-0613 2024.01		50.69	50.37	34.83	37.17	50.46	79.33	41.11	73.15	60.63	44.3	47.42	35.27
ChatQA-1.0-7B 2024.01		46.96	47.71	37.88	29.69	46.97	76.61	41.57	51.61	61.87	45.45	54.51	30.96
Llama3-Instruct-8B 2024.01		46.76	46.7	33.59	33.6	45.7	75.26	37.33	58.05	59.72	44.96	46.2	32.59
Llama2-Chat-70B 2024.01		44.64	45.21	36.87	32.47	49.4	80.41	38.97	46.85	37.62	44.31	50.35	34.88
Llama2-Chat-7B 2024.01		37.96	38.86	33.27	25.83	46.02	72.28	33.15	36.58	26.14	36.68	47.02	31.67