Share your thoughts, 1 month free Claude Pro on usSee more

Multilingual Factual Reasoning on mTruthfulQA

54.46Score (ar)

LLaMA3.1-8B-AdaMCOT

Updated 4d ago

Evaluation Results

Method	Links
LLaMA3.1-8B-AdaMCOT 2025.01		54.46	49.68	56.11	50.7	57.49	59.24	57.41	48.45	58.83	49.25	52.78	57.94	68.04	36.35	59.64	57.09	49.12	45.24	50.39	51.81	57.83	71.45	57.25	56.6	60.03	56.61	56.2	45.63	45.82	58.05	61.53	59.14
Qwen2.5-7B 2025.01		52.78	42.25	51.87	50.32	51.27	60.59	54.88	20.8	55.53	26.81	44.11	45.29	46.04	18.44	59.13	56.07	29.06	25.07	32.2	30.62	57.2	61.17	52.5	56.98	46.02	46.08	56.33	23.01	22.55	52.6	54.9	59.77
Qwen2.5-7B-AdaMCOT 2025.01		47.35	44.81	52.64	51.34	55.33	62.55	56.4	37.21	58.83	31.33	50.06	54.06	69.95	31.65	52.44	56.32	36.43	30.26	37.96	42.38	55.8	76.93	55.2	60.15	56.04	54.17	54.13	31.22	32.77	58.7	47.52	58.5
LLaMA3.1-8B 2025.01		46.96	36.11	51.09	48.78	51.9	57.16	54.37	32.56	53.62	48.7	43.73	49.94	47.86	32.01	46.92	54.15	48.38	44.81	47.77	43.41	51.21	57.87	53.53	44.16	48.97	48.78	53.1	49.39	42.84	48.57	52.48	50.13
Qwen2.5-7B-QAlign 2025.01		45.8	27.14	46.07	47.25	52.66	59.49	52.09	15.76	52.86	17.24	33.59	41.88	35.15	14.31	52.31	51.85	16.96	15.71	22.38	21.73	52.74	54.06	47.37	50.13	40.49	43.57	48.32	16.55	18.87	44.29	47.39	55.39
Qwen2.5-7B-AutoCAP 2025.01		44.18	33.94	45.32	43.75	44.83	53.46	47.02	16.87	50.34	22.15	35.09	42.15	36.72	19.95	50.13	50.86	23.57	21.18	27.25	25.06	50.34	52.61	46.92	50.81	41.67	43.25	48.71	25.9	21.43	45.93	46.66	51.72
LLaMA3.1-8B-QAlign 2025.01		43.47	40.46	40.67	36.62	43.4	38.31	38.53	36.05	42.44	42.68	42.3	38.84	41.63	37.01	42.16	39.34	41.45	46.54	45.81	36.82	41.53	40.61	41.21	40.86	38.05	37.74	42.76	42.66	41.13	38.96	38.85	37.39
LLaMA3.1-8B-AutoCAP 2025.01		41.01	42.89	50.97	50.66	53.05	49.2	50.44	27.26	52.73	43.64	38.42	48.63	43.71	54.79	50.51	49.55	58.85	43.66	30.63	37.73	51.34	52.54	52.76	45.3	48.2	48.28	52.84	48.32	44.54	45.06	51.97	44.04