Our new X account is live! Follow @wizwand_team for updates

Opinion Summarization Evaluation on SUMMEVAL-OP Round-I

0.63Relevance (RE) Spearman Rho

OP-MISTRAL

Updated 4d ago

Evaluation Results

Method	Links
OP-MISTRAL 2024.02		0.63	-	-	-	-	0.55	0.59	0.45	0.57	0.46	0.8	0.67	0.6	0.49
OP-I-MISTRAL 2024.02		0.59	0.54	0.45	0.58	0.47	0.47	0.63	0.51	0.82	0.7	0.73	0.61	0.71	0.58
OP-I-GPT-3.5 2024.02		0.56	-	-	0.51	0.69	0.71	0.59	0.8	0.68	0.73	0.61	0.74	0.61	0.71
G-EVAL 3.5 2024.02		0.49	0.45	0.36	0.28	0.52	0.68	0.56	0.7	0.58	0.79	0.67	0.73	0.61	0.75
G-EVAL-MISTRAL 2024.02		0.38	0.5	0.43	0.54	0.45	0.32	0.58	0.47	0.54	0.44	0.61	0.49	0.55	0.46
SUMMAC 2024.02		0.3	0.23	0.2	0.18	0.14	0.25	0.25	0.21	0.24	0.19	0.25	0.2	0.26	0.21
UNIEVAL 2024.02		0.25	0.59	0.52	0.42	0.33	0.17	0.14	-	-	-	-	-	-	-
ROUGE-2 2024.02		-0.17	-0.23	-0.18	-0.14	-0.1	-0.12	-0.21	-0.16	-0.26	-0.19	-0.24	-0.18	-0.14	-0.09
BERTSCORE 2024.02		-0.29	-0.32	-0.27	-0.28	-0.22	-0.22	-0.34	-0.26	-0.51	-0.43	-0.41	-0.33	-0.37	-0.28
BARTSCORE 2024.02		-0.29	-0.19	-0.15	-0.19	-0.14	-0.22	-0.33	-0.25	-0.45	-0.35	-0.37	-0.28	-0.36	-0.27
ROUGE-1 2024.02		-0.31	-0.36	-0.28	-0.3	-0.24	-0.23	-0.35	-0.26	-0.44	-0.32	-0.38	-0.29	-0.3	-0.23
ROUGE-L 2024.02		-0.34	-0.39	-0.32	-0.3	-0.23	-0.25	-0.4	-0.3	-0.51	-0.37	-0.45	-0.33	-0.38	-0.27