Our new X account is live! Follow @wizwand_team for updates

AMR Parsing on LDC2017T10 AMR 2.0 (test)

86.26Smatch

Graphene Smatch

Updated 4d ago

Evaluation Results

Method	Links
Graphene Smatch 2021.10		86.26	89.03	86.75	92.47	84.61	76.72	91.39	76.25	84.85	-	-	-	-	-
LeakDistill 2023.06		86.1	88.8	86.5	91.6	83.9	76.6	91.4	75.1	82.4	-	-	-	-	-
Barzdins 2021.10		85.93	88.85	86.63	92.51	84.01	75.73	91.16	76.53	84.64	-	-	-	-	-
Struct-BART + MBSE-A silver1+2+3 + ensemble dec. 2021.12		85.9	-	-	-	-	-	-	-	-	-	-	-	-	-
Graphene Support 2021.10		85.85	88.68	86.35	92.3	84.63	77.01	91.23	74.49	84.41	-	-	-	-	-
Struct-BART + MBSE-G silver1+2+3 + ensemble dec. 2021.12		85.7	-	-	-	-	-	-	-	-	-	-	-	-	-
LeakDistill 2023.06		85.7	88.6	86.2	91.1	83.9	76.8	91	74.2	81.8	-	-	-	-	-
Struct-BART + MBSE-G silver1+2+3 + ensemble dec. 2021.12		85.6	-	-	-	-	-	-	-	-	-	-	-	-	-
Bai et al. 2021.12		85.4	-	-	-	-	-	-	-	-	-	-	-	-	-
AMRBART (large) 2022.03		85.4	88.3	85.8	91.5	81.4	74	91.2	73.5	81.5	-	-	-	-	-
AMRBART 2023.06		85.4	88.3	85.8	91.5	81.4	74	91.2	73.5	81.5	-	-	-	-	-
ATP 2023.06		85.2	88.3	85.6	93.1	83.3	74.9	90.7	74.7	83.3	-	-	-	-	-
StructBART-J ensem. 2021.10		84.9	-	-	-	-	-	-	-	-	-	-	-	-	-
Structured-BART-baseline + self-trained silver1 + ensemble dec. 2021.12		84.9	-	-	-	-	-	-	-	-	-	-	-	-	-
Ancestor 2023.06		84.8	88.1	85.3	91.8	84.1	74	90.5	75.1	83.4	-	-	-	-	-
Graphene 4S 2021.10		84.78	87.96	85.29	92.19	83.88	75.22	90.64	71.42	83.46	-	-	-	-	-
StructBART-J 2021.10		84.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Structured-BART-baseline + self-trained silver1 2021.12		84.7	-	-	-	-	-	-	-	-	-	-	-	-	-
BiBL 2023.06		84.6	87.8	85.1	92.5	83.6	73.9	90.3	74.4	83.1	-	-	-	-	-
Bevilacqua et al. (2021) 2021.10		84.5	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua et al. 2021.12		84.5	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua+ (2021, large) 2022.03		84.5	86.7	84.9	83.7	87.3	79.9	89.6	72.3	79.7	-	-	-	-	-
SPRING (ours) 2023.06		84.4	87.4	84.8	90.9	84.1	73.5	90.4	71.6	80.1	-	-	-	-	-
Bevilacqua et al. 2021.04		84.3	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua et al. (2021) 2021.10		84.3	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua+ (2021, large) 2022.03		84.3	86.7	84.8	90.5	83.1	73.6	90.8	72.4	80.5	-	-	-	-	-
SPRING 2023.06		84.3	86.7	84.8	90.5	83.1	73.6	90.8	72.4	80.5	-	-	-	-	-
SPRING 2021.10		84.22	87.38	84.72	90.77	82.76	72.65	89.98	74.3	82.89	-	-	-	-	-
StructBART-J 2021.10		84.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Xia et al. 2021.12		84.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Structured-BART-baseline 2021.12		84.2	-	-	-	-	-	-	-	-	-	-	-	-	-
StructBART-S 2021.10		84	-	-	-	-	-	-	-	-	-	-	-	-	-
Structured-BART-baseline 2021.12		84	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua et al. 2021.04		83.8	-	-	-	-	-	-	-	-	-	-	-	-	-
Bevilacqua et al. (2021) 2021.10		83.8	-	-	-	-	-	-	-	-	-	-	-	-	-
AMRBART (base) 2022.03		83.6	86.7	84	90	78.6	73.7	90.2	71.3	79.5	-	-	-	-	-
APT base 2021.04		83.4	-	-	-	-	-	-	-	-	-	-	-	-	-
T5 2021.10		82.98	86.17	83.43	90.65	77.99	73.43	89.85	72.44	82.02	-	-	-	-	-
APT base 2021.04		82.8	-	-	-	-	-	-	-	-	-	-	-	-	-
APT base 2021.04		82.8	-	-	-	-	-	-	-	-	-	-	-	-	-
APT 2021.10		82.7	86.18	83.23	90.2	78.87	67.27	89.48	73.19	82.01	-	-	-	-	-
Bevilacqua+ (2021, base) 2022.03		82.7	85.1	83.3	90	82.2	72	89.7	70.8	79.1	-	-	-	-	-
Zhou et al. (2021) 2021.10		82.6	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhou et al. 2021.12		82.6	-	-	-	-	-	-	-	-	-	-	-	-	-
APT small 2021.04		82.5	-	-	-	-	-	-	-	-	-	-	-	-	-
APT base 2021.04		81.8	-	-	-	-	-	-	-	-	-	-	-	-	-
APT small 2021.04		81.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhou et al. (2021) 2021.10		81.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Xu et al. (2020) 2021.10		81.4	-	-	-	-	-	-	-	-	-	-	-	-	-
Xu et al. 2021.12		81.4	-	-	-	-	-	-	-	-	-	-	-	-	-
mining + synTxt + synAMR 2020.10		81.3	-	-	-	-	-	-	-	-	-	-	-	-	-
Lee et al. 2021.04		81.3	-	-	-	-	-	-	-	-	-	-	-	-	-
Lee et al. (2020) 2021.10		81.3	-	-	-	-	-	-	-	-	-	-	-	-	-
Lee et al. 2021.12		81.3	-	-	-	-	-	-	-	-	-	-	-	-	-
synAMR 2020.10		81	-	-	-	-	-	-	-	-	-	-	-	-	-
synTxt + synAMR 2020.10		81	-	-	-	-	-	-	-	-	-	-	-	-	-
mining + synAMR 2020.10		80.9	-	-	-	-	-	-	-	-	-	-	-	-	-
synTxt 2020.10		80.7	-	-	-	-	-	-	-	-	-	-	-	-	-
mining + synTxt 2020.10		80.4	-	-	-	-	-	-	-	-	-	-	-	-	-
oracle mining 2020.10		80.3	-	-	-	-	-	-	-	-	-	-	-	-	-
AMR-gs 2020.04		80.2	82.8	80.8	81.1	86.3	78.9	88.1	64.6	74.2	-	-	-	-	-
Cai and Lam 2020.10		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam 2020.10		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
F. A. et al. + SE^R 2020.10		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam 2021.04		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Astudillo et al. 2021.04		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Xu et al. 2021.04		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam (2020) 2021.10		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Astudillo et al. (2020) 2021.10		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam 2021.12		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
Fernandez Astudillo et al. 2021.12		80.2	-	-	-	-	-	-	-	-	-	-	-	-	-
CaiL (2020a) 2022.03		80.2	82.8	80	81.1	86.3	78.9	88.1	64.6	74.2	-	-	-	-	-
Xu+ (2020) 2022.03		80.2	83.7	80.8	85.4	75.1	71.5	87.4	66.5	78.9	-	-	-	-	-
Lee20 2020.10		80.2	-	-	-	-	-	88.1	-	78.2	-	-	-	-	-
Cai20 w/ rules 2020.10		80.2	-	-	-	-	-	88.1	-	74.2	-	-	-	-	-
Xu20 2020.10		80.2	-	-	-	-	-	87.4	-	78.9	-	-	-	-	-
Cai&Lam 2021.10		80.15	83.6	80.66	82.25	85.36	78.09	87.39	66.46	77.35	-	-	-	-	-
Stack-Transformer 2020.10		79	-	-	-	-	-	-	-	-	-	-	-	-	-
F. A. et al. 2020.10		79	-	-	-	-	-	-	-	-	-	-	-	-	-
Stack-Transformer 2020.10		78.8	-	-	-	-	-	-	-	-	-	-	-	-	-
AMR-gs 2020.04		78.7	81.5	79.2	87.1	81.3	66.1	88.1	63.8	74.5	-	-	-	-	-
Cai and Lam 2020.10		78.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam 2020.10		78.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam 2021.04		78.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai and Lam (2020) 2021.10		78.7	-	-	-	-	-	-	-	-	-	-	-	-	-
Cai20 w/o rules 2020.10		78.7	-	-	-	-	-	88.1	-	74.5	-	-	-	-	-
Transformer 2020.10		78	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhou+ (2020) 2022.03		77.5	80.4	78.2	78.8	86.5	76.1	85.9	61.1	71	-	-	-	-	-
AMR-gs 2020.04		77.3	80.1	77.9	78.4	86.1	75.6	86.4	58.5	69.4	-	-	-	-	-
Transformer 2020.10		77.2	-	-	-	-	-	-	-	-	-	-	-	-	-
attention-based neural transducer 2019.09		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang et al. 2020.04		77	80	78	79	86	77	86	61	71	-	-	-	-	-
Zhang et al. 2020.10		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang et al. 2020.10		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang et al. 2021.04		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang et al. (2019b) 2021.10		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang et al. 2021.12		77	-	-	-	-	-	-	-	-	-	-	-	-	-
Zhang+ (2019b) 2022.03		77	80	78	79	86	77	86	61	71	-	-	-	-	-
Zhang19 2020.10		77	-	-	-	-	-	86	-	71	-	-	-	-	-
Lindemann20 2020.10		76.8	-	-	-	-	-	-	-	-	-	-	-	-	-

Showing 100 of 205 rows