Our new X account is live! Follow @wizwand_team for updates

Natural Language Understanding on GLUE v1 (dev)

93.8MRPC Score

AutoBERT-Zero*

Updated 4d ago

Evaluation Results

Method
AutoBERT-Zero* 2021.07	93.8	67.3	86.4	86.3	90.8	85.2	91.7	92.5	95.2	87.7	-
AutoBERT-Zero 2021.07	93.3	64.5	85.5	85.3	90.8	81.9	88.9	92	94.2	86.3	-
AutoBERT-w/o-desc 2021.07	92.8	65.1	84.5	85	90.5	78.7	88.2	91.6	93.7	85.6	-
AutoBERT-conv 2021.07	92.6	63.8	84.4	84.6	90.1	80.5	88.3	91.7	93.5	85.5	-
AutoBERT-att 2021.07	92.2	65.4	84.6	85	90.4	81.6	88.5	91.8	93.8	85.9	-
BERTbase (Teacher) 2023.05	91.4	60.3	84.9	84.8	89.4	69.7	91.5	91.7	93.7	84.1	-
AD-KD 2023.05	91.2	58.3	83.4	84.2	89.2	70.9	91.2	91.2	91.9	83.4	-
MGSKD 2023.05	89.8	49.1	83.3	83.9	88.5	67.9	91.2	90.3	91.7	81.5	-
BERT-base(ours) 2021.07	89.7	58.1	84.8	85.2	88.8	69	88.2	91.5	92.9	83.1	-
Flexible Multi-Task Model (w/o KD) 2021.07	89.7	55.2	-	-	88.9	71.5	91.1	91.7	93.1	83.2	84.5
CKD 2023.05	89.6	55.1	83.6	84.1	89	67.3	91.2	90.5	93	82.4	-
Vanilla KD 2023.05	89.4	53.6	82.7	83.1	88.7	66.8	90.5	90.1	91.1	81.6	-
PD 2023.05	89.4	-	82.5	83.4	-	66.7	90.7	89.4	91.1	-	-
BERT6 (Student) 2023.05	89.2	51.2	81.7	82.6	88.3	66.1	90.4	89.3	91	80.9	-
BERT-of-Theseus 2021.07	89	51.1	-	-	88.7	68.2	89.6	89.5	91.5	81.2	82.3
TinyBERT 2023.05	88.8	53.8	83.1	83.4	88.3	66.9	90.5	89.9	92.3	81.7	-
TinyBERT 2021.07	88.4	42.8	-	-	-	72.2	90.6	90.5	91.6	-	83.5
Full fine-tuning 2021.07	88.2	54.7	-	-	88.8	69.7	91.1	91.6	93.4	82.8	84.6
Flexible Multi-Task Model (KD-3) 2021.07	88	50	-	-	88.9	66.8	91.2	90.2	92.7	81.3	82.9
Flexible Multi-Task Model (mixed) 2021.07	88	55.2	-	-	88.3	71.5	91	90.2	92.7	82.5	82.9
DistillBERT 2021.07	87.5	51.3	-	-	86.9	59.9	88.5	89.2	91.3	79.6	82.2
MT-DNN (full) 2021.07	87.4	51.3	-	-	86.8	80.9	87.6	91.1	93.5	82.9	84.4
PD-BERT 2021.07	87.2	-	-	-	-	66.7	89.1	89	91.1	-	83
Flexible Multi-Task Model (KD-2) 2021.07	86.3	44	-	-	88.6	64.6	91.3	88.6	92.7	79.7	81.7
Vanilla-KD 2021.07	86.2	45.1	-	-	84.9	64.9	88.1	88	90.5	78.5	80.1
BERT-PKD 2021.07	85.7	45.5	-	-	86.2	66.5	88.4	88.4	91.3	79.2	81.3
MiniLM 2021.07	85.7	45.5	-	-	86.2	66.5	88.4	88.4	91.3	79.2	81.3
PKD 2023.05	85.7	45.5	81.3	-	86.2	66.5	88.4	88.4	91.3	79.2	-
Flexible Multi-Task Model (KD-1) 2021.07	85.1	36.4	-	-	88.3	66.1	91	86.4	90.7	77.4	77.5
MT-DNN (LOO) 2021.07	74.2	10.2	-	-	72.9	60.6	66.5	69.7	79.2	-	56.7