Our new X account is live! Follow @wizwand_team for updates

Natural Language Understanding on SuperGLUE (dev)

93.2Average Score

ST-MoE 32B

Updated 2d ago

Evaluation Results

Method
ST-MoE 32B 2022.05	93.2	95.7	93.1	-	100	100	81	100	95	-	100	90.4	69.9	95.6	-
ST-MoE-32B 2022.04	93.2	95.7	93.1	-	100	100	81	100	95	-	100	90.4	69.9	95.6	-
PaLM 540B 2022.05	92.6	95.7	92.2	-	100	100	78.8	100	94	-	100	90.1	69.2	94.6	-
PaLM 540B 2022.04	92.6	95.7	92.2	-	100	100	78.8	100	94	-	100	90.1	69.2	94.6	-
UL2 20B 2022.05	90.7	92.1	90.8	-	99	98.7	77.3	98.1	93.7	-	98.2	88.4	64.8	93.2	-
T5 11B 2022.05	89.9	93.9	90.8	-	98	94.9	77.3	96.2	93.8	-	96.4	87.4	66.1	93.2	-
T5-11B 2022.04	89.9	93.9	90.8	-	98	94.9	77.3	96.2	93.8	-	96.4	87.4	66.1	93.2	-
PaLM 62B 2022.05	89.2	89.5	90.6	-	98	96.4	75.9	96.2	93	-	95.7	87.7	61.9	92.4	-
GLM 2023.06	89	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc 2023.06	84.76	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PaLM 8B 2022.05	83.4	84.5	87.6	-	86	96.4	73.4	88.5	89.7	-	92.1	81.6	64	89.3	-
GLM_RoBERTa 2021.03	82.9	87.7	84.7	-	82	98.2	71.2	83.7	89.6	89	98.7	82.4	50.1	-	-
RoBERTa_Large 2021.03	81.5	87	86.1	-	90	94.6	72.6	63.5	89	88.4	96.1	84.4	52.9	-	-
T5_Large 2021.03	81.2	84.8	84.3	-	78	98.2	71.6	84.6	85.7	85	96.4	80.9	46.6	-	-
GLM_515M 2021.03	78.8	79.1	81.3	-	85	96.4	69.4	81.7	82.3	81.7	95	77.2	35	-	-
PaLM 540B 2022.08	78.8	72.9	88	-	93	51.8	59.1	89.1	-	92.9	-	83.5	-	-	-
GLM_410M 2021.03	78	79.4	81.9	-	80	96.4	69	81.7	81.5	80.9	93.2	76.2	35.5	-	-
GLM 2023.06	77.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLM_Large 2021.03	77	74	82.1	-	76	94.6	68.5	81.7	81.7	81.1	96.1	77.1	36.3	-	-
GLM_Sent 2021.03	76.8	79.1	80.8	-	77	93.7	70.4	79.8	80.7	80.2	94.6	76.9	36.1	-	-
T5_Base 2021.03	76	78.3	80.8	-	73	92.9	67.9	79.8	76.2	75.4	94.8	76.4	40	-	-
BART_Large 2021.03	76	84.5	84.3	-	60	92.9	69	65.4	88.3	87.8	90.5	81.8	48	-	-
GLM_Doc 2021.03	75.7	76.2	79.8	-	77	96.4	63.6	78.8	80.2	79.6	97.3	74.6	32.1	-	-
UniLM_Large 2021.03	74.1	76.5	80.5	-	72	91.1	69.7	65.4	80	79.1	91	77.2	38.2	-	-
GLMD 2023.06	72.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc 2023.06	72.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
BERT_Large 2021.03	72	73.6	80.1	-	69	92.9	71	64.4	76.3	75.6	94.8	71.9	24.1	-	-
GLM 2023.06	71.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc 2023.06	71.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD 2023.06	70.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-
TinyBERT 2023.06	70.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLM_Base 2021.03	70.7	71.2	77	-	71	85.7	64.7	72.1	73.5	72.8	89.5	72.1	26.1	-	-
AlexaTM 20B 2022.08	69.16	68.59	69.44	-	78	67.9	53.29	68.27	-	88.4	-	59.57	-	-	-
TinyBERT 2023.06	69.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc 2023.06	67.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD 2023.06	67.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD+al 2023.06	67.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
TinyBERT 2023.06	67.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc+al 2023.06	67.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GLMD-vc+mo 2023.06	67	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DIITO 2023.06	66.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PKD 2023.06	66.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Universal-KD 2023.06	66.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-
BERT_Base 2021.03	66.1	70	74.9	-	66	76.8	68.8	65.4	65.4	64.9	70.9	68.4	21.5	-	-
Theseus 2023.06	66.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLMv2 2023.06	66.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
TMKD 2023.06	65.6	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Uncertainty 2023.06	65.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
RL-KD 2023.06	65.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MobileBERT25M 2023.06	65.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
TinyBERT 2023.06	65.09	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ALP-KD 2023.06	64.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DistilBERT 2023.06	63.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLM 2023.06	63.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLMv2 2023.06	63.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-
RAIL-KD 2023.06	63.6	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MGSKD 2023.06	63.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Annealing-KD 2023.06	63.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
CKD 2023.06	63.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Annealing-KD 2023.06	63.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
UL 20B 2022.08	63	60.7	63.1	-	85	41.1	49.8	79.9	-	88.1	-	36.2	-	-	-
Continuation-KD 2023.06	62.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-
CKD 2023.06	62.6	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLM 2023.06	62.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
LRC-BERT 2023.06	62.3	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLMv2 2023.06	62.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PD 2023.06	62	-	-	-	-	-	-	-	-	-	-	-	-	-	-
CKD 2023.06	61.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Continuation-KD 2023.06	61.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MiniLM 2023.06	61.6	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PD 2023.06	61.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
RAIL-KD 2023.06	61.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GPT3 175B 2022.08	61.2	63.5	60.5	-	91	46.4	0	65.4	-	90.2	-	72.9	-	-	-
PD 2023.06	61	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Annealing-KD 2023.06	61	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Continuation-KD 2023.06	61	-	-	-	-	-	-	-	-	-	-	-	-	-	-
LRC-BERT 2023.06	60.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
RAIL-KD 2023.06	60.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MGSKD 2023.06	59.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MT-BERT 2023.06	59.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
MGSKD 2023.06	58.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-
LRC-BERT 2023.06	58.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
GPT3 13B 2022.08	57.2	62.8	66.2	-	84	19.6	0	64.4	-	89	-	71.4	-	-	-
SID 2023.06	55.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Universal-KD 2023.06	54.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-
TAKD 2023.06	54.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DGKD 2023.06	54	-	-	-	-	-	-	-	-	-	-	-	-	-	-
KD 2023.06	53.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Universal-KD 2023.06	52.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-
KD 2023.06	52.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-
KD 2023.06	52	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Swift (Large) 2022.03	-	88.5	84.3	80.7	81	92	72.7	86.5	-	-	-	-	-	-	-
Super (11B) 2022.03	-	93.1	89.4	84.9	93	93.1	77.4	89.4	-	-	-	-	-	-	-
E-LANG 2022.03	-	93.2	88.7	84.9	90	93.1	78.1	89.4	-	-	-	-	-	-	-
Baseline 2023.05	-	-	78.1	-	72	91.1	-	-	-	-	-	70.3	-	-	-
token drop 2023.05	-	-	79.9	-	68	91.1	-	-	-	-	-	72.8	-	-	-
SCTD 2023.05	-	-	79.7	-	72	92.9	-	-	-	-	-	72.8	-	-	-
Baseline 2023.05	-	-	74.4	-	63	83.9	-	-	-	-	-	68.1	-	-	-
token drop 2023.05	-	-	73	-	64	83.9	-	-	-	-	-	67.7	-	-	-
SCTD 2023.05	-	-	73.8	-	68	87.5	-	-	-	-	-	68.9	-	-	-

Showing 100 of 103 rows