Our new X account is live! Follow @wizwand_team for updates

Machine Reading Comprehension on RACE (test)

95.4RACE Accuracy (Medium)

Human Performance (Ceiling)

Updated 4d ago

Evaluation Results

Method
Human Performance (Ceiling) 2019.01	95.4	94.2	94.5	-	-
Ceiling 2019.08	95.4	94.2	94.5	-	-
Megatron-3.9B ensemble 2019.09	93.1	90	-	-	90.9
Megatron-BERT 2020.01	93.1	90	-	90.9	-
ALBERTxxlarge 2022.03	92.8	89.8	-	90.7	-
ALBERT_xxlarge+DUMA 2020.01	92.6	88.7	-	89.8	-
Megatron-3.9B 2019.09	91.8	88.6	-	-	89.5
Megatron-BERT 2020.01	91.8	88.6	-	89.5	-
Megatron-BERT-3.9B 2022.03	91.8	88.6	-	89.5	-
POI-Net 2022.03	91.5	86.8	-	88.3	-
POI-Net 2022.03	91.5	86.8	-	88.3	-
ALBERT 2019.09	91.2	88.6	-	89.4	-
ALBERT ensemble 2019.09	91.2	88.6	-	-	89.4
ALBERT 2020.01	91.2	88.6	-	89.4	-
ALBERT_xxlarge+DUMA 2020.01	90.9	86.7	-	88	-
ALBERTxxlarge 2022.03	90.9	86.7	-	88	-
ALBERTxxlarge + DUMA 2022.03	90.9	86.7	-	88	-
Megatron-1.3B 2019.09	90.4	86.1	-	-	87.3
ALBERTxxlarge 2022.03	89.2	85.4	-	86.5	-
ALBERTxxlarge 2022.03	89.2	85.4	-	86.5	-
RoBERTa + MMM 2020.01	89.1	83.3	-	85	-
RoBERTalarge 2022.03	89.1	83.3	-	85	-
RoBERTalarge+MMM 2022.03	89.1	83.3	-	85	-
ALBERT 2019.09	89	85.5	-	86.5	-
ALBERT 2019.09	89	85.5	-	-	86.5
ALBERT 2020.01	89	85.5	-	86.5	-
ALBERT_xxlarge 2020.01	89	85.5	-	86.6	-
ALBERTxxlarge 2022.03	89	85.5	-	86.5	-
B10-10-10 2020.06	88.8	84.4	85.7	-	-
XLNet 2019.06	88.6	84	-	85.4	-
XLNet 2019.09	88.6	84	-	-	85.4
XLNet_Large 2020.06	88.6	84	85.4	-	-
DCMN+ 2019.09	88.5	82.3	-	84.1	-
B8-8-8 2020.06	88.4	83.9	85.2	-	-
ALBERT 2019.09	88.2	85.1	-	86	-
FLOATER 2020.03	87.1	81.7	-	83.3	-
Megatron-336M 2019.09	86.9	81.5	-	-	83
RoBERTa 2019.07	86.5	81.3	83.2	-	-
RoBERTa 2019.06	86.5	81.8	-	83.2	-
RoBERTa 2019.09	86.5	81.3	-	83.2	-
RoBERTa 2019.09	86.5	81.8	-	-	83.2
RoBERTa 2020.03	86.5	81.3	-	82.8	-
ROBERTA_Large 2020.06	86.5	81.3	83.2	-	-
XLNet_xxlarge + DCMN+ 2020.01	86.5	81.3	-	82.8	-
XLNetlarge 2022.03	86.5	81.3	-	82.8	-
RoBERTalarge 2022.03	86.5	81.8	-	83.2	-
RoBERTalarge 2022.03	86.5	81.8	-	83.2	-
XLNet 2019.09	85.5	80.2	-	81.8	-
XLNet 2020.01	85.5	80.2	-	81.8	-
XLNetlarge 2022.03	85.5	80.2	-	81.8	-
XLNet_LARGE 2019.07	85.4	80.2	81.7	-	-
Human performance 2018.10	85.1	69.4	73.3	-	-
Human Performance (Turkers) 2019.01	85.1	69.4	73.3	-	-
Turkers 2019.08	85.1	69.4	73.3	-	-
B6-6-6 2020.06	83.4	78.2	79.7	-	-
B6-3x2-3x2 2020.06	82	77.5	78.8	-	-
B4-4-4 2020.06	80	74.6	76.2	-	-
MPNet 2020.04	79.7	74.5	76.1	-	-
DMN+_large 2019.01	79.5	71.8	74.1	-	-
BERT+DCMN 2019.06	79.5	71.8	-	74.1	-
SG-Net 2019.08	78.8	72.2	74.2	-	-
SG-Net 2022.03	78.8	72.2	-	74.2	-
OCN 2019.01	78.4	71.5	73.5	-	-
Baseline 2019.08	78.4	70.4	72.6	-	-
DMN_large 2019.01	77.6	70.1	72.3	-	-
DCMN 2019.08	77.6	70.1	72.3	-	-
MPNet 2020.04	76.8	67.7	70.4	-	-
OCN 2019.08	76.7	69.6	71.7	-	-
OCN 2020.01	76.7	69.6	-	71.7	-
BERT_LARGE 2019.07	76.6	70.1	72	-	-
BERT_large 2019.01	76.6	70.1	72	-	-
BERT_LARGE 2019.08	76.6	70.1	72	-	-
BERT 2019.06	76.6	70.1	-	72	-
BERT-large 2019.09	76.6	70.1	-	72	-
BERTlarge 2022.03	76.6	70.1	-	72	-
BERTlarge 2022.03	76.6	70.1	-	72	-
POI-Net 2022.03	75.7	69	-	71	-
POI-Net 2022.03	75.7	69	-	71	-
DMN_base 2019.01	72.3	64.2	66.5	-	-
ALBERTbase 2022.03	72.1	65.2	-	67.2	-
ALBERTbase 2022.03	72.1	65.2	-	67.2	-
GPT + Self-Assessment + Highlighting + Back and Forth Reading 2018.10	72	64.5	66.7	-	-
BERT 2020.04	71.7	62.3	65	-	-
BERT_base 2019.01	71.1	62.3	65	-	-
BERTbase 2022.03	71.1	62.3	-	65	-
BERTbase 2022.03	71.1	62.3	-	65	-
GPT + Self-Assessment + Highlighting + Back and Forth Reading 2018.10	70.9	63.2	65.4	-	-
MPNet_Base 2020.06	70.3	76.3	72	-	-
GPT + Self-Assessment + Highlighting 2018.10	69.2	61.5	63.8	-	-
RSM 2019.01	69.2	61.5	63.8	-	-
RSM 2020.01	69.2	61.5	-	63.8	-
GPT + Highlighting 2018.10	67.4	61.5	63.2	-	-
GPT + Back and Forth Reading 2018.10	67.3	60.7	62.6	-	-
MMN 2018.10	64.7	55.5	58.2	-	-
GPT (our implementation) 2018.10	63.5	59.3	60.6	-	-
GPT + Self-Assessment 2018.10	63.2	59.2	60.4	-	-
GPT 2018.10	62.9	57.4	59	-	-
GPT 2019.01	62.9	57.4	59	-	-
GPT 2019.06	62.9	57.4	-	59	-
GPT 2020.01	62.9	57.4	-	59	-

Showing 100 of 132 rows