Ultra-Fine Entity Typing (dev)

60.7Total Precision

LRN w/o IR

Updated 4d ago

Evaluation Results

Method
LRN w/o IR 2021.09	60.7	32.5	42.3	79.3	75.5	77.4	59.6	44.8	51.2	45.7	18.7	26.5
DenoiseET + ELMo 2019.05	55.6	28.1	37.3	69.3	77.3	73	47.9	35.4	40.7	48.9	12.6	20
DenoiseET + ELMo (Augmented) 2019.05	55.2	26.4	35.7	69.4	72	70.7	46.6	38.5	42.2	48.7	10.3	17.1
BERT[in-house] 2021.09	54.1	32.1	40.3	68.8	79.2	73.6	43.8	57.4	49.7	50.7	14.6	22.6
LRN 2021.09	53.7	38.6	44.9	77.8	76.4	77.1	55.8	50.6	53	43.4	26	32.5
Vector 2021.01	53.3	36.7	43.5	71.7	79.9	75.6	51.9	48.5	50.2	43.7	22.7	29.8
Box 2021.01	52.9	39.1	45	71.2	82.5	76.4	50.9	55.2	53	45.4	24.5	31.9
BERT-Base, Uncased 2019.05	51.6	32.8	40.1	67.4	80.6	73.4	41.6	54.7	47.3	46.3	15.6	23.4
BERT 2021.09	51.6	32.8	40.1	67.4	80.6	73.4	41.6	54.7	47.3	46.3	15.6	23.4
DenoiseET + ELMo (Augmented + Filter/Relabel) 2019.05	50.7	33.1	40.1	66.9	80.7	73.2	41.7	46.2	43.8	45.6	17.4	25.2
ELMO 2021.01	50.7	33.1	40.1	66.9	80.7	73.2	41.7	46.2	43.8	45.6	17.4	25.2
ELMO 2021.09	50.7	33.1	40.1	66.9	80.7	73.2	41.7	46.2	43.8	45.6	17.4	25.2
Label GCN 2021.01	49.3	28.1	35.8	66.2	68.8	67.5	43.9	40.7	42.2	42.4	14.2	21.3
LABELGCN 2021.09	49.3	28.1	35.8	66.2	68.8	67.5	43.9	40.7	42.2	42.4	14.2	21.3
Choi et al. (2018) w augmentation 2019.05	48.1	23.2	31.3	60.3	61.6	61	40.4	38.4	39.4	42.8	8.8	14.6
Choi et al. (2018) 2021.01	48.1	23.2	31.3	60.3	61.6	61	40.4	38.4	39.4	42.8	8.8	14.6
Choi et al. 2021.09	48.1	23.2	31.3	60.3	61.6	61	40.4	38.4	39.4	42.8	8.8	14.6
DenoiseET + GloVe 2019.05	46.4	23.3	31	57.7	65.5	61.4	41.3	31.3	35.6	42.4	9.2	15.1
HY XLarge 2021.01	43.4	34.2	38.2	61.4	73.9	67.1	35.7	46.6	40.4	36.5	19.9	25.7
HY Large 2021.09	43.4	34.2	38.2	61.4	73.9	67.1	35.7	46.6	40.4	36.5	19.9	25.7