Our new X account is live! Follow @wizwand_team for updates

Multi-hop Question Answering on HotpotQA fullwiki setting (test)

75.9Answer F1

IDRQA

Updated 4d ago

Evaluation Results

Method
IDRQA 2020.09	75.9	62.5	51	78.9	36	63.9
MDR 2020.09	75.3	62.3	57.5	80.9	41.8	66.6
Multi-hop Dense Retrieval 2020.09	75.3	62.3	57.5	80.9	41.8	66.6
HopRetriever 2020.09	73.9	60.8	53.1	79.3	38	63.9
Graph Recurrent Retriever 2020.09	73	60	49.1	76.4	35.4	61.2
Recurrent Retriever 2020.09	73	60	49	76.4	35.3	61.1
Graph Recur. Retriever 2019.11	72.96	60.04	49.08	76.41	35.35	61.18
HGN 2019.11	71.41	59.74	51.03	77.37	37.92	62.26
Hierarchical Graph Network 2020.09	71.4	59.7	51	77.4	37.9	62.2
HGN 2019.11	69.93	57.85	51.01	76.82	37.17	60.74
HGN 2020.09	69.2	56.7	50	76.4	35.6	59.9
HGN 2020.04	69.16	56.71	49.97	76.39	35.36	59.86
QUARK 2020.04	67.51	55.5	45.64	72.95	32.89	56.23
MIR+EPS+BERT 2019.11	64.79	52.86	42.75	72	31.19	54.75
Transformer-XH 2020.09	64.1	51.6	40.9	71.4	26.1	51.3
Transformer-XH 2020.09	64	51.6	40.9	71.4	26.1	51.2
Transformer-XH 2019.11	60.75	48.95	41.66	70.01	27.13	49.57
Semantic Retrieval 2020.09	58.8	46.5	39.9	71.5	26.6	49.2
BERT pip. 2019.10	57.34	45.32	38.67	70.83	25.14	47.6
SR-MRS 2020.04	57.34	45.32	38.67	70.83	25.14	47.6
SemanticRetrievalMRS 2019.11	57.34	45.32	38.67	70.83	25.14	47.6
Semantic Retrieval MRS 2020.09	57.3	45.3	38.6	70.8	25.1	47.6
PR-Bert 2019.10	53.79	43.33	21.9	59.63	14.5	39.11
PR-BERT 2019.11	53.79	43.33	21.9	59.63	14.5	39.11
Entity-centric BERT Pipeline 2019.10	53.09	41.82	26.26	57.29	17.01	39.18
Entity-centric BERT 2019.11	53.09	41.82	26.26	57.29	17.01	39.18
DrKIT 2020.09	51.7	42.1	37.1	59.8	24.7	42.9
DrKIT 2020.09	51.7	42.1	37	59.8	24.6	42.8
CogQA 2019.05	48.9	37.1	22.8	57.7	12.4	34.9
Cognitive Graph 2019.05	48.9	37.1	22.8	57.8	12.4	34.9
Cognitive Graph 2019.06	48.87	-	-	-	-	-
CogQA 2019.10	48.87	37.12	22.82	57.69	12.42	34.92
Cognitive Graph 2019.11	48.87	37.12	22.82	57.69	12.42	34.92
Cognitive Graph 2020.09	48.8	37.1	22.8	57.6	12.4	34.9
GoldEn Retriever 2020.09	48.6	37.9	30.7	64.2	18.9	39.1
GOLDEN Retriever 2019.10	48.58	37.92	30.69	64.24	18.04	39.13
Golden Retriever 2019.11	48.58	37.92	30.69	64.24	18.04	39.13
GoldEn Retriever 2020.09	48.5	37.9	30.6	64.2	18	39.1
TPReasoner 2019.11	47.43	36.04	-	-	-	-
DecompRC 2019.05	40.7	30	-	-	-	-
DECOMPRC 2019.06	40.65	-	-	-	-	-
DecompRC 2020.09	40.6	30	-	-	-	-
MUPPET 2019.10	40.26	30.61	16.65	47.33	10.85	27.01
MUPPET 2019.11	40.26	30.61	16.65	47.33	10.85	27.01
MultiQA 2019.06	40.23	-	-	-	-	-
MultiQA 2019.05	40.2	30.7	-	-	-	-
MUPPET 2020.09	40.2	30.6	16.6	47.3	10.8	27
GRN + BERT 2019.10	39.14	29.87	13.16	49.67	8.26	25.84
QFE 2019.05	38.1	28.7	14.2	44.4	8.7	23.1
QFE 2019.05	38.1	28.7	14.2	44.4	8.69	23.1
QFE 2019.06	38.06	-	-	-	-	-
QFE 2020.04	38.06	28.66	14.2	44.35	8.69	23.1
QFE 2019.11	38.06	28.66	14.2	44.35	8.69	23.1
QFE 2020.09	38	28.6	14.2	44.3	8.6	23.1
KGNN 2019.11	37.19	27.65	12.65	47.19	7.03	24.66
GRN 2019.05	36.5	27.3	12.2	48.8	7.4	23.6
GRN 2019.05	36.5	27.3	12.2	48.8	7.4	23.6
GRN 2019.06	36.48	-	-	-	-	-
Baseline 2019.10	34.4	25.23	5.07	40.69	2.63	17.85
Yang et al. (2018) 2019.05	32.9	24	3.86	37.7	1.9	16.2
Baseline 2019.05	32.9	24	3.86	37.7	1.85	16.2
BiDAF 2019.06	32.89	-	-	-	-	-
Yang et al. (2018) 2019.11	32.89	23.95	3.86	37.71	1.85	16.15
Baseline Model 2019.11	32.89	23.95	3.86	37.71	1.85	16.15