Our new X account is live! Follow @wizwand_team for updates

Semantic Parsing on OVERNIGHT v1.0 (test)

65.7Blocks Domain Score

TWO-STAGE

Updated 4d ago

Evaluation Results

Method	Links
TWO-STAGE 2021.06		65.7	87.2	80.4	75.7	80.1	86.1	82.8	82.7	80.1
SSD 2021.06		64.9	86.2	81.7	72.7	82.3	81.7	81.5	82.7	79.2
SSD 2021.06		64.9	86.2	81.7	72.7	82.3	81.7	81.5	82.7	79
DUAL 2021.06		63.7	87.5	79.8	73	81.4	81.5	81.6	83	78.9
GPT-3 2021.10		63.4	85.9	79.2	74.1	77.6	79.2	84	68.7	76.5
GPT-3 2021.10		62	80	82	71	79	84	89	72	77.4
T5-xl 2021.10		61.9	85.6	80.6	77.9	82.4	83	82.2	79.3	79.1
SEQ2ACTION 2021.06		61.4	88.2	81.5	74.1	80.7	82.9	80.7	82.1	79
CROSSDOMAIN 2021.06		60.2	86.2	79.8	71.4	78.9	84.7	81.6	82.9	78.2
T5-xl 2021.10		59.2	84.1	80.2	76.5	77.6	81.4	78.9	72.5	76.3
SSD-SAMPLES 2021.06		58.8	71.3	60.6	62.2	58.8	65.4	71.1	49.1	62.2
SSD-SAMPLES 2021.06		58.7	71.7	60.1	61.7	57.6	64.3	70.9	46	61.4
RECOMBINATION 2021.06		58.1	85.2	78	71.4	76.4	79.6	76.2	81.4	75.8
SSD 2021.06		58.1	68.8	56.5	56.1	57.8	59.3	66.9	37.1	57.6
BART 2021.10		55.4	86.4	78	67.2	75.8	80.1	80.1	66.6	73.7
SSD 2021.06		54.9	68.3	51.2	55	54.7	60.2	65.4	33.6	55.4
GPT-2 2021.10		54	83.6	76.6	66.6	71.5	76.4	76.8	62.3	71
TWO-STAGE 2021.06		53.4	64.7	58.3	59.3	60.3	68.1	73.2	48.4	60.7
GPT-3 2021.10		53	68	68	58	63	75	78	63	65.7
BART 2021.10		49.9	83.4	75	61.9	73.9	79.6	77.4	62	70.4
GPT-2 2021.10		47.9	76	73.6	57.1	64.5	69.9	66	60.6	64.4
SYNTHPARA-SEQ2SEQ 2021.06		37.3	28.4	33.9	38.1	39.1	41.7	62.7	23.3	38.1
WMDSAMPLES 2021.06		29	31.9	36.1	47.9	34.2	41	53.8	35.8	38.7
Cross-domain Zero Shot 2021.06		28.3	-	53.6	52.4	55.3	60.2	61.7	-	-
GENOVERNIGHT 2021.06		27.7	15.6	17.3	45.9	46.7	26.3	61.3	9.7	31.3
SYNTH-SEQ2SEQ 2021.06		23.6	16.1	16.1	30.2	36.6	26.9	43.1	9.2	25.2