Semantic Parsing

Benchmarks

Dataset Name	SOTA Method	Metric
CFQ MCD3	Dynamic Least-to-Most Prompting	Accuracy95.5	33	3d ago
CFQ (MCD2)	Dynamic Least-to-Most Prompting	Accuracy95.3	33	3d ago
CFQ (MCD1)	Grounded Graph Decoding	Accuracy98.6	33	3d ago
GeoQuery (i.i.d.)	T5 11B (Prompt Tuning)	Exact Match Accuracy93.6	32	2d ago
GeoQuery compositional	SpanSub	Accuracy89.5	29	2d ago
WikiSQL (test)	TypeSQL+TC	Execution Accuracy82.6	27	3d ago
OVERNIGHT v1.0 (test)	TWO-STAGE	Blocks Domain Score65.7	26	3d ago
GEO	COARSE2FINE	Accuracy0.939	26	3d ago
COGS (generalization)	Lear	Accuracy (Generalization)99	25	3d ago
SMCalFlow	Grammar Prompting (w. oracle grammar)	Program Accuracy88.9	22	3d ago
CFQ MCD avg	Dynamic Least-to-Most Prompting	Exact Match Accuracy95	22	3d ago
SMCalFlow-CS (16-C)	Cover-LS (Oracle)	Accuracy73.5	20	3d ago
ATIS	COARSE2FINE	Accuracy95.1	19	3d ago
Break	CLG	Accuracy42.21	18	3d ago
GeoQuery (Len.)	DPP	Exact Match Accuracy74.3	17	3d ago
mTOP (test)	POS-based code-switching	Average Score91.8	17	3d ago
WIKITABLEQUESTIONS (test)	TaBERT-Large	Execution Accuracy (Best)52.3	17	3d ago
COGS (test)	SpanSub+L2S2	Exact Match Accuracy92.3	16	2d ago
SCAN Around Right	SpanSub+L2S2	Exact-match Accuracy100	16	2d ago
WIKITABLEQUESTIONS (dev)	TaBERT-Large	Execution Accuracy (Best)53	16	3d ago
CFQ MCD3 (test)	LIRind+RIR	Accuracy77.9	15	3d ago
CFQ MCD2 (test)	LIRind+RIR	Accuracy0.853	15	3d ago
CFQ MCD1 (test)	RIR	Accuracy88.7	15	3d ago
NL2Bash	CLG	Accuracy45.19	14	3d ago
Geoquery (test)	DCS+L	Accuracy91.1	14	3d ago

Showing 25 of 114 rows