Our new X account is live! Follow @wizwand_team for updates

Natural Language Processing

Benchmarks

Dataset Name	SOTA Method	Metric
T0 MTest11 P3 (test)		Accuracy61.4	42	4d ago
7 NLP Tasks (test)		Average Accuracy88.9	20	4d ago
T0 benchmark	T0*	RTE85.8	18	4d ago
NLP	SAIR	Cost per 1K Requests ($)0.007	15	4d ago
T0 Without SCloze dataset HyperT5 variant (test)		Accuracy60.6	14	4d ago
decaNLP Tasks (unseen)	Diana	AN'34.69	14	4d ago
decaNLP Tasks seen (test)		AN Score77.97	14	4d ago
SuperGLUE Full, excl. ReCoRD (dev)	Multi-CLS BERT	Macro Avg Score70.03	13	4d ago
SuperGLUE 1k samples, excl. ReCoRD (dev)	Multi-CLS BERT	Macro Avg Score65.84	13	4d ago
SuperGLUE 100 samples, excl. ReCoRD (dev)	Multi-CLS BERT	Macro Avg Score59.88	13	4d ago
GLUE 1k samples (dev)	Multi-CLS BERT	Macro Avg Score76.27	13	4d ago
GLUE 100 samples (dev)	Multi-CLS BERT	Macro Avg Score64.24	13	4d ago
BERT NLP Task Suite (ANLI, Rotten Tomatoes, CoLA, SMS) (test)	TA	ANLI Accuracy51.5	12	4d ago
7 NLP Tasks Aggregate T5-Base T5-Large (average)		Accuracy (%)88.9	12	4d ago
8 NLP Tasks (avg)	TSV-M	Accuracy82.21	10	4d ago
eleven NLP tasks		Average Accuracy73.1	10	4d ago
GLUE	RMT-KD	Red Score80.9	8	4d ago
FLAN 8-task subset: arc_challenge, cosmos_qa, definite_pronoun_resolution, glue_qqp, hellaswag, mnli, squad_v1, sst2	FFA-LoRA	Closed-book QA71	7	4d ago
Federated Dataset Personalization 2	FedDPA-T	Paraphrasing Accuracy90.5	6	4d ago
Federated Dataset 1 (Personalization)	FedDPA-T	Paraphrasing Score0.805	6	4d ago
29 public NLP benchmarks (average)	GLaM	Accuracy68.1	6	4d ago
Federated Dataset Test-Time Personalization 2	FedDPA-F	Paraphrasing71.64	4	4d ago
Federated Dataset 1 Test-Time Personalization	FedDPA-F	Paraphrase Accuracy78.1	4	4d ago
12 NLP task categories Average across distributions	FlexLoRA	Avg Improvement (%)1.56	4	4d ago
12 NLP task categories Heavy-Tail (S) distribution	FlexLoRA	Avg Percentage Improvement0.0166	4	4d ago

Showing 25 of 30 rows