Text Generation

Benchmarks

Dataset Name	SOTA Method	Metric
LM1B (test)	UDLM	Entropy2.46	72	4d ago
OpenWebText	DFM-MMF-EXACT-EMA	Perplexity132.55	66	4d ago
CNN/Daily Mail (test)	DEL	COH Score83	64	4d ago
Medical Chatbot		ASR100	42	4d ago
5 Generation tasks	POP	Accuracy57.96	36	4d ago
Text Generation		PPL11.9	33	4d ago
Text model inference M4 Max	vllm-mlx	Throughput (tok/s)525.5	31	4d ago
MSCOCO	SARE	BLEU-157.2	26	4d ago
NoveltyBench	STATe of Thoughts	Diversity5.39	21	4d ago
AbGen	ROMA	Importance4.91	20	4d ago
Aggregate NLP Tasks (GEC, Smart Reply, Summarization, Tone Adjustment, QA) (test)		Average Score32.9	18	4d ago
WebNLG seen categories (test)	CGE-LW	BLEU63.69	18	4d ago
Decoding Throughput		Decoding Throughput (tokens/s)320	16	4d ago
Wikipedia Biographies (test)	Pretrained	Delta Precision (δ-P)25.37	16	4d ago
VPI Generation Tasks Llama3-8B Mistral-7B (test)		ASR100	16	4d ago
AutoPoison Generation Llama3-8B Mistral-7B (test)		ASR82.7	16	4d ago
DTBA Llama3-8B Mistral-7B (test)		ASR77	16	4d ago
TextGen	SAIR	Cost per 1K requests ($)0.006	15	4d ago
Harry Potter forget data (400 chunks)	Target LLM	BLEU8.02	15	4d ago
C4	Unigram	TPR @ FPR=1%99.88	15	4d ago
WikiText-103	Entropy Equilibrium Sampling (EES)	Quality Better Count24	14	4d ago
OpenWebText (OWT) GPT-2 tokenizer (val)	MDLM-Prime	PPL15.36	12	4d ago
WikiBIO	Table-LLaVA 7B	BLEU9.68	11	4d ago
Rotowire	Table-LLaVA 7B	BLEU10.46	11	4d ago
HiTab T2T	Re-Table-7B-rerank	BLEU16.96	11	4d ago

Showing 25 of 91 rows