Spoken Question Answering

Benchmarks

Dataset Name	SOTA Method	Metric
S-SQuAD (test)	ALBERT-base + DA + KD	EM64.1	16	1mo ago
Spoken-SQuAD	contr-cos-all + asr + giga	EM76.11	15	1mo ago
SQuAD v1.1 (test)	AEG (with LFE)	EM89.24	12	1mo ago
MuSiQue v1.0 (test)	AEG (with LFE)	Exact Match (EM)53.99	12	1mo ago
HotpotQA v1.0 (test)	AEG (with LFE)	Exact Match (EM)79.16	12	1mo ago
LibriSQA Part II	Transcripts + Llama 3.1 8B	Accuracy74.9	11	1mo ago
LibriSQA (Part I)	Transcripts + EuroLLM 9B	Accuracy87.6	11	1mo ago
Spoken SQuAD (test)	Transcripts + EuroLLM 9B	Accuracy91.1	11	1mo ago
UltraEval-Audio S2S		AlpacaEval Score0.7338	9	1mo ago
Spoken SQuAD (B) ASR (test)	Ensembled [(e) plus (d)]	EM60.37	9	1mo ago
Our Bench		Accuracy76.34	8	1mo ago
Spoken-MQA		Accuracy83.95	8	1mo ago
VERA		Accuracy18.54	8	1mo ago
AlpacaEval	LLaMA-Omni2	WER3.88	7	1mo ago
TriviaQA	Qwen2.5-Omni-7B	WER1.47	7	1mo ago
Web Questions	Qwen2.5-Omni-7B	WER1.68	7	1mo ago
Llama Questions	Qwen2.5-Omni-7B	WER0.99	7	1mo ago
Spoken SQuAD NoiseV2 (test)	BiDAF (WORD+CHAR+PHONEME+SYLLABLE)	EM20.66	5	1mo ago
Spoken SQuAD NoiseV1 (test)	BiDAF (WORD+CHAR+PHONEME+SYLLABLE)	EM29.73	5	1mo ago
Spoken SQuAD No noise (test)	BiDAF (WORD+CHAR+PHONEME+SYLLABLE)	EM45.78	5	1mo ago
VoiceBench S2T	Longcat-Flash-Omni-Instruct	AlpacaEval4.94	4	1mo ago
OpenAudioBench S2T	Fun-Audio-Chat-30B-A3B	AlpacaEval Score88.89	4	1mo ago
Spoken SQuAD (A) Text (test)	BERT	EM76.9	3	1mo ago
SQuAD lost (test)	SpeechBERT	F1 Score37.31	2	1mo ago

Showing 24 of 24 rows