Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Universal Phone Recognition with a Multilingual Allophone System

About

Multilingual models can improve language processing, particularly for low resource situations, by sharing parameters across languages. Multilingual acoustic models, however, generally ignore the difference between phonemes (sounds that can support lexical contrasts in a particular language) and their corresponding phones (the sounds that are actually spoken, which are language independent). This can lead to performance degradation when combining a variety of training languages, as identically annotated phonemes can actually correspond to several different underlying phonetic realizations. In this work, we propose a joint model of both language-independent phone and language-dependent phoneme distributions. In multilingual ASR experiments over 11 languages, we find that this model improves testing performance by 2% phoneme error rate absolute in low-resource conditions. Additionally, because we are explicitly modeling language-independent phones, we can build a (nearly-)universal phone recognizer that, when combined with the PHOIBLE large, manually curated database of phone inventories, can be customized into 2,000 language dependent recognizers. Experiments on two low-resourced indigenous languages, Inuktitut and Tusom, show that our recognizer achieves phone accuracy improvements of more than 17%, moving a step closer to speech recognition for all languages in the world.

Xinjian Li, Siddharth Dalmia, Juncheng Li, Matthew Lee, Patrick Littell, Jiali Yao, Antonios Anastasopoulos, David R. Mortensen, Graham Neubig, Alan W Black, Florian Metze• 2020

Related benchmarks

TaskDatasetResultRank
Phone Feature RecognitionBuckeye (sociophonetic)
PFER5.04
25
Phone Feature RecognitionL2-Standard (sociophonetic)
PFER5.21
17
Phone Feature RecognitionL2-Perceived sociophonetic
PFER5.72
17
Phone Feature RecognitionDoreco (unseen languages)
PFER8.89
17
Phone Feature RecognitionVoxAngeles unseen languages
PFER1.35
17
Phone recognitionSeen Languages
English Error Rate (C)4.18
15
Phonetic PerceptionSO762 (SpeechOcean762)
PFER16.73
8
Phonetic PerceptionEpaDB
PFER0.1282
8
Phonetic PerceptionL2-ARCTIC
PFER13.03
8
Phonetic PerceptionDRC-SE (DoReCo South-England)
PFER0.2536
8
Showing 10 of 10 rows

Other info

Follow for update