Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation

About

An important problem in ad-hoc microphone speech separation is how to guarantee the robustness of a system with respect to the locations and numbers of microphones. The former requires the system to be invariant to different indexing of the microphones with the same locations, while the latter requires the system to be able to process inputs with varying dimensions. Conventional optimization-based beamforming techniques satisfy these requirements by definition, while for deep learning-based end-to-end systems those constraints are not fully addressed. In this paper, we propose transform-average-concatenate (TAC), a simple design paradigm for channel permutation and number invariant multi-channel speech separation. Based on the filter-and-sum network (FaSNet), a recently proposed end-to-end time-domain beamforming system, we show how TAC significantly improves the separation performance across various numbers of microphones in noisy reverberant separation tasks with ad-hoc arrays. Moreover, we show that TAC also significantly improves the separation performance with fixed geometry array configuration, further proving the effectiveness of the proposed paradigm in the general problem of multi-microphone speech separation.

Yi Luo, Zhuo Chen, Nima Mesgarani, Takuya Yoshioka• 2019

Related benchmarks

TaskDatasetResultRank
Speech EnhancementAd-hoc microphone array 8-channel (test)
STOI0.856
59
Speech EnhancementAd-hoc microphone array 4-channel (test)
STOI0.839
38
Multi-channel speech enhancementSimulated adhoc microphone array dataset 4-channel (test)
STOI83.3
21
Speech EnhancementWSJ0-WHAM!
PESQ2.25
9
Speech EnhancementRealMAN
PESQ1.65
8
Speech RecognitionSMS-2 (cv_dev93)
WER22.3
4
Speech RecognitionSMS-2 eval92 (test)
WER25.1
4
Showing 7 of 7 rows

Other info

Follow for update