Zero Resource Code-Switched Speech Benchmark Using Speech Utterance Pairs for Multiple Spoken Languages
Part Of
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing
Start Page
10006
End Page
10010
ISSN
15206149
ISBN
[9798350344851]
Date Issued
2024-04-14
Author(s)
Abstract
We introduce a new zero resource code-switched speech bench-mark designed to assess the code-switching capabilities of self-supervised speech encoders directly. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed in a zero-resource manner. Our experiments encompass a variety of well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc., on three tracks of different code-switched language pairs: Spanish-English, French-English, and Chinese-English. We examine the impact of pre-training languages and model size on benchmark performance. Notably, though our results demonstrate that speech encoders with multilingual pre-training, exemplified by XLSR, outperform monolingual variants (Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial room for improvement in their code-switching linguistic abilities.
Event(s)
2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
SDGs
Publisher
IEEE
Type
conference paper