Mandarin Electrolaryngeal Speech Voice Conversion with Sequence-to-Sequence Modeling
Journal
2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Proceedings
Pages
650-657
Date Issued
2021
Author(s)
Abstract
The electrolaryngeal speech (EL speech) is typically spoken with an electrolarynx device that generates excitation signals to substitute human vocal fold vibrations. Because the excitation signals cannot perfectly characterize sound sources generated by vocal folds, the naturalness and intelligibility of the EL speech are inevitably worse than that of the natural speech (NL speech). To improve speech naturalness, statistical models, such as Gaussian mixture models and deep-learning-based models, have been employed for EL speech voice conversion (ELVC). The ELVC task aims to convert EL speech into NL speech through an ELVC model. To implement a frame-wise ELVC system, accurate feature alignment is crucial for model training. However, the abnormal acoustic characteristics of the EL speech cause misalignments and accordingly limit the ELVC performance. To address this issue, we propose a novel ELVC system based on sequence-to-sequence (seq2seq) modeling with text-to-speech (TTS) pretraining. The seq2seq model involves an attention mechanism to concurrently perform representation learning and alignment. Meanwhile, TTS pretraining provides efficient training with limited data. Experimental results show that the proposed ELVC system yields notable improvements in terms of standardized evaluation metrics and subjective listening tests over a well-known frame-wise ELVC system. © 2021 IEEE.
Subjects
electrolaryngeal speech; pretraining; sequence-to-sequence learning; transformer; voice conversion
Other Subjects
Petroleum reservoir evaluation; Speech intelligibility; Conversion systems; Electrolaryngeal speech; Excitation signals; Natural speech; Pre-training; Sequence learning; Sequence-to-sequence learning; Text to speech; Transformer; Voice conversion; Deep learning
Type
conference paper