Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learning
Journal
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Journal Volume
2020-May
Pages
7259-7263
Date Issued
2020
Author(s)
Abstract
In this paper we propose a Sequential Representation Quantization AutoEncoder (SeqRQ-AE) to learn from primarily unpaired audio data and produce sequences of representations very close to phoneme sequences of speech utterances. This is achieved by proper temporal segmentation to make the representations phoneme-synchronized, and proper phonetic clustering to have total number of distinct representations close to the number of phonemes. Mapping between the distinct representations and phonemes is learned from a small amount of annotated paired data. Preliminary experiments on LJSpeech demonstrated the learned representations for vowels have relative locations in latent space in good parallel to that shown in the IPA vowel chart defined by linguistics experts. With less than 20 minutes of annotated speech, our method outperformed existing methods on phoneme recognition and is able to synthesize intelligible speech that beats our baseline model. © 2020 IEEE.
Subjects
representation quantization; speech recognition; speech representation; speech synthesis
Other Subjects
Audio signal processing; Linguistics; Speech; Speech communication; Audio data; Auto encoders; Baseline models; Phoneme recognition; Relative location; Speech utterance; Temporal segmentations; Speech recognition
Type
conference paper