Numbers, monetary amounts, dates, and other concepts that are verbalized differently than they are spelled.

University of Michigan - Ann Arbor

Text to speech, or speech synthesis, is the inverse problem of automatic speech recognition. It is a sequence-to-sequence learning task where the input is a sequence of text and the output is a generated audio file, meaning the resulting output sequence is significantly longer than the input sequence.

Text to Speech

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

The first component is an encoder-decoder model for spectrogram prediction: it maps from strings of letters to mel spectrographs: sequences of mel spectral values over time. The second component maps from mel spectrograms to waveforms.

Components of TTS

Non-standard Words

Often the verbalization of a non-standard word depends on its meaning. The meaning is called Semiotic Class.

Semiotic Class

Rule-based normalization is done in two stages: tokenization and verbalization. 

Rule-based Normalization

Encoder-decoder-based Models have been shown to work better than rules for such transduction tasks, but do require expert-labeled training sets in which non-standard words have been replaced with the appropriate verbalization; such training sets for some languages are available

Encoder-decoder-based Normalization

The exact same architecture we described for ASR—the encoder-decoder with attention – can be used for the first component of TTS. 

The Tacotron2 architecture is an encoder-decoder that maps from graphmes to spectrograms, followed by a vocoder that maps to wavefiles. 

Spectrogram Prediction

Speech synthesis systems are evaluated by human listeners. We evaluate the quality of synthesized utterances by playing a sentence to listeners and ask them to give a mean opinion score (MOS), a rating of how good the synthesized utterances are, usually on a scale from 1–5. 

Paired t-tests to test for significant differences can be used to compare systems on the same sentence. AB tests can be used to compare exactly two systems.

Learn Before

Related