Automatic speech recognition is a sequence-to-sequence learning task where the input sequence is an audio recording of a speaker and the output is a text transcript of the spoken words. A significant challenge in this domain is that there is no one-to-one correspondence between audio frames and text, as thousands of audio samples may correspond to a single word, making the input sequence much longer than the output sequence.

Automatic Speech Recognition

Text to speech, or speech synthesis, is the inverse problem of automatic speech recognition. It is a sequence-to-sequence learning task where the input is a sequence of text and the output is a generated audio file, meaning the resulting output sequence is significantly longer than the input sequence.

Text to Speech

Machine translation is a sequence-to-sequence learning problem that involves ingesting a sentence in a source language and predicting its corresponding translation in a target language. A major challenge in this task is unaligned data, because the input and output sequences can have varying lengths and the corresponding words or regions may appear in completely different orders.

Machine Translation

Sequence-to-sequence learning is a specific subset of sequence learning problems where both the inputs and the outputs consist of variable-length sequences. In these tasks, the length of the input sequence does not necessarily match the length of the output sequence. Prominent applications include machine translation and speech-to-text transcription.

Claude

Sequence learning encompasses machine learning problems that require a model to ingest sequences of inputs, emit sequences of outputs, or both. Unlike standard tasks that process a fixed number of inputs independently, sequence learning models maintain context across successive observations, making them essential for handling varying-length data such as video snippets, text, or patient histories over time.

Sequence Learning

Dive into Deep Learning

Sequence-to-Sequence Learning

Sequence tagging and parsing are sequence learning tasks that involve annotating a text sequence with specific attributes. In these problems, the inputs and outputs are strictly aligned, meaning they share the exact same number of elements and occur in a corresponding order. An example is part-of-speech tagging, where every word in a sentence is annotated with its corresponding grammatical role.

Sequence Tagging and Parsing

An attention mechanism acts as a learnable pointer structure that allows a model to dynamically focus on specific parts of a sequence. This solves the challenge of increasing a system's memory and complexity without adding learnable parameters, and bypasses the bottleneck of compressing an entire sequence into a fixed-dimensional representation.

Learn Before

Related

Learn After