Question answering can be formatted as a sequence-to-sequence mapping by prepending a task-specific prompt and a source start symbol to the input query, and using a target start symbol for the output. For example, the source sequence might be formatted as "$$[\mathrm{CLS}]$$ Answer: when was Albert Einstein born?", which is mapped to the target sequence "$$\langle s \rangle$$ He was born on March 14, 1879."

Google

Natural Language Processing tasks can be formulated as sequence-to-sequence mappings by employing specific start symbols to differentiate the input (source) from the output (target). For instance, the token $$[\mathrm{CLS}]$$ is conventionally used as the start symbol on the source side, while $$\langle s \rangle$$ is used as the start symbol on the target side. This unified representation allows diverse problems to be expressed in the exact same format.

Formulating NLP Tasks as Sequence-to-Sequence Mappings using Start Symbols

Reference of Foundations of Large Language Models Course

Sequence-to-Sequence Formatting Example: Question Answering

Text simplification can be framed as a sequence-to-sequence mapping using distinct start symbols for the input and output. For instance, an input like "$$[\mathrm{CLS}]$$ Simplify: the professor, who has published numerous papers in his field, will be giving a lecture on the topic next week." is mapped to the simplified target output "$$\langle s \rangle$$ The experienced professor will give a lecture next week."

Sequence-to-Sequence Formatting Example: Text Simplification

The task of evaluating translation quality can be structured as a sequence-to-sequence text generation problem rather than a standard numerical scoring task. In this format, the source sequence includes a specific task instruction, such as "$$[\mathrm{CLS}]$$ Score the translation from English to Chinese. English: when in Rome, do as the Romans do. Chinese: 人在罗马就像罗马人一样做事。", and the model maps this to a target sequence containing the predicted score represented as generated text, like "$$\langle s \rangle$$ $${}0.81$$".

Learn Before

Related