Learn Before
Sequence Generation Models
In Natural Language Processing, sequence generation models are designed to produce a sequence of tokens based on a given context. They are applied to various language generation tasks, including question answering and machine translation. The nature of the 'context' they use is application-dependent; for instance, in language modeling, it consists of the preceding tokens, while in machine translation, it is the source-language sequence.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Sequence Encoding Models
Sequence Generation Models
Architectural Differences Between Sequence Encoding and Generation Models
General Formulation of a Sequence Model
A large language model is pre-trained on a vast text corpus. Its training objective is to take a sentence, randomly mask 15% of the words, and then predict only the original masked words by looking at all the surrounding unmasked words (both to the left and right). Which statement best analyzes the primary goal of this specific pre-training approach?
Analyzing Pre-training Objectives
Match each Natural Language Processing (NLP) task with the primary pre-training problem type it is designed to solve.
Learn After
Architectural Differences Between Sequence Encoding and Generation Models
Large Language Models (LLMs)
A developer is building a system to translate English sentences into French. The system takes an English sentence like 'The cat is on the mat' as input. Which of the following actions best demonstrates the primary function of a sequence generation model in this system?
Ease of Fine-Tuning Sequence Generation Models
Analyzing Context in Sequence Generation Tasks
A sequence generation model produces a sequence of tokens based on a given context. Match each natural language processing task with the specific type of context the model would use to generate its output.