Choosing a Pre-training Objective
A research team is developing a foundational sequence model intended to be adapted for a wide variety of text understanding tasks (e.g., sentiment analysis, question answering, text summarization). They have a massive, unlabeled text dataset for the initial training phase. Analyze the two objectives described in the case study and determine which is more suitable for their goal. Justify your choice by explaining how the chosen objective helps the model learn representations that are more broadly useful for diverse downstream applications.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A company aims to build a model that classifies customer reviews as 'positive', 'negative', or 'neutral'. They only have a small, specialized dataset of 2,000 labeled reviews. Considering the limited data, which of the following development strategies would be the most effective for achieving high accuracy?
A research team wants to use a large, pre-existing sequence model to build a system that can automatically identify the sentiment (positive or negative) of movie reviews. Arrange the following steps in the logical order they would typically follow to accomplish this.
Choosing a Pre-training Objective