Short Answer

Evaluating a Model's Training Objective

A researcher is training a model to reconstruct original sentences from corrupted versions where some words are replaced by a [MASK] token. The training process only calculates an error signal based on the model's predictions for these [MASK] positions. The researcher observes that the model becomes very good at filling in the blanks, but struggles to generate complete, fluent sentences from scratch. Explain why this specific training method might lead to this outcome.

0

1

Updated 2025-10-04

Contributors are:

Who are from:

Tags

Ch.1 Pre-training - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Analysis in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science