Learn Before
Analyzing Self-Supervised Training Procedures
A research team is exploring different self-supervised methods to pre-train a model for image understanding. They propose three distinct training procedures. Analyze the procedures below and identify which one does NOT correctly implement the mask-predict framework. Justify your answer by explaining why the chosen procedure violates the core principles of this framework.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Masked Language Modeling (MLM)
A researcher is developing a model to understand patterns in unlabeled time-series data from weather sensors. The data for each day is a sequence of 24 hourly temperature readings. The researcher's training strategy involves taking a sequence, randomly hiding the temperature reading for a single hour, and then training the model to estimate the hidden temperature value by looking at the readings from the other 23 hours. Which fundamental training strategy does this approach best exemplify?
Dual Role of Data in a Self-Supervised Task
Analyzing Self-Supervised Training Procedures