Learn Before
A machine learning engineer describes their text classification pipeline: 1) An input text is formatted with a special token at the beginning. 2) The token sequence is converted into embeddings. 3) A model processes the embeddings and outputs a sequence of hidden state vectors, one for each input token. 4) The hidden state vectors for all tokens except the special first one are averaged together. 5) This averaged vector is passed to a prediction network to get the final class. Which step represents a deviation from the standard, illustrated procedure for this task?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A piece of text is being classified using a common transformer-based architecture. Arrange the following stages of the process in the correct chronological order, from initial input to final output.
A model processes an input text for a classification task. The process involves converting the text into a sequence of tokens, which are then transformed into a corresponding sequence of hidden state vectors. According to the standard procedure for this type of task, which specific output from the model is typically isolated and passed to the final prediction network to determine the class label?
A machine learning engineer describes their text classification pipeline: 1) An input text is formatted with a special token at the beginning. 2) The token sequence is converted into embeddings. 3) A model processes the embeddings and outputs a sequence of hidden state vectors, one for each input token. 4) The hidden state vectors for all tokens except the special first one are averaged together. 5) This averaged vector is passed to a prediction network to get the final class. Which step represents a deviation from the standard, illustrated procedure for this task?