Language Model Performance Analysis
A development team is using a neural network model to process legal documents. They observe that processing a 1,000-token document excerpt takes approximately 3 seconds. However, when they attempt to process a full 10,000-token document, the processing time increases to approximately 5 minutes (300 seconds). Based on this performance data, what is the most likely computational characteristic of the model's architecture that explains this disproportionate increase in processing time? Explain your reasoning using the data provided.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Language Model Performance Analysis
A developer observes that a standard Transformer-based language model takes approximately 2 seconds to process a text sequence of 500 tokens. Based on the computational properties of the model's core mechanism, what is the most likely processing time if the input sequence length is doubled to 1000 tokens?
Model Selection for Long-Document Summarization