A key component in a modern neural network architecture for processing text has a computational cost that grows quadratically with the length of the input sequence. If processing a sequence of 512 tokens takes 2 seconds on a specific hardware setup, approximately how long would it take to process a sequence of 2048 tokens, assuming all other factors are constant?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Architectural Adaptation of LLMs for Long Sequences
Quadratic Complexity's Impact on Transformer Inference Speed
Computational Infeasibility of Standard Transformers for Long Sequences
Shared Weight and Shared Activation Methods
Key-Value (KV) Cache in Transformer Inference
Analyzing Model Processing Time
A key component in a modern neural network architecture for processing text has a computational cost that grows quadratically with the length of the input sequence. If processing a sequence of 512 tokens takes 2 seconds on a specific hardware setup, approximately how long would it take to process a sequence of 2048 tokens, assuming all other factors are constant?
Analyzing Computational Scaling