Initial Batch Formation
An LLM inference server employing a continuous batching strategy receives three new, independent user prompts at the same time. Describe the immediate action the system's scheduler will perform with these prompts and name the initial processing phase that this action initiates.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example of the First Decoding Step in Continuous Batching (Iteration 2)
An inference server's scheduler receives two new, independent user requests at the same time. Assuming the system has the capacity to handle both, what is the most accurate description of the scheduler's immediate action and the primary goal of this initial processing step?
Initial Batch Formation
An inference scheduler receives two new, independent requests. Arrange the following events to accurately describe the initial processing step for these requests.