Calculating Pipeline Completion Time
A computational model is distributed across 4 workers in a pipeline. A data batch is split into 8 identical micro-batches. The time required for any single worker to process any single micro-batch is a constant T. The first worker starts processing the first micro-batch at time t=0. At what time will the fourth (final) worker complete its processing of the eighth (final) micro-batch? Explain the components of your calculation.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A large computational model is divided into 4 sequential stages, with each stage running on a separate hardware worker. To improve efficiency, a data batch is split into multiple smaller 'micro-batches' which are processed sequentially through the 4 workers. A worker begins processing a new micro-batch as soon as it has passed the previous one to the next worker in the sequence. At the exact moment the 4th worker begins processing the 1st micro-batch, what is the 1st worker doing?
Calculating Pipeline Completion Time
Consider a computational pipeline with 4 sequential workers processing a stream of micro-batches. The pipeline operates such that a worker begins processing a micro-batch as soon as it receives it from the previous worker. At the exact moment that the second worker (worker 2) finishes processing the first micro-batch, which of the following statements accurately describes the state of the entire system?