Case Study

Pipeline Efficiency Analysis

A team is training a large computational model distributed across a 4-device system, where each device handles a sequential stage of the computation. They observe that the overall device utilization is very low. A detailed activity log shows that at any given moment, only one of the four devices is actively computing, while the other three are idle, waiting for data. The team proposes the following change: 'Instead of processing the entire data batch at once, we will partition it into many smaller, sequential units. As soon as the first device finishes with the first unit, it will pass it to the second device and immediately start on the second unit. This process will continue for all devices and all units.' Evaluate the team's proposed solution. Will it effectively address the problem of low device utilization? Justify your reasoning.

0

1

Updated 2025-10-05

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science