Short Answer

Calculating Sequential Processing Inefficiency

Consider a 4-layer neural network being trained using a strategy where each layer is assigned to a separate worker device (Worker 1 for Layer 1, Worker 2 for Layer 2, etc.). The processing of a single batch of data proceeds sequentially from Worker 1 to Worker 4. If the forward pass for each layer takes exactly 100 milliseconds (ms), what is the total combined idle time for all four workers during the processing of one complete forward pass? Explain your reasoning.

0

1

Updated 2025-10-06

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Analysis in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science