Concept

GPU Warm-up for Accurate Benchmarking

When benchmarking GPU computations, it is important to perform a preliminary warm-up pass on each device before taking timing measurements. This initial execution ensures that hardware caches, memory allocators, and GPU driver overhead are fully initialized and do not distort the benchmark results. In practice, this involves running the target workload once on every device involved—without recording the time—and then synchronizing all devices before beginning the actual timed measurement.

0

1

Updated 2026-05-18

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L