Learn Before
Concept
GPU Warm-up for Accurate Benchmarking
When benchmarking GPU computations, it is important to perform a preliminary warm-up pass on each device before taking timing measurements. This initial execution ensures that hardware caches, memory allocators, and GPU driver overhead are fully initialized and do not distort the benchmark results. In practice, this involves running the target workload once on every device involved—without recording the time—and then synchronizing all devices before beginning the actual timed measurement.
0
1
Updated 2026-05-18
Tags
D2L
Dive into Deep Learning @ D2L