When multiple GPUs are available in PyTorch, the $$i^{	extrm{th}}$$ GPU is represented using torch.device(f'cuda:{i}'), where the index $$i$$ starts at $$0$$. The notations cuda:0 and cuda are equivalent and both refer to the first GPU.

Indexing Multiple GPUs in PyTorch

In PyTorch, a single GPU can be indicated by torch.device('cuda'). Unlike the CPU device, a GPU device only represents one specific graphics card and its corresponding memory, not all GPUs on the system.

Claude

CUDA helps keep track of which GPU is being used and ensures the tensors allocated will be on that device. It allows you to leverage GPU computational speed for certain tensor operations but also has functionality that allows you to move the tensor between the CPU and GPU.

CUDA in PyTorch

Dive into Deep Learning

GPU Device Allocation in PyTorch

The number of available GPUs on a system can be queried in PyTorch using the function torch.cuda.device_count().

Learn Before

Related

Learn After