When multiple GPUs are available in PyTorch, the $i^{ extrm{th}}$ GPU is represented using torch.device(f'cuda:{i}'), where the index $i$ starts at $0$. The notations cuda:0 and cuda are equivalent and both refer to the first GPU.