When multiple GPUs are available in PyTorch, the $i^{\textrm{th}}$ GPU is represented using torch.device(f'cuda:{i}'), where the index $i$ starts at 0. The notations cuda:0 and cuda are equivalent and both refer to the first GPU.