How to use multiple gpus for tnet?

[veralauee]

The papers says that the performance will be better if embedding size gets larger. In the pretrained model, the embedding size is 64. I want to train this model with embedding size =512.
In such case, I need to use multiple gpus, which should be dataparallel in pytorch.
tnet = nn.DataParallel(tnet).cuda() print('now we are using %d gpus'%torch.cuda.device_count())

However, it shows that

Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 0 does not equal 1 (while checking arguments for cudnn_convolution)

Would you please tell me how to use multiple gpus to train this model?

[BryanPlummer]

We never used multiple GPUs in these experiments. Rather, we reduced the batch size when necessary. We would expect, however, that performance would be further improved if the batch size remained the same (or was even increased further) since more accurate gradients would be provided to the model during training.