training crash

[zidanexu]

• Python
• PyTorch 1.4
• tesla P40
• centos
  the command line: python train.py --data data/coco.yaml --cfg models/yolov5s.yaml --weights '' --batch-size 128 --resume --device='0,1,2,3,4,5,6,7'
  
  btw, this case has also happened at 46 epochs

[glenn-jocher]

@zidanexu I've not seen this error before, so I'm sorry but I don't think I can help you.

I might recommend training in a common environment, such as our docker container to avoid any potential environment issues. For example environments see https://github.com/ultralytics/yolov5/wiki

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.