Model training process halted for small dataset

[Shaw-184]

🐛 Describe the bug

I am using yolonas to train a model on a dataset with 10 images. validation has 1 image. The model trains for 7 epochs and then the process just halts. No errors are printed and no logs are printed. I cant even kill the process.
But the same script works for larger datasets.

Versions

super-gradients==3.7.1
torch==2.3.1
torchmetrics==0.8.0
torchvision==0.18.1

[Shaw-184]

@shaydeci @BloodAxe could you check this out. Some urgent work. Thank you

[BloodAxe]

The more information is provided the better. Would be great if you share what batch size you are using and whether you are training in single gpu or DDP. Is it launch from CLI or Jupyter

[Shaw-184]

Batch size is 2. SIngle GPU training. Running it from CLI

[BloodAxe]

Pytorch DataLoader does not like when dataloader length is smaller than num_workers. Any chance you have len(dataloader) < num_workers?
That is my only idea at the moment.