FFCV with CIFAR10

[salelkafrawy]

🐛 Bug

ValueError: Tried to step 527 times. The specified number of total steps is 525

a bug surfaced when I used FFCV loader with CIFAR10 dataset. I read that FFCV hasn't been fully supported by PL but as @carmocca mentioned here that we can use FFCV and remove the ToDevice transformations with bugs that could surface, so this issue is one of two bugs I've been facing.

I tried FFCV with MosaicML and it worked fine (the code is in the same notebook), but it had errors when used with PL

To Reproduce

The last part in this Colab has the bug:
https://colab.research.google.com/drive/1DKJfDsDLAGrJLcSB9-kn7G29vc_6EHuU?usp=sharing

Expected behavior

I expected to run smoothly as the Pytorch's DataLoader has.

Environment

• CUDA:
  - GPU:
  - Quadro RTX 8000
  - available: True
  - version: 11.3
• Packages:
  - numpy: 1.22.4
  - pyTorch_debug: False
  - pyTorch_version: 1.12.0
  - pytorch-lightning: 1.6.5
  - mosaicml: 0.8.0
  - tqdm: 4.64.0
• System:
  - OS: Linux
  - architecture:
  - 64bit
  - ELF
  - processor: x86_64
  - python: 3.9.13
  - version: Logging of GPU memory utilization can significantly slow down training #189-Ubuntu SMP Wed May 18 14:13:57 UTC 2022

Additional context

if you could explain how PL is different than MosaicML in handling dataloading I'd appreciate it.

[salelkafrawy]

I figured out that using ffcv.transforms.NormalizeImage instead of torchvision.transforms.Normalize caused the errors and now it works with PL