support pytorch-lightning

[paantya]

Please tell me, is it possible to replace the dataloader in pytorch-lightning, inside the datamodule with yours and will it start up easily?
or will dances with tambourines be needed?

[paantya]

Lightning-AI/pytorch-lightning#11553

[paantya]

Lightning-AI/pytorch-lightning#11538

[GuillaumeLeclerc]

Hello,

We currently have a temporary solution (i.e. dances with tambourines) but PTL guys are working hard so that you can get the benefit of both 😃

https://gist.github.com/GuillaumeLeclerc/49faabbe399c6cf21cfb8e9711249e10

[tchaton]

Hey @GuillaumeLeclerc. Would you mind trying out our latest integration (Bagua) in PyTorch Lightning (not merged yet). Here is the branch: https://github.com/PyTorchLightning/pytorch-lightning/pull/11146/files#diff-ce0c8bb0f522dbc594912bc908a6511a9e237c54eaca71043026a2c62a71de4eR543

Here is their repo: https://github.com/BaguaSys/bagua

I am quite curious to see ffcv + PL + Bagua Async (with their launcher and autotuning mechanism) speed :)

[GuillaumeLeclerc]

@lengstrom I know you have all the benchmark setups already on hand, would you mind trying that out ?