add TPU tests

[Borda]

🚀 Feature

we shall also cover TPU usage as we are supporting it

Motivation

now all changes are tested for GPUs and CPU but we do not have a check for TPU yet

Pitch

getting coverage back to ~99%

[awaelchli]

TPU's are available in CI testing? Or are you saying there is a way to deploy it to colab?

[Borda]

TPU's are available in CI testing? Or are you saying there is a way to deploy it to colab?

not yet, by we will try to get one as we got also the GPUs some time ago :]

[Borda]

@jeremyjordan mind add optional TPU tests which would also help to you #1211 which would be helpful and simplify your testing...

[jeremyjordan]

i'm probably not the best person for TPU tests, i haven't dug into that code much yet (and not sure if i'll have time in the upcoming week or two - want to make progress on #1099 this weekend)

[luiscape]

@Borda we could do TPU testing much like we do GPU testing, i.e. deploy a GCP instance with a CI system (e.g. Drone :) ) and then have it read a webook from GitHub. There are cost implications to that, but I think you can make that work.

Let's chat about how to make that happen. Maybe we can also get free credits from Google to test on TPUs?

[Borda]

Let's chat about how to make that happen. Maybe we can also get free credits from Google to test on TPUs?

That would be great! Just thinking coukd it be possible to have instance which has both - two GPUs and TPU in the same time?

[luiscape]

I am not sure, actually. We'll have to look into it.

[lezwon]

@Borda could we have something like this in lightning? pytorch/ignite#981

[Borda]

Yes, that is nice, the other option is to launch another instance of Drone.io

[lezwon]

@Borda That's great. I'll be happy to take up this issue if we can get the CI system for TPU's set up. :)

[Borda]

Mind prepare in meantime tests for TPU and we prepare the test env :]