sat-12l-sm running on GPU

[Randwow]

Hi @bminixhofer,

I'm trying to use sat-12l-sm on GPU with the following code:

MODEL_NAME = "sat-12l-sm"
sat = SaT(MODEL_NAME)
sat.to("cuda")

However, when I run nvidia-smi in the terminal, it doesn't show any usage of the GPU, and it seems that the GPU is not being utilized. Could you please provide any guidance or suggestions on how to ensure that the model is actually using the GPU?

Thank you!

[markus583]

Hi, this is a bit odd, and should not be the case. Is your torch properly set up? Did you check with nvidia-smi also after calling sat.split("some text")?
I just tried it myself and it works as intended. The time needed is also much lower after doing sat.cuda(). I suggest you compare time needed to segment some sentences on CPU and GPU; the latter should be an order of magnitude faster.

[Randwow]

Hi, yes I will check thank you )

[lifeiteng]

Hi, this is a bit odd, and should not be the case. Is your torch properly set up? Did you check with nvidia-smi also after calling sat.split("some text")? I just tried it myself and it works as intended. The time needed is also much lower after doing sat.cuda(). I suggest you compare time needed to segment some sentences on CPU and GPU; the latter should be an order of magnitude faster.

can we set cuda index?

[markus583]

Hi, this is a bit odd, and should not be the case. Is your torch properly set up? Did you check with nvidia-smi also after calling sat.split("some text")? I just tried it myself and it works as intended. The time needed is also much lower after doing sat.cuda(). I suggest you compare time needed to segment some sentences on CPU and GPU; the latter should be an order of magnitude faster.

can we set cuda index?

Should be possible, yes. It is no different than other PyTorch models in this regard.

[lifeiteng]

There is a bug in class PyTorchWrapper

sat = SaT("sat-3l")

# this is not right
sat = sat.to(device)

# I used this
_ = sat.model.model.to(device)

# source code
class PyTorchWrapper:
    def __init__(self, model):
        self.model = model
        self.config = model.config

    def __getattr__(self, name):
        assert hasattr(self, "model")
        return getattr(self.model, name)

[markus583]

Why is this a bug? I have been using sat = sat.to("cuda") and it worked just fine. Is it any different if you use, e.g., "cuda:2"?