NotImplementedError: Cannot copy out of meta tensor; no data!

[A-Akhil]

When i run this

//set device
device = 'cuda' # or cpu
torch.set_default_device(device)

all other code (https://huggingface.co/BAAI/Bunny-v1_1-Llama-3-8B-V)

//generate
output_ids = model.generate(
input_ids,
images=image_tensor,
max_new_tokens=100,
use_cache=True,
repetition_penalty=1.0 # increase this to avoid chattering
)[0]

I get a error stating

NotImplementedError Traceback (most recent call last)

in <cell line: 2>()
1 # generate
----> 2 output_ids = model.generate(
3 input_ids,
4 images=image_tensor,
5 max_new_tokens=100,

26 frames

/usr/local/lib/python3.10/dist-packages/torch/utils/_device.py in torch_function(self, func, types, args, kwargs)
76 if func in _device_constructors() and kwargs.get('device') is None:
77 kwargs['device'] = self.device
---> 78 return func(*args, **kwargs)
79
80 # NB: This is directly called from C++ in torch/csrc/Device.cpp

NotImplementedError: Cannot copy out of meta tensor; no data!

[Isaachhh]

What is your GPU?