Error when generating sample from a trained SkyGPT

[GabbySuwichaya]

Hello, I would like to kindly ask for your help about running the inference of SkyGPT. Here, I want to compare the generated prediction of the images in 15 mins a head produced by SkyGPT. Your answer will greatly my evaluation on the performance of your work against other state-of-the-art methods, and your paper will be cited accordingly...

I encountered the following problems associated with the inference part of SkyGPT. Please kindly let me know if my understanding is incorrect.

Firstly, I am trying to get the generated prediction of the images by running SkyGPT.sample(n, batch) where n is (what I assumed) the number of samples to be generated, and batch is the input video (or image squence). From the Stanford data, the size of the input video is Nx3x16x64x64.

• First question: are the prediction images generated from the following command ? Is this correct?

self.vqvae.decode(samples)

which is the first output from SkyGPT.sample(n, batch)...

• However, the number of predicted samples from self.vqvae.decode(samples) is equal to sequence_length (or 16 samples) .... But what if I want the number of predicted samples to be 15 , which parameters should be adjusted?

• Another problem is that I always at the error about the following lines regarding the mismatching dimension between the initialized samples_phycell[batch_idx] and embedding_phycell

# newly added
samples_phycell[batch_idx] = embedding_phycell
samples_transformer[batch_idx] = embedding_transformer

Here is the screen shot for SkyGPT.sample(n, batch)...