RPG support since it's using regional prompting?

[andupotorac]

This is a nice update to StreamDiffusion. I'm wondering if it can support the RPG-DiffusionMaster improvements, that are enabling high-fidelity images from the prompts. Right now it would be a compromise between using it with a lower speed, vs using a higher speed with StreamMultiDiffusion, but lower quality output in terms of prompt fidelity.

[andupotorac]

The reason why I'm asking is because I read in your doc you solved the regional prompting issue with the blending of the regions with quantized masks, and basically this same approach is likely to fix the issue with RPG too, as they're using this approach (and a visual LLM) to solve for better alignment in the output of the prompts.

[ironjr]

Sorry for the late reply. I just have read the RPG paper. Thanks for the nice suggestion.
I will run their code and try adopting their pipeline into mine and see if this works. But this will take some time!

[andupotorac]

Thanks! We can try to do it and ask you questions here and there if you want, and then submit an update to your code if required. Would that be ok?

[ironjr]

Issues and pull requests are always welcome :)