diffusers: add esrgan

[mudler]

Add esrgan support to enhance image output size.

https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler

[localai-bot]

⚠️⚠️⚠️⚠️⚠️

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful 😸 and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me,
I don't support (yet) replying!

⚠️⚠️⚠️⚠️⚠️

To add ESRGAN support for enhancing the image output size, you can use the following steps:

1. Install the StableDiffusionXL model from the huggingface transformers library using the pip install command in your terminal:

pip install transformers==4.0.0

2. Add the ESRGAN backend to the LocalAI configuration file in the models directory, under the diffusers section:

# Use ESRGAN for image output size enhancement
diffusers:
  pipeline_type: StableDiffusionXLPipeline
  cuda: false # Enable for GPU usage (CUDA)
  scheduler_type: EulerAncestralDiscreteScheduler
  esrgan: true

3. Use the esrgan parameter in the image generation request to enable the output size enhancement:

curl http://localhost:8080/v1/images/generations \
    -H "Content-Type: application/json" \
    -d '{
      "prompt": "<positive prompt>|<negative prompt>", 
      "model": "animagine-xl", 
      "step": 51,
      "size": "1024x1024",
      "esrgan": true
    }'

This will generate an image with enhanced output size using the ESRGAN backend.

Sources:

• https://localai.io/model-compatibility/diffusers/index.html
• https://localai.io/features/image-generation/index.html
• https://localai.io/basics/news/index.html

[LeonSijiaLu]

Hello @mudler , I would want to try taking on this ticket.

Before I begin, I wanted to clarify my understandings. I think the primary goal is to update Diffusers to take on an additional boolean field Upscaler. If this field is set to true, we'll update the python server to utilize the 'StableDiffusionUpscalePipeline'. Please correct me if you think I misunderstood.