Add integration for ExllamaV2

[kimjaewon96]

library: https://github.com/turboderp/exllamav2
To install exllamav2, you have to install the correct python & cuda version from the releases.

With proper caching, it's 3~4x faster than gptq.

[rlouf]

Thank you! Did you try guided generation with this integration?

[kimjaewon96]

prompt = "What is the IP address of the Google DNS servers? "
unguided = outlines.generate.text(model, max_tokens=30)(prompt)
guided = outlines.generate.regex(
    model,
    r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)",
    max_tokens=30,
)(prompt)
print(unguided)
#\n\n Google Public DNS is a free, global, open recursive DNS service from Google. There are two IP addresses for Google Public D
print(guided)
#0.0.0.0

Yes, it works.

One thing I forgot to mention is that exl2 doesn't automatically download models from huggingface, so you should write the local folder path containing the model.

[rlouf]

Thank you for contributing! We will need to add some documentation for that in the near future.

[dnhkng]

Can someone provide a quick example for loading the exllama model?