LoRAX allows serving multiple fine-tuned models optimized on the same endpoint by dynamically loading and switching LoRA adapters.
The following command deploys Mistral 7B Instruct as a base model via a service:
dstack run . -f examples/deployment/lorax/serve.dstack.yml
See the configuration at serve.dstack.yml.
The following command runs Mistral 7B Instruct as a base model via a task:
dstack run . -f examples/deployment/lorax/serve-task.dstack.yml
See the configuration at serve.dstack.yml.