The following command deploys Llama 7B Instruct as a service:
dstack run . -f examples/deployment/tgi/serve.dstack.yml
See the configuration at serve.dstack.yml.
The following command runs Llama 7B Instruct as a task:
dstack run . -f examples/deployment/tgi/serve-task.dstack.yml
See the configuration at serve.dstack.yml.