Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Modal refs from stub to app #974

Merged
merged 1 commit into from
Jun 15, 2024
Merged

Update Modal refs from stub to app #974

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/cookbook/deploy-using-modal.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ In this guide we will show you how you can use Modal to run programs written wit
First we need to define our container image. We download the Mistral-7B-v0.1 model from HuggingFace as part of the definition of the image so it only needs to be done once.

```python
from modal import Image, Stub, gpu
from modal import Image, App, gpu

stub = Stub(name="outlines-app")
app = App(name="outlines-app")

outlines_image = Image.debian_slim(python_version="3.11").pip_install(
"outlines==0.0.37",
Expand Down Expand Up @@ -70,10 +70,10 @@ schema = """{
}"""
```

To make the inference work on Modal we need to wrap the corresponding function in a `@stub.function` decorator. We pass to this decorator the image and GPU on which we want this function to run (here an A100 with 80GB memory):
To make the inference work on Modal we need to wrap the corresponding function in a `@app.function` decorator. We pass to this decorator the image and GPU on which we want this function to run (here an A100 with 80GB memory):

```python
@stub.function(image=outlines_image, gpu=gpu.A100(memory=80))
@app.function(image=outlines_image, gpu=gpu.A100(memory=80))
def generate(
prompt: str = "Amiri, a 53 year old warrior woman with a sword and leather armor.",
):
Expand All @@ -94,14 +94,14 @@ def generate(
We then need to define a `local_entrypoint` to call our function `generate` remotely:

```python
@stub.local_entrypoint()
@app.local_entrypoint()
def main(
prompt: str = "Amiri, a 53 year old warrior woman with a sword and leather armor.",
):
generate.remote(prompt)
```

Here `@stub.local_entrypoin()` decorator defines `main` as the function to start from locally when running the Modal CLI. You can save above code to `example.py` (or use [this implementation](https://github.com/outlines-dev/outlines/blob/main/examples/modal_example.py)). Let's now see how to run the code on the cloud using the Modal CLI.
Here `@app.local_entrypoint()` decorator defines `main` as the function to start from locally when running the Modal CLI. You can save above code to `example.py` (or use [this implementation](https://github.com/outlines-dev/outlines/blob/main/examples/modal_example.py)). Let's now see how to run the code on the cloud using the Modal CLI.

## Run on the cloud

Expand Down
6 changes: 3 additions & 3 deletions examples/modal_example.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import modal

stub = modal.Stub(name="outlines-app")
app = modal.App(name="outlines-app")


outlines_image = modal.Image.debian_slim(python_version="3.11").pip_install(
Expand Down Expand Up @@ -58,7 +58,7 @@ def import_model():
}"""


@stub.function(image=outlines_image, gpu=modal.gpu.A100(memory=80))
@app.function(image=outlines_image, gpu=modal.gpu.A100(memory=80))
def generate(
prompt: str = "Amiri, a 53 year old warrior woman with a sword and leather armor.",
):
Expand All @@ -74,7 +74,7 @@ def generate(
print(character)


@stub.local_entrypoint()
@app.local_entrypoint()
def main(
prompt: str = "Amiri, a 53 year old warrior woman with a sword and leather armor.",
):
Expand Down