diff --git a/docs/content/docs/getting-started/manual.md b/docs/content/docs/getting-started/manual.md index a09432c2f50..c2da82f7c10 100644 --- a/docs/content/docs/getting-started/manual.md +++ b/docs/content/docs/getting-started/manual.md @@ -43,6 +43,40 @@ curl http://localhost:8080/v1/completions -H "Content-Type: application/json" -d }' ``` +{{% alert icon="💡" %}} + +**Other Docker Images**: + +For other Docker images, please see the table in +https://localai.io/basics/getting_started/#container-images. + +{{% /alert %}} + +Here is a more specific example: + +```bash +mkdir models + +# Download luna-ai-llama2 to models/ +wget https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGUF/resolve/main/luna-ai-llama2-uncensored.Q4_0.gguf -O models/luna-ai-llama2 + +# Use a template from the examples +cp -rf prompt-templates/getting_started.tmpl models/luna-ai-llama2.tmpl + +docker run -p 8080:8080 -v $PWD/models:/models -ti --rm quay.io/go-skynet/local-ai:latest --models-path /models --context-size 700 --threads 4 + +# Now API is accessible at localhost:8080 +curl http://localhost:8080/v1/models +# {"object":"list","data":[{"id":"luna-ai-llama2","object":"model"}]} + +curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ + "model": "luna-ai-llama2", + "messages": [{"role": "user", "content": "How are you?"}], + "temperature": 0.9 + }' +# {"model":"luna-ai-llama2","choices":[{"message":{"role":"assistant","content":"I'm doing well, thanks. How about you?"}}]} +``` + {{% alert note %}} - If running on Apple Silicon (ARM) it is **not** suggested to run on Docker due to emulation. Follow the [build instructions]({{%relref "docs/getting-started/build" %}}) to use Metal acceleration for full GPU support. - If you are running Apple x86_64 you can use `docker`, there is no additional gain into building it from source. @@ -82,6 +116,15 @@ curl http://localhost:8080/v1/completions -H "Content-Type: application/json" -d }' ``` +{{% alert icon="💡" %}} + +**Other Docker Images**: + +For other Docker images, please see the table in +https://localai.io/basics/getting_started/#container-images. + +{{% /alert %}} + Note: If you are on Windows, please make sure the project is on the Linux Filesystem, otherwise loading models might be slow. For more Info: [Microsoft Docs](https://learn.microsoft.com/en-us/windows/wsl/filesystems) {{% /tab %}} @@ -120,31 +163,4 @@ See the [build section]({{%relref "docs/getting-started/build" %}}). {{< /tabs >}} - -### Example (Docker) - -```bash -mkdir models - -# Download luna-ai-llama2 to models/ -wget https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGUF/resolve/main/luna-ai-llama2-uncensored.Q4_0.gguf -O models/luna-ai-llama2 - -# Use a template from the examples -cp -rf prompt-templates/getting_started.tmpl models/luna-ai-llama2.tmpl - -docker run -p 8080:8080 -v $PWD/models:/models -ti --rm quay.io/go-skynet/local-ai:latest --models-path /models --context-size 700 --threads 4 - -# Now API is accessible at localhost:8080 -curl http://localhost:8080/v1/models -# {"object":"list","data":[{"id":"luna-ai-llama2","object":"model"}]} - -curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ - "model": "luna-ai-llama2", - "messages": [{"role": "user", "content": "How are you?"}], - "temperature": 0.9 - }' - -# {"model":"luna-ai-llama2","choices":[{"message":{"role":"assistant","content":"I'm doing well, thanks. How about you?"}}]} -``` - For more model configurations, visit the [Examples Section](https://github.com/mudler/LocalAI/tree/master/examples/configurations). diff --git a/docs/content/docs/getting-started/quickstart.md b/docs/content/docs/getting-started/quickstart.md index 8a405a06b1a..5a3bf088e40 100644 --- a/docs/content/docs/getting-started/quickstart.md +++ b/docs/content/docs/getting-started/quickstart.md @@ -12,7 +12,7 @@ icon = "rocket_launch" ## Installation Methods -LocalAI is available as a container image and binary, compatible with various container engines like Docker, Podman, and Kubernetes. Container images are published on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags&tag=latest) and [Dockerhub](https://hub.docker.com/r/localai/localai). Binaries can be downloaded from [GitHub](https://github.com/mudler/LocalAI/releases). +LocalAI is available as a container image and binary, compatible with various container engines like Docker, Podman, and Kubernetes. Container images are published on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags&tag=latest) and [Docker Hub](https://hub.docker.com/r/localai/localai). Binaries can be downloaded from [GitHub](https://github.com/mudler/LocalAI/releases). {{% alert icon="💡" %}} @@ -121,7 +121,7 @@ docker run -ti -p 8080:8080 localai/localai:{{< version >}}-ffmpeg-core llava ph ## Container images -LocalAI provides a variety of images to support different environments. These images are available on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags) and [Dockerhub](https://hub.docker.com/r/localai/localai). +LocalAI provides a variety of images to support different environments. These images are available on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags) and [Docker Hub](https://hub.docker.com/r/localai/localai). For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA images, if you don't have a GPU, use the CPU images. If you have AMD or Mac Silicon, see the [build section]({{%relref "docs/getting-started/build" %}}). @@ -138,41 +138,38 @@ For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA {{< tabs tabTotal="3" >}} {{% tab tabName="Vanilla / CPU Images" %}} -| Description | Quay | Dockerhub | -| --- | --- | --- | -| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master` | `localai/localai:master` | -| Latest tag | `quay.io/go-skynet/local-ai:latest` | `localai/localai:latest` | -| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}` | `localai/localai:{{< version >}}` | -| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-ffmpeg` | `localai/localai:{{< version >}}-ffmpeg` | +| Description | Quay | Docker Hub | +| --- | --- |-----------------------------------------------| +| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master` | `localai/localai:master` | +| Latest tag | `quay.io/go-skynet/local-ai:latest` | `localai/localai:latest` | +| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}` | `localai/localai:{{< version >}}` | +| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-ffmpeg` | `localai/localai:{{< version >}}-ffmpeg` | | Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-ffmpeg-core` | `localai/localai:{{< version >}}-ffmpeg-core` | {{% /tab %}} {{% tab tabName="GPU Images CUDA 11" %}} - -| Description | Quay | Dockerhub | -| --- | --- | --- | -| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda11` | `localai/localai:master-cublas-cuda11` | -| Latest tag | `quay.io/go-skynet/local-ai:latest-cublas-cuda11` | `localai/localai:latest-cublas-cuda11` | -| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11` | `localai/localai:{{< version >}}-cublas-cuda11` | -| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda11-ffmpeg` | +| Description | Quay | Docker Hub | +| --- | --- |-------------------------------------------------------------| +| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda11` | `localai/localai:master-cublas-cuda11` | +| Latest tag | `quay.io/go-skynet/local-ai:latest-cublas-cuda11` | `localai/localai:latest-cublas-cuda11` | +| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11` | `localai/localai:{{< version >}}-cublas-cuda11` | +| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda11-ffmpeg` | | Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda11-ffmpeg-core` | `localai/localai:{{< version >}}-cublas-cuda11-ffmpeg-core` | {{% /tab %}} {{% tab tabName="GPU Images CUDA 12" %}} - -| Description | Quay | Dockerhub | -| --- | --- | --- | -| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda12` | `localai/localai:master-cublas-cuda12` | -| Latest tag | `quay.io/go-skynet/local-ai:latest-cublas-cuda12` | `localai/localai:latest-cublas-cuda12` | -| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12` | `localai/localai:{{< version >}}-cublas-cuda12` | -| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda12-ffmpeg` | +| Description | Quay | Docker Hub | +| --- | --- |-------------------------------------------------------------| +| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-cublas-cuda12` | `localai/localai:master-cublas-cuda12` | +| Latest tag | `quay.io/go-skynet/local-ai:latest-cublas-cuda12` | `localai/localai:latest-cublas-cuda12` | +| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12` | `localai/localai:{{< version >}}-cublas-cuda12` | +| Versioned image including FFMpeg| `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12-ffmpeg` | `localai/localai:{{< version >}}-cublas-cuda12-ffmpeg` | | Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-cublas-cuda12-ffmpeg-core` | `localai/localai:{{< version >}}-cublas-cuda12-ffmpeg-core` | - {{% /tab %}} {{< /tabs >}}