Skip to content

Commit

Permalink
Add Projects using lightllm in README.md and add LazyLLM (#453)
Browse files Browse the repository at this point in the history
  • Loading branch information
wzh1994 committed Jul 4, 2024
1 parent b8501a9 commit 45d12ba
Showing 1 changed file with 20 additions and 1 deletion.
21 changes: 20 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,25 @@ python test_llama.py
- `error : PTX .version 7.4 does not support .target sm_89`
- launch with `bash tools/resolve_ptx_version python -m lightllm.server.api_server ... `

## Projects using lightllm

If you have a project that should be incorporated, please contact via email or create a pull request.

1. <details><summary> <b><a href=https://github.com/LazyAGI/LazyLLM>LazyLLM</a></b>: Easyest and lazyest way for building multi-agent LLMs applications.</summary>

Once you have installed `lightllm` and `lazyllm`, and then you can use the following code to build your own chatbot:

~~~python
from lazyllm import TrainableModule, deploy, WebModule
# Model will be download automatically if you have an internet connection
m = TrainableModule('internlm2-chat-7b').deploy_method(deploy.lightllm)
WebModule(t).start().wait()
~~~

Documents: https://lazyllm.readthedocs.io/

</details>

## Community

For further information and discussion, [join our discord server](https://discord.gg/WzzfwVSguU).
Expand All @@ -411,4 +430,4 @@ We learned a lot from the following projects when developing LightLLM.
- [Text Generation Inference](https://github.com/huggingface/text-generation-inference)
- [vLLM](https://github.com/vllm-project/vllm)
- [Flash Attention 1&2](https://github.com/Dao-AILab/flash-attention)
- [OpenAI Triton](https://github.com/openai/triton)
- [OpenAI Triton](https://github.com/openai/triton)

0 comments on commit 45d12ba

Please sign in to comment.