Name		Name	Last commit message	Last commit date
parent directory ..
README.md		README.md
__init__.py		__init__.py
cli.py		cli.py
constants.py		constants.py
controller.py		controller.py
conversation.py		conversation.py
gradio_css.py		gradio_css.py
gradio_patch.py		gradio_patch.py
gradio_web_server.py		gradio_web_server.py
inference.py		inference.py
model_worker.py		model_worker.py
monkey_patch_non_inplace.py		monkey_patch_non_inplace.py
register_worker.py		register_worker.py
test_message.py		test_message.py
test_throughput.py		test_throughput.py
utils.py		utils.py

README.md

How to serve

The serving code is based on Gradio and largely adapted from Vicuna repository.

Run the controller

python3 -m serve.controller

Run the model worker

python3 -m serve.model_worker --model-path $MODEL_PATH --port 21002 --worker-address=http://localhost:21002 --model-name=SelFee-13b

Run the web server

python3 -m serve.gradio_web_server --share