We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
用fastchat部署AgentLM-13B,推理的时候格式是乱的,尤其是streaming的模式,每行只有几个字符就切换到下一行了,一个单词被切成了好几个字母或字母组合。如果手动用transformer加载并用gradio展示的话就没有这个问题,用fastchat的debug模式看了一下,用的是LlamaForCausalLM加载的模型,应该没错
The text was updated successfully, but these errors were encountered:
我们之前没有在 FastChat 上进行过推理实验。能否给出更详细的报错、不符合预期的场景或者配置?
Sorry, something went wrong.
老师,请问您是怎么操作的呢
hi @Btlmd @heimy2000 , 其实就是简单的使用 fastchat.serve.cli 来加载模型进行基础对话,其实没有报错,只是输出的chunk就很奇怪并且不成句,其实你们可以试试看,操作很简单 python3 -m fastchat.serve.cli --model-path AgentTuning/AgentLM --model-path后面就是加的实际存储的AgentLM权重
python3 -m fastchat.serve.cli --model-path AgentTuning/AgentLM
--model-path
No branches or pull requests
用fastchat部署AgentLM-13B,推理的时候格式是乱的,尤其是streaming的模式,每行只有几个字符就切换到下一行了,一个单词被切成了好几个字母或字母组合。如果手动用transformer加载并用gradio展示的话就没有这个问题,用fastchat的debug模式看了一下,用的是LlamaForCausalLM加载的模型,应该没错
The text was updated successfully, but these errors were encountered: