-
-
Notifications
You must be signed in to change notification settings - Fork 9.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
๐ fix: MiniMax output long content interrupted by non-existent error #4088
Conversation
@sxjeru is attempting to deploy a commit to the LobeHub Pro Team on Vercel. A member of the Team first needs to authorize it. |
๐ @sxjeru Thank you for raising your pull request and contributing to our Community |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4088 +/- ##
==========================================
+ Coverage 92.45% 92.48% +0.02%
==========================================
Files 482 482
Lines 34354 34497 +143
Branches 3214 3232 +18
==========================================
+ Hits 31763 31905 +142
- Misses 2591 2592 +1
Flags with carried forward coverage won't be shown. Click here to find out more. โ View full report in Codecov by Sentry. |
The latest updates on your projects. Learn more about Vercel for Git โ๏ธ
|
{
"message": "chat response streaming chunk parse error, please contact your API Provider to fix it.",
"context": {
"error": {
"message": "Cannot read properties of undefined (reading '0')",
"name": "TypeError"
},
"chunk": {
"raw": "4890\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"ๆๆฐไปฌไธไธชๆฅไธไธชๅฐๅไธไบ่ฏๅ๏ผไปไปฌ็็
ๆ
ๅผๅงๅฅฝ่ฝฌ๏ผๆ็ปๅ
จ้จๅบทๅคใ่\",\"role\":\"assistant\",\"name\":\"MMๆบ่ฝๅฉ็\",\"audio_content\":\"\"}}],\"created\":1727073422,\"model\":\"abab6.5s-chat\",\"object\":\"chat.completion.chunk\",\"usage\":{\"total_tokens\":0,\"total_characters\":0},\"input_sensitive\":false,\"output_sensitive\":false,\"input_sensitive_type\":0,\"output_sensitive_type\":0,\"output_sensitive_int\":0}\n\n"
}
}
} ่ฏฅ็ๆฌไพๆง่พๅบ้ฟๆๆฌไธญๆญ |
{ This version still outputs long text interruptions |
maxtoken่ฆ้ๅถๅจ4kๅ8kๅ๏ผๆๆๆๆด้ซ็ไธ้ |
Should maxtoken be limited to 4k and 8k? There is obviously a higher limit |
ๆๆๆกฃๅ |
Is there any documentation? |
max_tokens ๅฏไปฅๅจ lobechat ๆๅจ่ฐ๏ผ็ฎๅๆฏ็ปไบไธไธช่พๅคง็้ป่ฎคๅผใ๏ผ็ธ่พไบๅๆฌ็ 256๏ผ ่ฏทๅ ไธ่ฆๅๅนถ๏ผ่ฟๆ้ฎ้ขๅพ ไฟฎๅค๏ผๅๆฐๅชๆฏๅ ถไธญไธไธช้ฎ้ขใ |
max_tokens can be adjusted manually in lobechat, and currently a larger default value is given. (compared to the original 256) Please don't merge yet, there are still problems to be fixed, and parameters are just one of them. |
@sxjeru ๅปบ่ฎฎๆ Minimax ็ๆๆกฃๆฅ็ปmax_token ๅผๅงใ็จๆทไธไธๅฎไผๆ่ฏๅฐ่ฟ้่ฆๆๅจๆนไธๆ่ฝ่ฎฉ 6.5s ่พๅบๅฐ200k+ ใ ไฝ็กฎๅฎ่ฟไธชๅผๆฏๆญฃ็กฎ็ๅ๏ผ6.5s ็กฎๅฎ่ฝ่พๅบ 200k ็output๏ผๆไธชไบบๆ็นไธๅคช็ธไฟก |
@sxjeru suggests setting the max_token value according to the Minimax documentation. Users may not necessarily realize that they need to manually change it to output 200k+ in 6.5s. But are you sure this value is correct? Are you sure it can output 200k output in 6.5s? I personally don't believe it |
ๆๆกฃๆฏ่ฟๆ ทๅ็ใ ่ฟ่พนๆฏ่พๆททๆท็ๆฏ minimax ๅๆฐ้็ max_tokens ็ธๅฝไบ openai ็ Max output tokens๏ผ่ไธๆฏไธไธๆ็ชๅฃๅคงๅฐ๏ผๆไปฅๆๆไผๆฏ่พ่ฐจๆ ๅฐ่ฎพๅฎใ |
The documentation says this. What is confusing here is that max_tokens in the minimax parameter is equivalent to openai's Max output tokens, so I will set it more carefully. |
ๅปบ่ฎฎๆๅฅฝๆตไธ output ่พๅบๆฏๅฆ่ฝๅฐ 20kโฆ ๅฆๆ่ฝๅฐ 20k ่ฏๅฎๅฐฑๆฒก้ฎ้ข |
ๅบ่ฏฅไฟฎๅฅฝไบใminimax ็ๆตๅผ่พๅบๆ็นๆๆ๏ผๆๅไผไธๆฌกๆงๅ่พๅบไธๆฌกๆๆๅ ๅฎนใ |
It should be fixed. |
@sxjeru ci ๆไบ |
@sxjeru ci died |
ๅทฒไฟฎๅคใ |
Fixed. |
@sxjeru ๆๅปบๆไบ |
@sxjeru @LovelyGuYiMeng ่ฏด่ฟๆๆชๆญ็้ฎ้ข๏ผๅฆๅคๅผไธไธช PR ไฟฎๅง |
@sxjeru @LovelyGuYiMeng said there is still a truncation problem, please open another PR to fix it |
โค๏ธ Great PR @sxjeru โค๏ธ The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world. |
็จๅฎขๆท็ซฏ่ฏทๆฑๅฐฑๆฏๆญฃๅธธ็๏ผไธๆธ ๆฅๆฏ่ถ ๆถ่ฟๆฏไปไนๅๅ ใ |
It's normal to use the client to request. I don't know if it's a timeout or what the reason is. |
### [Version 1.19.33](v1.19.32...v1.19.33) <sup>Released on **2024-09-25**</sup> #### ๐ Bug Fixes - **misc**: MiniMax output long content interrupted by non-existent error. #### ๐ Styles - **misc**: Update google provider model info. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's fixed * **misc**: MiniMax output long content interrupted by non-existent error, closes [#4088](#4088) ([4f6e20d](4f6e20d)) #### Styles * **misc**: Update google provider model info, closes [#4129](#4129) ([b1442b9](b1442b9)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>
๐ This PR is included in version 1.19.33 ๐ The release is available on: Your semantic-release bot ๐ฆ๐ |
### [Version 1.62.14](v1.62.13...v1.62.14) <sup>Released on **2024-09-25**</sup> #### ๐ Bug Fixes - **misc**: MiniMax output long content interrupted by non-existent error. #### ๐ Styles - **misc**: Add function call for `taichu_llm`, update google provider model info. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's fixed * **misc**: MiniMax output long content interrupted by non-existent error, closes [lobehub#4088](https://github.com/bentwnghk/lobe-chat/issues/4088) ([4f6e20d](4f6e20d)) #### Styles * **misc**: Add function call for `taichu_llm`, closes [lobehub#4119](https://github.com/bentwnghk/lobe-chat/issues/4119) ([8f629d8](8f629d8)) * **misc**: Update google provider model info, closes [lobehub#4129](https://github.com/bentwnghk/lobe-chat/issues/4129) ([b1442b9](b1442b9)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>
๐ป ๅๆด็ฑปๅ | Change Type
๐ ๅๆด่ฏดๆ | Description of Change
ๅป้คไบ้ป่ฎค็ frequency_penalty ๅๆฐใ
ๅๆถ็ปๅไธไธๆ็ชๅฃ็ญๆจกๅไฟกๆฏ๏ผไผๅไบ้ป่ฎค็ max_tokens ๅๆฐใ
ๅฆๅค็ฑไบ minimax ๆตๅผ่พๅบๅจๆๅไผไธๆฌกๆง่พๅบๆๆๅ ๅฎน๏ผๅฏผ่ดๆๅไธไธช chunk ๆ ๆณ่ขซ่งฃๆไธบ json๏ผๆ ๅธธๅจ็ๆๅฎๆฏๆถๆฅ้ใ็ฎๅๅขๅ ไบๅค็ๆต็จ๏ผๅป้คๆๅไธๆฌก็ data ๆตๆฐๆฎ๏ผไปฅไฟฎๅคๆญค้ฎ้ขใ
ไธชไบบๆฒกๆๆกไปถๆต่ฏ๏ผๅฏไปฅๆต่ฏๅๅๅๅนถใๆต่ฏๅฏ็จใ
๐ ่กฅๅ ไฟกๆฏ | Additional Information
fix #4054