Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added chatglm3 conversation type for training models like TinyLLama #1036

Merged
merged 6 commits into from
Jan 4, 2024

Conversation

xaviviro
Copy link
Contributor

@xaviviro xaviviro commented Jan 3, 2024

No description provided.

@xaviviro xaviviro changed the title Added chatgml3 conversation type for training models like TinyLLama Added chatglm3 conversation type for training models like TinyLLama Jan 3, 2024
@NanoCode012
Copy link
Collaborator

Hey! Thanks for PR. Could you run this through the linter also?

@xaviviro
Copy link
Contributor Author

xaviviro commented Jan 3, 2024

Hi there, I've run the code through Pylint and made a couple of minor spacing adjustments as suggested by the tool. Please let me know if there's anything else that needs to be addressed for this PR. Thank you!

@NanoCode012
Copy link
Collaborator

Leaving this here for reference implementation: https://github.com/lm-sys/FastChat/blob/722ab0299fd10221fa4686267fe068a688bacd4c/fastchat/conversation.py#L169-L178

I find it weird that there's a space before message since I don't see it within their docs, but fschat has it.

https://github.com/THUDM/ChatGLM3/blob/main/PROMPT_en.md

@xaviviro
Copy link
Contributor Author

xaviviro commented Jan 3, 2024

You're right, I have copied the code from https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py

I've run a train and the fine-tuned model seems to be working incredibly well.

@NanoCode012 NanoCode012 merged commit 59b2d30 into axolotl-ai-cloud:main Jan 4, 2024
4 of 6 checks passed
@NanoCode012
Copy link
Collaborator

Thank you for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants