Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fine-Tuning Mistral-7b for Real-World Chatbot Applications Using Axolotl (Lora used) #1155

Merged
merged 9 commits into from
Jan 23, 2024
Merged

Fine-Tuning Mistral-7b for Real-World Chatbot Applications Using Axolotl (Lora used) #1155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
dfa8f25
Mistral-7b finetune example using axolotl with code,config,data
twenty8th Jan 19, 2024
7b3a386
Create README.md
Tilemachoc Jan 19, 2024
29a45e1
Update README.md
Tilemachoc Jan 19, 2024
a98d531
Update README.md
Tilemachoc Jan 19, 2024
afd5893
Update README.md
Tilemachoc Jan 20, 2024
a965d37
Update data.jsonl
Tilemachoc Jan 21, 2024
4b1c88c
Update config.yml
Tilemachoc Jan 21, 2024
1900a0c
Update data.jsonl
Tilemachoc Jan 21, 2024
7ece965
chore: lint
winglian Jan 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
## What's New
- Added `Mistral-7b-example`: A comprehensive example for fine-tuning Mistral-7b model. [Check it out here](https://github.com/Tilemachoc/axolotl/tree/mistral-7b-example/examples/mistral/Mistral-7b-example).

# Axolotl

Axolotl is a tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures.
Expand Down
12 changes: 12 additions & 0 deletions examples/mistral/Mistral-7b-example/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Description
This repository presents an in-depth guide for fine-tuning Mistral-7b or any other compatible model using Axolotl, tailored specifically for chatbot development. It streamlines the process of fine-tuning and uploading the enhanced model to HuggingFace 🤗, thereby serving as an invaluable tool for developers in the AI and chatbot domain.

**What’s Inside:**

Beginner-Friendly Instructions: Comprehensive steps to guide you through fine-tuning your chosen model, including details on the data structure (jsonl), configuration, and the code itself.

Hardware Utilized: For reference, the fine-tuning in this guide was performed using 4x NVIDIA GeForce RTX 3090 (rented 2.1.2-cuda12.1-cudnn8-devel).

**Uploading to HuggingFace 🤗:**
To upload your fine-tuned model to Hugging Face, include the following files:
![Screenshot 2024-01-19 213932](https://github.com/OpenAccess-AI-Collective/axolotl/assets/138583191/d660eb84-2d76-46a1-9846-cf0aeb3006d9)
Loading