-
-
Notifications
You must be signed in to change notification settings - Fork 780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update README with some explanations #700
Conversation
README.md
Outdated
gradient_accumulation_steps: 1 | ||
# The number of samples to accumulate gradients for, before performing a backward/update pass. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
micro batch size is the per gpu number of samples to accumulate in each forward pass.
micro batch size * gradient accumulations steps * # of gpus = total batch size
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added more explanation below
I absolutely appreciate the work on this! If I may, I'd really like to ask for a couple short examples (a sentence or two) on each option for dummies - technical people who are not from Machine Learning/AI background. For example:
|
I added a link in the doc for more details about the LoRA hyperparameters. PTAL |
@seungduk-yanolja thanks for doing this! much needed. are you happy with the state of this? Should I go ahead and merge? |
yes, please! thanks |
I added one more explanation about
|
Many thanks for this! It means the world to me and to many others. I also found this resource written up for StableDiffusion LoRAs. It seems to be quite relevant https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters. |
* Update README with some explanations * revert commit-hook change * add more explanation about batch size and gradient accum * not use latex foromat * decorate * git hook again * Attach a link that explains about LoRA hyperparameters * update table of content * Explanation about lora_modules_to_save
Description
Added some explanation and examples to the YAML config documentation to help future users.
Motivation and Context
Sharing lesson-learned
How has this been tested?
MD file viewer
Screenshots (if appropriate)
N/A
Types of changes
Comments