Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mistral dropout #798

Closed
wants to merge 7 commits into from
Closed

Mistral dropout #798

wants to merge 7 commits into from

Conversation

winglian
Copy link
Collaborator

adds a gaussian dropout layer after the MLP layer and cleans up the model config so it's more easily modifiable from the YML

dropout for mistral can be added via

model_config:
  dropout_p: 0.001

@winglian winglian mentioned this pull request Oct 28, 2023
@Mihaiii
Copy link

Mihaiii commented Nov 4, 2023

Any news on this? :) I'd love to try out the new config.

@winglian
Copy link
Collaborator Author

superseded by #858

@winglian winglian closed this Nov 15, 2023
@winglian winglian deleted the mistral-dropout branch November 17, 2023 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants