Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

forward lang ids in TransformerWordEmbeddings #2417

Merged
merged 2 commits into from
Sep 10, 2021

Conversation

helpmefindaname
Copy link
Collaborator

closes #2416

@helpmefindaname helpmefindaname marked this pull request as draft September 9, 2021 08:32
@helpmefindaname helpmefindaname marked this pull request as ready for review September 9, 2021 08:55
@alanakbik
Copy link
Collaborator

Hello @helpmefindaname thanks for the PR! I did not know that some transformers require language IDs to improve embeddings. Is there documentation on this somewhere, bzw a list of all transformer models that use language IDs?

@helpmefindaname
Copy link
Collaborator Author

Hey @alanakbik sure, here is the documentation: https://huggingface.co/transformers/multilingual.html#xlm-language-embeddings
it is basically a feature of XLM, there are also more models based on XLM that benefit from it, in my case it is https://huggingface.co/microsoft/unihanlm-base

@alanakbik
Copy link
Collaborator

@helpmefindaname thanks for adding this!

@alanakbik alanakbik merged commit cb3946b into flairNLP:master Sep 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Propagate language ids in TransformerEmbeddings
2 participants