Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--prune-lexicon option in abkhazia language #10

Open
mmmaat opened this issue Jul 28, 2017 · 0 comments
Open

--prune-lexicon option in abkhazia language #10

mmmaat opened this issue Jul 28, 2017 · 0 comments
Labels
enhancement

Comments

@mmmaat
Copy link
Collaborator

mmmaat commented Jul 28, 2017

Removes from the lexicon in test and train all words that are not present at least once in the training set.

Could be useful when using a lexicon that is tailored to the corpus to the point of overfitting (i.e. only words occuring in the corpus were included and many other common words weren't), which could lead to overestimated performance on words from the lexicon appearing in the test only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mmmaat