-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About Preprocessing in Juman.apply_to_sentence
#121
Comments
Juman.apply_to_sentence
Juman.apply_to_sentence
The preprocessing steps performed by
|
It's important to note that sentences beginning with |
Let's carry on this discussion on ku-nlp/jumanpp#154. |
#123 will fix the handling of half-width spaces and straight double quotation marks. |
It appears that some preprocessing takes place when performing morphological analysis with
Juman.apply_to_sentence
. For example, half-width spaces are replaced with full-width spaces, and line breaks are removed.Are there other such preprocessings?
The text was updated successfully, but these errors were encountered: