-
Notifications
You must be signed in to change notification settings - Fork 885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorrect Spanish verb decomposition #1395
Comments
Thanks, that's a useful observation. We can add it to the training data for MWT, and I'll have a new model ready probably Monday or so. If you find others between now and then, please let us know and I'll add those as well. |
EDIT: Please disregard this comment as it is incorrect. |
I'm trying to figure out - why would the tokenized Generally speaking, the GSD treebank we base the Spanish models from doesn't have accents on any of the uses of
so my thinking is that |
@AngledLuffa You're correct, I'm sorry. Decírselo would indeed be decomposed as decir+se+lo, without the accent, as you mentioned. The only actual error made by Stanza, then, is the first one I pointed out: decírselo as decar-se-lo which should be decir-se-lo (without the accent, as you pointed out). |
Actually it seems the standard in GSD is to keep the text intact but remove
the accents in the lemma.
Some part of me wonders if that means we can deterministic split all the
words aside from a few known exceptions. We found that for English, there
are no exceptions at all, so we split all words into the raw text. Could
do the same thing with exceptions for Spanish
…On Sat, Jun 15, 2024, 6:00 AM Guilherme Gama ***@***.***> wrote:
@AngledLuffa <https://github.com/AngledLuffa> You're correct, I'm sorry.
Decírselo would indeed be decomposed as decir+se+lo, without the accent, as
you mentioned.
The only actual error made by Stanza, then, is the first one I pointed
out: decírselo as decir-se-lo (without the accent, as you pointed out).
—
Reply to this email directly, view it on GitHub
<#1395 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2AYWIJWZJULARJBPDPH33ZHQ3HTAVCNFSM6AAAAABJIT3F2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRZGU2DENJUGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Well, that's certainly a valid line of investigation, and I wish I could contribute further, but unfortunately I don't actually speak Spanish. I use the Stanza constituency tree parser as part of a larger application which requires a constituency tree where the actual tokens are leaves, not the words, so the application reconstructs the tree, but merging the MWT into single leaves, which is how I caught the initial discrepancy in the first place. |
Hey, ran into an issue or two with the Spanish GSD dataset. Once we get that cleaned up I'll retry the models tomorrow. Guess it's time to put on my UD annotator hat... |
Gah, I apologize for how long this is taking. So in the one treebank I was looking at, GSD, the tokens keep the accents after splitting. The same is true in PUD. In AnCora, though, which we were treating as the default, your observation that the accents disappear is correct. Do you have a preference? I don't care too much either way. |
Hi! I have no preference either, and there's also no urgency on my end -
I've written a patch here to just ignore the spellings of the words and use
the full token when merging, so the problem is solved as far as I'm
concerned.
…On Wed, Jun 19, 2024, 3:51 AM John Bauer ***@***.***> wrote:
Gah, I apologize for how long this is taking. So in the one treebank I was
looking at, GSD, the tokens keep the accents after splitting. The same is
true in PUD. In AnCora, though, which we were treating as the default, your
observation that the accents disappear is correct.
Do you have a preference? I don't care too much either way.
UniversalDependencies/UD_Spanish-AnCora#9
<UniversalDependencies/UD_Spanish-AnCora#9>
—
Reply to this email directly, view it on GitHub
<#1395 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACBPVXS6XHK7VNUEWNXLIATZIES6NAVCNFSM6AAAAABJIT3F2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZXHA4DEOJUGA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Got a bunch of data improvements from the UD team. I added the examples above and recreated all of the Spanish models as combined models with both AnCora and GSD. There's a bit of a performance hit when POS tagging the AnCora dev set, which we can investigate some. Otherwise, it seems to be working. I pushed those models as the new defaults for Spanish |
Describe the bug
The token "decírselo" is incorrectly decomposed into the words "decar", "se", "lo". "Decar" is not a word in Spanish. It should be "decír".
To Reproduce
This yields the output
Expected behavior
The expected output is
Environment (please complete the following information):
Additional context
None at the moment.
The text was updated successfully, but these errors were encountered: