_ea_ and _eo_ CONJ+[DET,ADP] wrong contractions

[ceramisch]

Strangely, the conjunction e (and) appears contracted with the next token o or a

This would be correct in Arabic but it is never done in Portuguese, e does not contract with any other word.

Correcting the problem for o can be automatic, but a requires manual intervention because it is ambiguous between a determiner (a, feminine definite article the -- most cases) and the preposition a (to - rarer).

[ceramisch]

Complement: 887 cases in the whole corpus

[dan-zeman]

This seems to be related to / duplicate of #1 and UniversalDependencies/docs#294 by @pedrobalage. (But that pull request has not been merged, so yes, we still need a fix.)

[pedrobalage]

Sorry guys, I didn't have time to follow up this issue. Maybe @ceramisch can help.

[ceramisch]

I think that the fix cannot be done fully automatically (but I can try to take care of it after release 2.2):

• we need to manually re-annotate the UPOS of o and a as PRON or DET
• then, we need to assign its dependencies:
  • we can automatically choose DEPREL=det of the next noun to the right, if UPOS=DET,
  • we can (probably) automatically choose DEPREL=obj if UPOS=PRON, but the target HEAD should be set manually

Then comes my question: any recommendation of an annotation tool capable of dealing with UD2.0 directly, to make this task easier?

[martinpopel]

any recommendation of an annotation tool capable of dealing with UD2.0 directly, to make this task easier?

For manual annotation, see http://universaldependencies.org/tools.html#third-party-tools (but not all the tools listed there support full UD 2.0 including enhanced deps).
For automatic edits, there is e.g. Udapi (I am the main author), which was used e.g. for detection of Portuguese multi-word tokens in the PT-PUD treebank.

[amir-zeldes]

For automatic edits using simple declarative rules (no coding needed) I can also recommend DepEdit, which we use to convert UD_English-GUM:

https://corpling.uis.georgetown.edu/depedit/

[arademaker]

@ceramisch we are using our library cl-conllu and the Emacs mode we developed, both listed in the UD tools.

I just checked that this corpus has many validation issues. Are you planning to solve these issues for this release 2.2 ? I can’t promise to help in the next 2 days.

[dan-zeman]

I think it will be better to leave this corpus out of the shared task. The participants will have enough on their plate even without it, and its inconsistency with Bosque worries me. However, it can still be in the full release 2.2 after the shared task. Then the deadline is June 15.

[ceramisch]

I just checked that this corpus has many validation issues. Are you planning to solve these issues for this release 2.2 ? I can’t promise to help in the next 2 days.

No (as you probably realized). I would be happy to help solving this annoying "eo" issue but I can only work on this after the rush of the PARSEME shared task is over (i.e. mid-May)

[arademaker]

The issue was done by @ceramisch , I only fixed the commits to avoid unnecessary copies of files in the repository.

The python code from @ceramisch marks all tokens that need manual revision with XXXXX and log those tokens in a log file. I moved all logs for a single file scripts/issue-5-py.log. My Lisp code scripts/issue-5.lisp remove the XXXXX from the MISC fields counting them on each file. Unfortunately, the numbers differ.

[ceramisch]

The numbers probably differ because I had manually removed the XXXXX from the cases I had already checked manually. I will try to finish the manual verification for the next release so that this issue can be definitely closed. Thanks for sorting things out with the duplications, @arademaker and sorry for the mess: I think now I understand better the idea of branches for UD treebank development :-)

[arademaker]

I will create a workbench branch in the next days and I will split the files into files with only 10 sentences. This will help to edit the files. So please contact me before any change in the files. Anyway, we are now in the freezing period of UD 2.6 release, no changes in dev and master are allowed.

[ceramisch]

OK, no worries.

[arademaker]

@ceramisch branch workbench created and folder documents/ created.

[arademaker]

see also #9

[arademaker]

this issue was solved, I didn't find cases of ea or eo