New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pretokenized span 개선 #170

Merged

bab2min merged 5 commits into main from fix_pretokenized

Jun 13, 2024

Owner

bab2min commented Jun 13, 2024

pretokenized span으로 JX 태그가 아닌 문자열에 대해 JX를 강제로 부여할 때 발생하는 오류 수정
pretokenized span으로 동사를 지정했을 때 문맥에 따라 규칙성 여부(-R / -I)를 자동으로 탐지하도록 기능 개선

bab2min added 5 commits

June 14, 2024 00:25


          Refactor Types.h and Joiner.cpp for improved tag comparison

93f180a


          Fix null pointer exception in Kiwi.cpp

b593c23


          Add auto inference for regularity to pretokenized tokens

02195e1


          Add support for pretokenized spans in KiwiCpp tests


          Fix compilation errors at gcc 4.8.x

69f5a21

bab2min merged commit f44d176 into main

14 checks passed

bab2min deleted the fix_pretokenized branch

June 13, 2024 16:09

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet