Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NER模型是否会支持fine-tune和自定义字典 #16

Open
Growing-bison opened this issue Sep 25, 2020 · 3 comments
Open

NER模型是否会支持fine-tune和自定义字典 #16

Growing-bison opened this issue Sep 25, 2020 · 3 comments

Comments

@Growing-bison
Copy link

老师您好,感谢您的工作和分享!
看到目前有base和larege两个模型,想请教下老师,这两个模型是否能支持fine-tune呢?另外,fastnlp中的训练好的模型,能否再次直接载入?
同时看到一些开源工作,不少有支持添加自定义字典,老师您这边会考虑吗?
谢谢

@fdugzc
Copy link
Member

fdugzc commented Sep 25, 2020

目前还没找到fine-tune后不影响其他任务性能的解决方案,所以暂时不会添加fine-tune功能;添加自定义词典的功能正在开发、测试,将在1-3周内更新上来

@Growing-bison
Copy link
Author

谢谢您!了解了,看来fine-tune还比较麻烦。fastNLP中提供有基于bert训练的功能,同时可用自主样本进行训练,期待fastHan融合更好。
(1)试了某NLP工具套件,提供有fine-tune功能,但它们的样本标注方式采用自家的(以NER为例)。如果您项目以后考虑fine-tune功能,希望到时考虑通用些的方式

期待自定义词典功能!

@caseylai
Copy link

caseylai commented Apr 7, 2022

请问下目前版本中NER可以支持自定义词典了吗?看了文档中介绍1.6已经将用户词典功能扩充到所有任务,但实际试了下1.7的词典好像对NER任务无效。

s = "在奈雪的茶买饮料"
model(s, 'NER')
[[['奈雪', 'NS']]]
model.add_user_dict(["奈雪的茶"])
model(s, 'NER')
[[['奈雪', 'NS']]]
model(s, 'NER', use_dict=True)
[[['奈雪', 'NS']]]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants