数据集预处理
包含lm,孪生,seq2seq,文本分类,示例参考dataDome目录下
默认使用Bert 21128分词方案,如果想要修改自己的分词可以修改config下的词典方案。
Download links:
SSH clone URL: ssh://git@git.jetbrains.space/terrychanorg/yuxunlianlm-bert/BulidDataset.git
HTTPS clone URL: https://git.jetbrains.space/terrychanorg/yuxunlianlm-bert/BulidDataset.git
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
What things you need to install the software and how to install them.
Examples
Add additional notes about how to deploy this on a production system.
Add links to external resources for this project, such as CI server, bug tracker, etc.
tkitDatasetEx各种函数