New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

疑惑：没有看到哪里执行了 text 2 tokens 的过程 #119

Open

EdwardChan5000 opened this issue May 18, 2023 · 1 comment

EdwardChan5000 commented May 18, 2023

看到 _create_multimodal_bert_features 是被 m3tl_preproc_fn.py 通过 @preprocessing_fn 调用 decorator
应该是 input_fn.py 中 write_tfrecord 调用 chain_processed_data 的时候触发？
还是哪里？可以 q 一下相应调用

Owner

JayYip commented May 29, 2023

这里: https://github.com/JayYip/m3tl/blob/63c44c37d4b2d695d5074dd76e28a0273d00064f/m3tl/bert_preprocessing/create_bert_features.py#LL290C28-L290C28

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment