How to generate alignment based on token level?

[michelleqyhqyh]

I can use your command to generate alignment based on bpe level. But how to generate alignment based on token level?

[carboncoo]

By default, the generated alignment is already word-level, as indicated by https://github.com/THUNLP-MT/Mask-Align/blob/main/thualign/utils/alignment.py#L168 with the remove_bpe parameter of the weights_to_align method.

[carboncoo]

This is hard coded because we normally only care about word-level alignment. You can change the default value of `remove_bpe` by yourself. 在2022-09-28 ***@***.***写道： By default, the generated alignment is already word-level, as indicated by https://github.com/THUNLP-MT/Mask-Align/blob/main/thualign/utils/alignment.py#L168 with the remove_bpe parameter of the weights_to_align method. Where can I set "remove_bpe" parameter? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>