A question about tokenizer file

[hideinhat]

Hello, I am trying to finetune the TrOCR to recognize latex math expression on my custom dataset.
Do I need to train a custom tokenizer for latex format?
Or is the pretrained tokenizer one included with TrOCR good enough?
Any pointers or help would be greatly appreciated.
Thank you!

[win5923]

Yes, my results using the pre-trained tokenizer were not satisfactory. I believe it's necessary to train a dedicated tokenizer based on mathematical symbols. Perhaps there already exists.

processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")

[hideinhat]

Yes, my results using the pre-trained tokenizer were not satisfactory. I believe it's necessary to train a dedicated tokenizer based on mathematical symbols. Perhaps there already exists.

processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")

Thank you for reply. I find one on https://huggingface.co/witiko/mathberta. I will try on it.