use_fast=False when loading OPT's tokenizer?

[jiangwangyi]

System Info

platform==Ubuntu 18.04.01
python==3.10
transformers==4.29.1

Who can help?

@sgugger @stevhliu @mk

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

It is shown in the OPT model documentation (in Tips) that it is required to pass use_fast=False when loading a OPT tokenizer, as OPT tokenizer will add </s> to the beginning of every prompt.

I made a trial:

>>> import transformers
>>> tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/opt-1.3b", use_fast=False)
>>> tokenizer_fast = transformers.AutoTokenizer.from_pretrained("facebook/opt-1.3b", use_fast=True)
>>> text = "I like you."
>>> tokenizer(text)
>>> {'input_ids': [2, 100, 101, 47, 4], 'attention_mask': [1, 1, 1, 1, 1]}
>>> tokenizer_fast(text)
>>> {'input_ids': [2, 100, 101, 47, 4], 'attention_mask': [1, 1, 1, 1, 1]}

</s> is correctly added and no difference is observed.

Expected behavior

Is the tips wrong or in some other cases use_fast=Fast is actually required?

[sgugger]

cc @ArthurZucker

[ArthurZucker]

Hey! Thanks for reporting. Pretty sure the doc is wrong, but use_fast=True use to not be supported for OPT, which could explain this.