fix(llama): fix LlamaTokenzier

[rockmagma02]

What does this PR do?

Fixes #22742

• Bug in LlamaTokenizer when return_token_type_ids=True #22742

This PR removes an extra sep token in the sequence length calculation.

Ref:

transformers/src/transformers/models/llama/tokenization_llama.py

Lines 178 to 187 in 7df1343

	def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
	bos_token_id = [self.bos_token_id] if self.add_bos_token else []
	eos_token_id = [self.eos_token_id] if self.add_eos_token else []
	output = bos_token_id + token_ids_0 + eos_token_id
	if token_ids_1 is not None:
	output = output + bos_token_id + token_ids_1 + eos_token_id
	return output

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ArthurZucker]

Nice! Good job on the fix 🚀 . Note that the seq and cls token were not even set, was part of the problem.

[amyeroberts]

Thanks for the quick fix! 🚀

It seems there is an issue with your CircleCI permissions, as the tests won't run.
Could you try refreshing your permissions as shown here?

[rockmagma02]

Thanks for your quick review, I have re-run the CI tests. 🤗 @amyeroberts @ArthurZucker