Restore the ability to stop when a specified string has been generated

[RobinPicard]

This PR aims at solving #417

The proposed change does 2 main things:

• break the generation loop in SequenceGenerator.__call__ if all sequences contain at least one of the stop_sequences provided by the user
• format the generated sequences to truncate them once the first stop_sequence is found in them

A downside of adding this logic at the level of the SequenceGenerator is that we generate text that is later discarded. However, incorporating this logic in the fsm would require much bigger change as we need to decode the token_ids to check whether the stop_sequences are found. Maybe this could be a first solution before some future refactoring.

As suggested in #433, I think we could treat the max_tokens parameter in a similar way as what's proposed here for the stop_sequences

I've only modified the tests required to pass the tests and the linting, but will modify them further along with documentation in another commit if you think this PR is going in the right direction.

[rlouf]

Go ahead, this is the right direction! We'll be able to simplify greatly StopAtFSM and rename it to BaseFSM or something like that.

[rlouf]

The old interface used stop_at, we should keep that name at least here

[RobinPicard]

I put stop because it's what continuation seemed to expect on line 49 and what you removed in your big PR. Was it stop_at in an earlier version?

[rlouf]

Yes it was. stop_at sounds like a better name anyway, wdyt?

[RobinPicard]

Absolutely!

[RobinPicard]

I have a few questions @rlouf:

• For passing the max_tokens argument when calling the generator instead of initializing it, should we keep both options for the moment not to break the existing interface or should we already remove the parameter in the __init__ method?
• Should we also pass end when calling the generator instead of initializing it?
• Should those 2 parameters be available for all the generate functions even for those for which it makes less sense (for instance json)?

[rlouf]

• For passing the max_tokens argument when calling the generator instead of initializing it, should we keep both options for the moment not to break the existing interface or should we already remove the parameter in the __init__ method?

We can do both with a deprecation warning if passed in __init__ (same version as for /txt/generate/api.py)

• Should we also pass end when calling the generator instead of initializing it?

Yes. Should call it end_at to be consistent with stop_at.

• Should those 2 parameters be available for all the generate functions even for those for which it makes less sense (for instance json)?

I don't think it's necessary for JSON, Regex, format. But might be useful for grammar-guided generation.

[rlouf]

Is it ready for review? I can take a look at these merge conflicts later.

[RobinPicard]

Is it ready for review? I can take a look at these merge conflicts later.

Yes!

[rlouf]

I made a few cosmetic changes (naming and docstrings) + rebased on main. Otherwise, looks great! You can have a last look and if you're good with the changes I made we can merge.

[RobinPicard]

I made a few cosmetic changes (naming and docstrings) + rebased on main. Otherwise, looks great! You can have a last look and if you're good with the changes I made we can merge.

Looks great! Thanks for adding the docstrings