OutlinesLogitsProcessor Benchmarks

[lapp0]

What behavior of the library made you think about the improvement?

In #956 I noticed some performance issues relating to a large number of allowed_tokens. We should profile how long it takes for a logits processor to augment logits:

The core performance issue with outlines.generate.regex(model, ".{200}") is the need to convert a large (~150,000 integer) list into a tensor in the logits processor

    allowed_tokens = self.fsm.get_next_instruction(self._fsm_state).tokens
    allowed_tokens = torch.tensor(allowed_tokens, device=logits.device)

To mitigate, we can create a separate issue to ensure the FSM index uses tensors of token IDs, not lists. This will result in self.fsm.get_next_instruction(self._fsm_state).tokens being a tensor of token IDs.

How would you like it to behave?

As part of our ASV benchmark test suite, we should benchmark the performance of OutlinesLogitsProcessor to ensure there aren't performance regressions, and track performance improvements.

[lapp0]

Resolved by #1013