Error in outlines.generate.choice: create_states_mapping throws ValueError: not enough values to unpack (expected 3, got 2) #585
Comments
|
Could you please include your version info in the version section? There was a change recently which may have fixed this There's a good chance upgrading to latest (unreleased) |
|
I was on 0.0.24 But now "outlines.generate.format" throws the same kind of error! This is very weird, as I see that "create_states_mapping" should return only two objects: states_to_token_maps and empty_token_ids. But when I print what is returned, I see its 3 objects: Running the outlines.generate.choice method returns 2 objects correctly, the dictionary, and the set. |
|
Maybe found a quick fix: commenting out the @cache seems to fix this! i.e. Not sure how this affects performance though, but: With cache commented out: With cache: |
|
#566 should have fixed this. It invalidates the cache if the version is upgraded. Can you confirm which version you are running via |
|
I'm on: 0.0.25.dev15+g0cd9608 |
|
I found the source of the issue. Outlines cache is cleared if there's a version upgrade, however installing from We need to ensure the version in @dnhkng as a temporary fix I recommend running |
|
Best route forward IMO:
|
|
Works in my environment. @dnhkng could you please confirm your reproduction code no longer fails in your conda environment if you run |
|
Looks ok now! I'm still getting strings instead of floats, but I've raised a separate issue for that. |
Describe the issue as clearly as possible:
When I try the examples on the github front page, some do not work from a fresh conda environment.
Steps/code to reproduce the bug:
Expected result:
Error message:
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[2], line 10 1 model = outlines.models.transformers("TinyLlama/TinyLlama-1.1B-Chat-v1.0", device="cuda") 4 prompt = """You are a sentiment-labelling assistant. 5 Is the following review positive or negative? 6 7 Review: This restaurant is just awesome! 8 """ ---> 10 generator = outlines.generate.choice(model, ["Positive", "Negative"]) 11 answer = generator(prompt) File ~/miniforge3/envs/frankenmerge/lib/python3.10/site-packages/outlines/generate/api.py:412, in choice(model, choices, max_tokens, sampler) 405 def choice( 406 model, 407 choices: List[str], 408 max_tokens: Optional[int] = None, 409 sampler: Sampler = multinomial, 410 ): 411 regex_str = r"(" + r"|".join(choices) + r")" --> 412 return regex(model, regex_str, max_tokens, sampler) File ~/miniforge3/envs/frankenmerge/lib/python3.10/site-packages/outlines/generate/api.py:373, in regex(model, regex_str, max_tokens, sampler) 367 def regex( 368 model, 369 regex_str: str, 370 max_tokens: Optional[int] = None, 371 sampler: Sampler = multinomial, 372 ): --> 373 fsm = RegexFSM(regex_str, model.tokenizer) 375 device = model.device 376 generator = SequenceGenerator(fsm, model, sampler, device, max_tokens=max_tokens) File ~/miniforge3/envs/frankenmerge/lib/python3.10/site-packages/outlines/fsm/fsm.py:136, in RegexFSM.__init__(self, regex_string, tokenizer) 131 final_states = regex_fsm.finals | { 132 -1 133 } # Include the EOS token in final states 134 return states_to_token_maps, empty_token_ids, final_states --> 136 ( 137 self.states_to_token_maps, 138 self.empty_token_ids, 139 self.final_states, 140 ) = create_states_mapping( 141 regex_string, tuple(sorted(tokenizer.vocabulary.items())) 142 ) 143 self.num_tokens_generated = 0 144 self.vocabulary = tokenizer.vocabulary.values() ValueError: not enough values to unpack (expected 3, got 2)Outlines/Python version information:
Version information
Context for the issue:
This is a very weird bug! If I run the "outlines.generate.format" code, very occasionally, I also get the "outlines.generate.choice" method to run too! But 99% of the time, I get this error.
I did some digging, and added some debug code:
When I run the working code, I see:
But the buggy code produces:
So, the function "create_states_mapping" is not returning the frozenset, so the tuple only has 2 on the 3 items to unpack!
The text was updated successfully, but these errors were encountered: