Decoder hidden states not found

[herbert-zhou]

I installed ecco with python 3.8 and 3.9 (I tried both) on Jan 31, 2024. However, the examples from the notebooks are not working with the common error of "decoder hidden states not found". Other functionalities worked, it is just with all the functions that requires the decoder hidden state (e.g. token rankings).

[herbert-zhou]

And I tried both installing ecco with pip and conda, but I don't think it would make any difference.

The problem is simple: I ran the following lines:

import ecco
lm = ecco.from_pretrained('distilgpt2')
text= "The keys to the cabinet"
output = lm.generate(text, generate=1, do_sample=False)

And I checked the states:
print(output.decoder_hidden_states), which prints None

And the error came in when I tried to call any function that involves the decoder hidden state:
output.rankings()

[TenshirenDNTU]

Has anyone been able to fix this error yet?

[sachaRfd]

The issue is that you are not telling the model to track the hidden layers. All you have to do is add "output_hidden_states=True" when you call the generate method. Here is the example:

output = lm.generate(text, generate=1, do_sample=False, output_hidden_states=True)

Hope that helps :)