Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Perplexity Eval for Text Generation Models #1073
Perplexity Eval for Text Generation Models #1073
Changes from 68 commits
48ac0ac
cf7f2b9
832630a
9958c83
e6d2b03
7f9935b
b1cf01b
0a3f48d
add4625
22d2746
7f1651d
b85746d
aadc608
58bc2b0
d538444
e19676b
7908b74
4bc3472
c07f7ed
fb77838
2097463
5eb10a9
9213f29
d9af004
476f25d
fab44e4
d454e2f
1613e25
b61055c
6ee25fc
5d3004b
ace6fa5
388586d
afd0139
30eeda7
5882b56
4bbe33d
afa5746
e2bb78c
2299009
2935b77
b89b156
dc3d61b
a294265
af97f2b
c117788
4ad5f49
9e816bb
f97467f
6be8d87
e2f088d
9fc6c64
a610faf
347d1fb
e11027c
a950910
c1d02dc
711cdfb
e602662
2085c37
2f7bc95
e18fab7
0358d87
06b5246
2cab681
63b116b
7001a6e
79251e6
9efbdb6
da5e93e
a680dac
f83dcab
e659c33
cf74ad7
853f876
e8da07e
58b12c8
eecd232
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need all the logits that are predicted from sequences: {}, {x1}, {x1, x2}, ... {x1, x2, ... x_n}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as discussed offline - running right padded for eval will likely not work for the engine (single token prefill) as internally they will build the KV cache assuming left padded and pop from the left side of cache as its built up. in right padded scenario I believe this will delete the actual non-padded values from cache too early.