-
Notifications
You must be signed in to change notification settings - Fork 168
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Text Generation] Causal Mask Feature Branch (#1126)
* Update helpers.py * correct implementation of the mapping from inputs to causal mask * [Text Generation] Causal Mask Support (#1127) * initial commit * clean up the PR * working implementation * Ben's review comments * [Text Generation] Multitoken prefill enablement (#1130) * initial commit * clean up the PR * working implementation * initial implementation, hacky lets clean it up * ready for review * few tiny quality improvements * simplify the logic for computing num of unmasked bits for creating attention_mask for the multitoken prefill * replace boolean causal mask for int64 causal mask * fix breaking tests
- Loading branch information
1 parent
68eb5ca
commit c988fe7
Showing
5 changed files
with
392 additions
and
44 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.