-
Notifications
You must be signed in to change notification settings - Fork 26.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GPT2
] Add SDPA support
#31172
[GPT2
] Add SDPA support
#31172
Changes from all commits
3dc08bd
e425b89
ad6c985
9c729a3
322fb61
bb30edf
3953454
d963ad5
91fe533
4a7b166
f0d7d2a
3c12ee0
95b2440
c665116
4811cb5
8b33cd7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -832,7 +832,8 @@ def test_gpt2_sample_max_time(self): | |
start = datetime.datetime.now() | ||
model.generate(input_ids, do_sample=False, max_time=None, max_length=256) | ||
duration = datetime.datetime.now() - start | ||
self.assertGreater(duration, datetime.timedelta(seconds=1.5 * MAX_TIME)) | ||
self.assertGreater(duration, datetime.timedelta(seconds=MAX_TIME)) | ||
self.assertLess(duration, datetime.timedelta(seconds=1.5 * MAX_TIME)) | ||
Comment on lines
-835
to
+836
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Only failing test without this modification. I'm not sure if this is how it is intended. Maybe There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that this change is totally fine, now SDPA is used so the generation is faster |
||
|
||
@slow | ||
def test_contrastive_search_gpt2(self): | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good!