Warn user when IterableDataset has __len__ defined #2437
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
awaelchli
changed the title
awaelchli
added
feature
awaelchli
force-pushed
the
bugfix/should_check_val
branch
from
5dabf85
to
45a05c4
Compare
awaelchli
force-pushed
the
bugfix/should_check_val
branch
from
45a05c4
to
d00b2d5
Compare
Codecov Report
@@ Coverage Diff @@
## master #2437 +/- ##
======================================
Coverage 88% 88%
======================================
Files 69 69
Lines 5526 5527 +1
======================================
+ Hits 4887 4888 +1
Misses 639 639 |
|
Hello @awaelchli! Thanks for updating this PR. There are currently no PEP 8 issues detected in this Pull Request. Cheers! 馃嵒 Comment last updated at 2020-07-01 11:33:06 UTC |
justusschock
approved these changes
Jul 1, 2020
Borda
approved these changes
Jul 1, 2020
| @@ -41,19 +43,33 @@ | |||
| HOROVOD_AVAILABLE = True | |||
|
|
|||
|
|
|||
| def _has_iterable_dataset(dataloader: DataLoader): | |||
| return ITERABLE_DATASET_EXISTS and hasattr(dataloader, 'dataset') \ | |||
There was a problem hiding this comment.
if you encapsulate the if statement in () you do not need to use \
Comment on lines
61
to
+64
| except TypeError: | ||
| return False | ||
| has_len = False | ||
| except NotImplementedError: # e.g. raised by torchtext if a batch_size_fn is used | ||
| return False | ||
| has_len = False |
There was a problem hiding this comment.
if the resulting action is the same, lets write it in one
except (TypeError, NotImplementedError): # e.g. raised by torchtext if a batch_size_fn is used
has_len = False
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #2429
torch < 1.4: An
IterableDatasetwith__len__defined raises a TypeErrortorch >= 1.4: It does not raise an error, but we still want to display a warning because it is most likely unintentional and could lead to side effects.
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 馃檭