DataParallel DeepLift Fixes #335
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Updated to use 1 hook instead of 2. |
Awesome! Thank you! I'll take a look! |
NarineK
approved these changes
Apr 3, 2020
There was a problem hiding this comment.
@vivekmig has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
NarineK
pushed a commit
to NarineK/captum-1
that referenced
this pull request
Nov 19, 2020
Summary: This PR adds only the bug fixes identified from refactoring and adding dynamic tests. 1. Additional forward arguments were not working appropriately with DeepLift on a DataParallel model. This is because the device split of expanded additional forward args didn't necessarily match that of inputs. The behavior has been changed to expand the additional args in the hook function (after device split), which ensures the additional args and inputs remain matched. 2. Different targets per example was not working appropriately with DeepLift on a DataParallel model. This is because the model output concatenated the outputs of the devices in DataParallel, which mixed input / baseline outputs, inhibiting appropriate matching between input example and target. Additional forward hooks have been added to appropriately return the output with all inputs followed by all baselines. 3. GradCAM is primarily intended for layers with >= 3 dimensions, since it computes average gradient for each example / channel. For layers with 2 dimensions, the mean gradient over all dimensions was being taken. This has been updated to use the layer gradients directly in this case, which better aligns with the behavior for >= 3 dimensions. 4. DeepLiftShap (and Neuron / Layer variants) were incorrectly repeating additional forward args, this has been fixed to use repeat interleave instead. Pull Request resolved: pytorch#335 Reviewed By: edward-io Differential Revision: D20844511 Pulled By: vivekmig fbshipit-source-id: c895b348c3d5c56355c39d429947f2f36dda37a7
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds only the bug fixes identified from refactoring and adding dynamic tests.