Captum v0.3.0 Release
The third release, v0.3.0, of Captum adds new attribution algorithms including Lime and KernelSHAP, metrics for assessing attribution results including infidelity and sensitivity, and improvements to existing attribution methods.
Metrics (Sensitivity and Infidelity)
Captum 0.3.0 adds metrics to estimate the trustworthiness of model explanations. Currently available metrics include Sensitivity-Max and Infidelity.
Infidelity measures the mean squared error between model explanations in the magnitudes of input perturbations and predictor function's changes to those input perturbations. Sensitivity measures the degree of explanation changes to subtle input perturbations using Monte Carlo sampling-based approximation. These metrics are available in captum.metrics and documentation can be found here.
Lime and KernelSHAP
In Captum 0.3.0, we also add surrogate-model interpretability methods including Lime and KernelSHAP. Lime is an interpretability method that trains an interpretable surrogate model by sampling points around a specified input example and using model evaluations at these points to train a simpler interpretable 'surrogate' model, such as a linear model.
We offer two implementation variants of this method, LimeBase and Lime. LimeBase provides a generic framework to train a surrogate interpretable model, while Lime provides a more specific implementation than LimeBase in order to expose a consistent API with other perturbation-based algorithms. KernelSHAP is a method that uses the Lime framework to compute Shapley Values.
New Tutorials
We have added new tutorials to demonstrate Captum with CV tasks such as segmentation as well as in distributed environments. These tutorials are:
- Using Captum with torch.distributed
- Interpreting a semantic segmentation model
Attribution Improvements
Captum 0.3.0 has added improvements to existing attribution methods including:
- LayerActivation and LayerGradientXActivation now support computing attributions for multiple layers simultaneously. (PR #456).
- Neuron attribution methods now support providing a callable to select or aggregate multiple neurons for attribution, as well as slices to select a range of neurons. (PR #490, #495). The parameter name neuron_index has been deprecated and is replaced by neuron_selector, which supports either indices or a callable.
- Feature ablation and feature permutation now allow attribution with respect to multiple batch-aggregate scalars (e.g.loss) simultaneously (PR #425).
- Most attribution methods now support a multiply_by_inputs argument. For attribution methods which include a multiplier of inputs or inputs - baselines, this argument selects whether these multipliers should be incorporated or left out to obtain marginal attributions. (PR #432)
- Methods accepting internal batch size were updated to generate batches lazily rather than splitting an expanded input tensor, eliminating memory constraints when experimenting with a large number of steps. (PR #333).
Captum Insights
Bug Fixes
- Providing target as a list with inputs on CUDA devices now works appropriately. (Issue #316, PR #317)
- DeepLift issues with DataParallel models, particularly when providing additional forward args or multiple targets, have been fixed. (PR #335)
- Hooks added within an attribution method were previously not being removed if the attribution method encountered an exception before removing the hook. All hooks are now removed even if an exception is raised during attribution. (PR #340)
- LayerDeepLift was fixed to avoid applying hooks on the target layer when attributing layer output, which caused incorrect results or errors with some non-linearities (Issue #382, PR #390, #415).
- Non-leaf tensor gradient warning when using NoiseTunnel with Saliency has been fixed. (Issue #421, PR #426)
- Text visualization helpers now have option to display legend. (Issue #401, PR #403)
- Image visualization helpers fixed to normalize even if outlier threshold is close to 0 (Issue #393, PR #458).