This is the repository for pytorch Implementation of "Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization".
- Plug-and-play usage
- Support multiple and arbitrary layers by their full name
- Save all useful intermediate results
- Support various modes by specifying
retain_graph
,create_graph
andvisualize
.
- Python >= 3.6
- imgaug
- numpy
- opencv-python
- pytorch
First import from gradcam.py
and create an instance:
from gradcam import GradCAM
visualizer = GradCAM(model, layer_names, detach=True)
Each string in layer_names
is the full name of a target layer in your model. Here is some examples for ResNet:
layer4
layer4.1
layer4.1.bn1
After forward propagation of model, the graph is established, and you can calculate Grad-CAM heatmap by:
visualizer.cal_grad(y, target_class)
visualizer.cal_cam(visualize=True)
To show Grad-CAM heatmap on original input images:
visualizer.show_cam_on_image(input) # input: (B, H, W, 3)
Now useful information are stored in visualizer.vis_info
now. For each target layer name layer_name
:
visualizer.vis_info[layer_name]['output']
: output feature mapsvisualizer.vis_info[layer_name]['grad']
: gradientsvisualizer.vis_info[layer_name]['cam']
: class activation mapsvisualizer.vis_info[layer_name]['mask_img']
: images transformed from class activation mapsvisualizer.vis_info[layer_name]['heatmap_img']
: heatmap images incv2.COLORMAP_JET
modevisualizer.vis_info[layer_name]['vis_img']
: visualization images
Finally, do not forget to reset state machine before next visualization:
visualizer.reset_info()
Sometimes we want to retain the graph after Grad-CAM calculation and do back propagation afterwards. To achieve this, specify retain_graph=True
:
visualizer.cal_grad(y, target_class, retain_graph=True)
If your purpose of calculating CAM is not for visualization, you can specify visualize=False
:
visualizer.cal_cam(visualize=False)
Now visualizer.vis_info[layer_name]['cam']
is an instance of torch.Tensor
instead of numpy.ndarray
.
Sometimes we want to obtain differentiable class activation maps. To achieve this, specify detach=False
when constructing the worker:
visualizer = GradCAM(model, layer_names, detach=False)
and specify visualize=False
,retain_graph=True
and create_graph=True
:
visualizer.cal_grad(y, target_class, retain_graph=True, create_graph=True)
visualizer.cal_cam(visualize=False)