Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to extract cam during training without creating issues to the bakpropagation #488

Open
PietroManganelliConforti opened this issue Mar 8, 2024 · 2 comments

Comments

@PietroManganelliConforti
Copy link

PietroManganelliConforti commented Mar 8, 2024

Hi,

I'm implementing the following custom function:

def return_cam_from_model(model, target_layer, batch, targets , cam_name = "gradcam"):

model.eval() # da rimettere in .train() se nel training loop

cam = None

if cam_name == "gradcam":
    cam = GradCAM(model=model, target_layers=target_layer, use_cuda=False)

elif cam_name == "scorecam":
    cam = ScoreCAM(model=model, target_layers=target_layer, use_cuda=True)

elif cam_name == "fullcam":
    cam = FullGrad(model=model, target_layers=target_layer, use_cuda=True)

elif cam_name == 'gradcamplspls':
    cam = GradCAMPlusPlus(model=model, target_layers=target_layer, use_cuda=True)

elif cam_name == 'xgradcam':
    cam = XGradCAM(model=model, target_layers=target_layer, use_cuda=True)
    
elif cam_name == 'eigencam':
    cam = EigenCAM(model=model, target_layers=target_layer, use_cuda=True)
 
elif cam_name == 'eigengradcam':
    cam = EigenGradCAM(model=model, target_layers=target_layer, use_cuda=True)
    
elif cam_name == 'layercam':
    cam = LayerCAM(model=model, target_layers=target_layer, use_cuda=True)

elif cam_name == 'fullgrad':
    cam = FullGrad(model=model, target_layers=target_layer, use_cuda=True)
 
elif cam_name == 'hirescam':
    cam = HiResCAM(model=model, target_layers=target_layer, use_cuda=True)
    
elif cam_name == 'gradcamelementwise':
    cam = GradCAMElementWise(model=model, target_layers=target_layer, use_cuda=True)

elif cam_name == 'NOXAI':
    return 0

else:
    raise Exception("Cam name", cam_name, "not recognized")

targets = [ClassifierOutputTarget(i) for i in targets]


ret_cam = cam(input_tensor=batch, targets=targets)

model.train()

return ret_cam

we have seen that model.train() before the CAM execution produces a wrong cam explanation, so we inverted their order.

Therefore, because the cam execution is affected by the model training, do you know if inside the cam functions there is something that may affect the model training/backward? And how to deal with it?

@PietroManganelliConforti
Copy link
Author

I'm trying to get tensors with gradient after the library execution

@m-aryayi
Copy link

I'm trying to get tensors with gradient after the library execution

Hi,
What exactly do you mean by that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants