-
|
Hello i have the following model: model = models.resnet50(pretrained=True)
for param in model.parameters():
param.requires_grad = False
fc_inputs = model.fc.in_features
model.fc = nn.Sequential(nn.Linear(fc_inputs, 512),
nn.ReLU(),
nn.Dropout(p=0.2),
nn.Linear(512, 256),
nn.ReLU(),
nn.Dropout(p=0.2),
nn.Linear(256, 2),
nn.LogSoftmax(dim=1))
model.to(device)
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.fc.parameters(), amsgrad=True)i trained it the following way for cat and dog classification epochs = 1
running_loss = 0
test_loss = 0
accuracy = 0
best_acc = 0
for epoch in range(epochs):
start_time = time.time()
for images, labels in trainloader:
images, labels = images.to(device), labels.to(device)
optimizer.zero_grad()
logps = model(images)
loss = criterion(logps, labels)
loss.backward()
optimizer.step()
running_loss += loss.item()
model.eval()
with torch.no_grad():
for images, labels in testloader:
images, labels = images.to(device), labels.to(device)
logps = model(images)
loss = criterion(logps, labels)
test_loss += loss.item()
ps = torch.exp(logps)
top_p, top_class = ps.topk(1, dim=1)
equals = top_class == labels.view(*top_class.shape)
accuracy += torch.mean(equals.type(torch.FloatTensor)).item()
print(f'Epoch {epoch+1}/{epochs}.. '
f'Train loss: {running_loss/len(trainloader):.3f}.. '
f'Test loss: {test_loss/len(testloader):.3f}.. '
f'Test accuracy: {accuracy/len(testloader):.3f}')
elapsed_time = time.time() - start_time
print(time.strftime("%H:%M:%S", time.gmtime(elapsed_time)))
running_loss = 0
test_loss = 0
accuracy = 0
model.train()but now if i want to draw the heatmap to overlay my imagelike this: from torchvision.io.image import read_image
from torchvision.transforms.functional import normalize, resize, to_pil_image
from torchvision.models import resnet50
from torchcam.methods import SmoothGradCAMpp
import matplotlib.pyplot as plt
from torchcam.utils import overlay_mask
img = read_image("./HandTest/m1.jpg")
input_tensor = normalize(resize(img, (224, 224)) / 255., [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
with SmoothGradCAMpp(model) as cam_extractor:
out = model(input_tensor.unsqueeze(0).to(device))
activation_map = cam_extractor(0, out)
result = overlay_mask(to_pil_image(img), to_pil_image(activation_map[0].squeeze(0), mode='F'), alpha=0.5)
plt.imshow(result); plt.axis('off'); plt.tight_layout(); plt.show()i always get the following error: what am I doing wrong? I'm really grateful for any help |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 9 replies
-
|
Hey @kvnae94 👋 It looks like you're trying to use a gradient-based CAM method on a model where you've disabled gradients. So here is a quick fix: from torchvision.io.image import read_image
from torchvision.transforms.functional import normalize, resize, to_pil_image
from torchvision.models import resnet50
from torchcam.methods import SmoothGradCAMpp
import matplotlib.pyplot as plt
from torchcam.utils import overlay_mask
img = read_image("./HandTest/m1.jpg")
input_tensor = normalize(resize(img, (224, 224)) / 255., [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
for param in model.parameters():
param.requires_grad_(True)
with SmoothGradCAMpp(model) as cam_extractor:
out = model(input_tensor.unsqueeze(0).to(device))
activation_map = cam_extractor(0, out)
result = overlay_mask(to_pil_image(img), to_pil_image(activation_map[0].squeeze(0), mode='F'), alpha=0.5)
plt.imshow(result); plt.axis('off'); plt.tight_layout(); plt.show()Let me know if that solves your problem or if you have any other questions! |
Beta Was this translation helpful? Give feedback.
-
|
I tried to enable the gradient for the model by adding this line between the target_layer variable and the cam dict (line 18 and 19): for param in model.parameters():
param.requires_grad_(True)the error isn't solved so will I have to instantiate the ScoreCAM seperately? I'm trying to create an API to get the CAM images upon request. Is there any better way to do this else I might have to create another instance for the model to register the hooks for activation based methods |
Beta Was this translation helpful? Give feedback.
-
|
Could you provide a strictly minimal snippet with the error please? My understanding is that error is thrown on the line where you defined |
Beta Was this translation helpful? Give feedback.
-
|
Hey this is the minimal snippet and error you required, the error happens whenever activation and gradient methods are hooked together onto the model to be used I believe, the gradient based-methods worked without error import torch
from torchvision.models import resnet18
from torchcam.methods import ScoreCAM, GradCAM
model = resnet18(pretrained=True).eval()
for param in model.parameters():
param.requires_grad = True
cam_score = ScoreCAM(model, 'layer4')
cam_grad = GradCAM(model, 'layer4')
input_tensor = torch.rand((1, 3, 3, 255))
out = model(input_tensor)
cam_score(class_idx=100)This is the error stack: |
Beta Was this translation helpful? Give feedback.
-
|
Alright, now it's clearer thanks! import torch
from torchvision.models import resnet18
from torchcam.methods import ScoreCAM, GradCAM
model = resnet18(pretrained=True).eval()
for param in model.parameters():
param.requires_grad = True
cam_score = ScoreCAM(model, 'layer4')
# cam_grad = GradCAM(model, 'layer4')
input_tensor = torch.rand((1, 3, 3, 255))
out = model(input_tensor)
cam_score(class_idx=100)this snippet works, no need to instantiate GradCAM. Now the reason why this throws an error is:
My advice is:
Let me know if that solves your problem! |
Beta Was this translation helpful? Give feedback.
-
|
Ok I do understand the problem. What I'm trying to build is I send an image to the server and it returns me back the grad image of whichever method I requested. I wrote the code so that the methods are initialized according to the list of methods requested from the API. Hence I cannot remove those methods. It is bound to happen that sometime the gradient based methods and the activation based methods are together in the list requested. Can you maybe tell more about tweeking with |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the context!
cam_method._hooks_enabled = Falseand when you receive a request you only enable the hooks of the method you want Let me know if this isn't clear! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the help @frgfm it totally works 😇 |
Beta Was this translation helpful? Give feedback.
Alright, now it's clearer thanks!
So to make this minimal:
this snippet works, no need to instantiate GradCAM. Now the reason why this throws an error is:
@torch.no_graddecorator. And for that method at some point, we forward the input in the model (did this to avoid ex…