New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is not l1 loss the smooth l1 mentioned in the paper? #17

Open

YoungjaeDev opened this issue May 28, 2022 · 1 comment

YoungjaeDev commented May 28, 2022

pytorch-slimming/main.py

Lines 97 to 101 in 7d13c09

    
           # additional subgradient descent on the sparsity-induced penalty term 
        
           def updateBN(): 
        
               for m in model.modules(): 
        
                   if isinstance(m, nn.BatchNorm2d): 
        
                       m.weight.grad.data.add_(args.s*torch.sign(m.weight.data))  # L1

is it right?

Author

YoungjaeDev commented May 29, 2022

Looks like he's taking the subgradient approach

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment