-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strategies to avoid overfitting #5
Comments
Hi @1243France , if you meet overfitting problem, please try:
|
@1243France, I was able to reproduce the model in pytorch. Please see the repo: https://github.com/nekitmm/FunnelAct_Pytorch Training Acc@1 was about 81.3%, so looks like you overfitted your model. |
Hi @nekitmm , could you offer your lr_scheduler settings? Thanks for your help. |
Sorry, I did not save the schedule, I was changing lr manually each time accuracy stopped improving. I started from 1e-3 and dropped it to 1e-6 at the very end... |
@1243France , I reviewed your code briefly, please note as we presented in both the paper and code, we do not apply frelu to the last stage to avoid overfitting. If you do not use this strategy, please follow the strategy I suggested above. |
Seems to be tedious and time-consuming.. Just use the simple linear decay schedule :) |
I reproduced your FReLU in mmclassification, but there is an serious overfitting.
For resnet50_batch256 wit FReLU,which should have 77.6% valid top-1 acc in your experiment. But in my experiment the valid top-1 acc is only 76.07%,and the train top-1 acc is 86.10%.
I try to solve it by ColorJitter,dropout,weight_decay。but they do not work.
Do you have a version of pytorch implementation?
The text was updated successfully, but these errors were encountered: