Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The behavior of EarlyStopping is not the same between 0.7.5 with 0.7.3 #1699

Closed
nickyi1990 opened this issue May 2, 2020 · 3 comments · Fixed by #2391
Closed

The behavior of EarlyStopping is not the same between 0.7.5 with 0.7.3 #1699

nickyi1990 opened this issue May 2, 2020 · 3 comments · Fixed by #2391
Assignees
Labels
bug Something isn't working help wanted Open to be worked on

Comments

@nickyi1990
Copy link

🐛 Bug

The behavior of EarlyStopping is not the same between 0.7.5 with 0.7.3, If i set patience=1, 0.7.3 will print two epoch when the second is worse than the first, but 0.7.5 will just print the first which is not consistent with the description of patience parameter

To Reproduce

Code sample

early_stop_callback = EarlyStopping(monitor='valid_roc_auc', patience=0, mode='max', verbose=True)
# Instance Model, Trainer and train model
model = PlantModel(hparams)
trainer = pl.Trainer(gpus=[hparams.gpus],
                     max_epochs=hparams.max_epochs,
                     early_stop_callback=early_stop_callback,
                     progress_bar_refresh_rate=0,
                     num_sanity_val_steps=0,
                     profiler=True,
                     gradient_clip_val=hparams.gradient_clip_val)
trainer.fit(model)
print(trainer.early_stop_callback.wait)

Expected behavior

Environment

  • CUDA:
    - GPU:
    - GeForce RTX 2080 Ti
    - GeForce RTX 2080 Ti
    - GeForce RTX 2080 Ti
    - GeForce RTX 2080 Ti
    - available: True
    - version: 10.1
  • Packages:
    - numpy: 1.18.1
    - pyTorch_debug: False
    - pyTorch_version: 1.4.0
    - pytorch-lightning: 0.7.5
    - tensorboard: 2.2.1
    - tqdm: 4.42.1
  • System:
    - OS: Linux
    - architecture:
    - 64bit
    -
    - processor: x86_64
    - python: 3.7.6
    - version: Fixed typo in single_cpu_template #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020

Additional context

@nickyi1990 nickyi1990 added bug Something isn't working help wanted Open to be worked on labels May 2, 2020
@williamFalcon
Copy link
Contributor

@jeremyjordan

@jeremyjordan
Copy link
Contributor

yes i believe this is the same issue reported in this comment and will be addressed in #1504

@nickyi1990
Copy link
Author

reported

Yes, exactly the same problem, It seems the only way to use 0.7.5 is to change the source code my self😢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants