Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable validation when val_percent_check=0 #1251

Merged
merged 5 commits into from
Mar 27, 2020
Merged

Disable validation when val_percent_check=0 #1251

merged 5 commits into from
Mar 27, 2020

Conversation

awaelchli
Copy link
Member

@awaelchli awaelchli commented Mar 26, 2020 •
edited
Loading

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #1248 .

TODO:

  • write unit test
  • update docs to clarify that fast_dev_run will override val_percent_check setting
  • add to changelog

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 馃檭

@codecov
Copy link

codecov bot commented Mar 26, 2020 •
edited
Loading

Codecov Report

Merging #1251 into master will increase coverage by <1%.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master   #1251    +/-   ##
=======================================
+ Coverage      91%     92%   +<1%     
=======================================
  Files          61      61            
  Lines        3153    3153            
=======================================
+ Hits         2879    2886     +7     
+ Misses        274     267     -7

@awaelchli awaelchli changed the title [WIP] Disable validation when val_percent_check=0 Disable validation when val_percent_check=0 Mar 26, 2020
ethanwharris
ethanwharris approved these changes Mar 26, 2020
View reviewed changes
Copy link
Member

@ethanwharris ethanwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! tested and documented :)

@ethanwharris ethanwharris requested a review from a team March 26, 2020 20:33
@Borda Borda added the feature Is an improvement or enhancement label Mar 26, 2020
@Borda Borda added this to the 0.7.2 milestone Mar 26, 2020
Borda
Borda approved these changes Mar 26, 2020
View reviewed changes
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 馃殌

@Borda Borda requested review from jeremyjordan and a team March 26, 2020 20:51
jeremyjordan
jeremyjordan approved these changes Mar 27, 2020
View reviewed changes
Copy link
Contributor

@jeremyjordan jeremyjordan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work on this PR! 馃殌

@mergify mergify bot merged commit 2a4cd47 into Lightning-AI:master Mar 27, 2020
@mergify
Copy link
Contributor

mergify bot commented Mar 27, 2020

Great job! =)

@awaelchli awaelchli deleted the fixdisablevalidation branch March 27, 2020 02:13
@Borda Borda mentioned this pull request Mar 27, 2020
Merged
5 tasks
williamFalcon added a commit that referenced this pull request Mar 28, 2020
@djbyrne @Borda
@williamFalcon @jeremyjordan @bmartinn @TylerYep @5n7-sk @pertschuk
Example: Simple RL example using DQN/Lightning (#1232)
dab3b96 
* Example: Simple RL example using DQN/Lightning

* DQN RL Agent using Lightning

* Uses Iterable Dataset for Replay Buffer

* Buffer is populated by agent as training is carried out, updating the
dataset

* Applied autopep8 fixes

* * Updated line length from 120 to 110

* Update pl_examples/domain_templates/dqn.py

simplify get_device method

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pl_examples/domain_templates/dqn.py

Re-ordered imports

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* CI: split tests-examples (#990)

* CI: split tests-examples

* tests without template

* comment depends

* CircleCI typo

* add doctest

* update test req.

* CI tests

* setup macOS

* longer train

* lover pred acc

* fix model

* rename default model

* lower tests acc

* typo

* imports

* fix test optimizer

* update calls

* fix Win

* lower Drone image

* fix call

* pytorch image

* fix test

* add dev image

* add dev image

* update image

* drone volume

* lint

* update test notes

* rename tests/models >> tests/base

* group models

* conftest

* optim imports

* typos

* fix import

* fix tests

* install AMP

* tests

* fix import

* Clean up

* added module docstring

* renamed variables to be more descriptive

* Added missing docstrings and type annotations

* Added gym to example requirements

* Added note to changelog

* updated example image

* update types

* rename script

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* another rename

* Disable validation when val_percent_check=0 (#1251)

* fix disable validation

* add test

* update changelog

* update docs for val_percent_check

* make "fast training" docs consistent

* calling self.forward() -> self() (#1211)

* self.forward() -> self()

* update changelog

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Fix requirements-extra.txt Trains package to release version (#1229)

* Fix requirement-extra use released Trains package

* Update README.md add Trains and links to the external Visualization section

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Remove unnecessary parameters to super() in documentation and source code (#1240)

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* update deprecation warning (#1258)

* update docs for progress bat values (#1253)

* lower timeouts for inactive issues (#1250)

* update contrib list (#1241)

Co-authored-by: William Falcon <waf2107@columbia.edu>

* Fix outdated docs (#1227)

* Fix typo (#1224)

* drop unused Tox (#1242)

* system info (#1234)

* system info

* update big info

* test script

* update config

* rename script

* import path

* Changed smoothing in tqdm to decrease variability of time remaining between training / eval (#1194)

* Example: Simple RL example using DQN/Lightning

* DQN RL Agent using Lightning

* Uses Iterable Dataset for Replay Buffer

* Buffer is populated by agent as training is carried out, updating the
dataset

* Applied autopep8 fixes

* * Updated line length from 120 to 110

* Update pl_examples/domain_templates/dqn.py

simplify get_device method

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pl_examples/domain_templates/dqn.py

Re-ordered imports

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Clean up

* added module docstring

* renamed variables to be more descriptive

* Added missing docstrings and type annotations

* Added gym to example requirements

* Added note to changelog

* update types

* rename script

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* another rename

Co-authored-by: Donal Byrne <Donal.Byrne@xperi.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Adrian W盲lchli <adrian.waelchli@students.unibe.ch>
Co-authored-by: Jeremy Jordan <13970565+jeremyjordan@users.noreply.github.com>
Co-authored-by: Martin.B <51887611+bmartinn@users.noreply.github.com>
Co-authored-by: Tyler Yep <tyep@stanford.edu>
Co-authored-by: Shunta Komatsu <59395084+skmatz@users.noreply.github.com>
Co-authored-by: Jack Pertschuk <jackpertschuk@gmail.com>
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 3, 2020
Disable validation when val_percent_check=0 (Lightning-AI#1251)
1db545a 
* fix disable validation

* add test

* update changelog

* update docs for val_percent_check

* make "fast training" docs consistent
alexeykarnachev pushed a commit to alexeykarnachev/pytorch-lightning that referenced this pull request Apr 3, 2020
@djbyrne @Borda
@williamFalcon @jeremyjordan @bmartinn @TylerYep @5n7-sk @pertschuk
Example: Simple RL example using DQN/Lightning (Lightning-AI#1232)
498dbda 
* Example: Simple RL example using DQN/Lightning

* DQN RL Agent using Lightning

* Uses Iterable Dataset for Replay Buffer

* Buffer is populated by agent as training is carried out, updating the
dataset

* Applied autopep8 fixes

* * Updated line length from 120 to 110

* Update pl_examples/domain_templates/dqn.py

simplify get_device method

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pl_examples/domain_templates/dqn.py

Re-ordered imports

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* CI: split tests-examples (Lightning-AI#990)

* CI: split tests-examples

* tests without template

* comment depends

* CircleCI typo

* add doctest

* update test req.

* CI tests

* setup macOS

* longer train

* lover pred acc

* fix model

* rename default model

* lower tests acc

* typo

* imports

* fix test optimizer

* update calls

* fix Win

* lower Drone image

* fix call

* pytorch image

* fix test

* add dev image

* add dev image

* update image

* drone volume

* lint

* update test notes

* rename tests/models >> tests/base

* group models

* conftest

* optim imports

* typos

* fix import

* fix tests

* install AMP

* tests

* fix import

* Clean up

* added module docstring

* renamed variables to be more descriptive

* Added missing docstrings and type annotations

* Added gym to example requirements

* Added note to changelog

* updated example image

* update types

* rename script

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* another rename

* Disable validation when val_percent_check=0 (Lightning-AI#1251)

* fix disable validation

* add test

* update changelog

* update docs for val_percent_check

* make "fast training" docs consistent

* calling self.forward() -> self() (Lightning-AI#1211)

* self.forward() -> self()

* update changelog

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Fix requirements-extra.txt Trains package to release version (Lightning-AI#1229)

* Fix requirement-extra use released Trains package

* Update README.md add Trains and links to the external Visualization section

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Remove unnecessary parameters to super() in documentation and source code (Lightning-AI#1240)

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* update deprecation warning (Lightning-AI#1258)

* update docs for progress bat values (Lightning-AI#1253)

* lower timeouts for inactive issues (Lightning-AI#1250)

* update contrib list (Lightning-AI#1241)

Co-authored-by: William Falcon <waf2107@columbia.edu>

* Fix outdated docs (Lightning-AI#1227)

* Fix typo (Lightning-AI#1224)

* drop unused Tox (Lightning-AI#1242)

* system info (Lightning-AI#1234)

* system info

* update big info

* test script

* update config

* rename script

* import path

* Changed smoothing in tqdm to decrease variability of time remaining between training / eval (Lightning-AI#1194)

* Example: Simple RL example using DQN/Lightning

* DQN RL Agent using Lightning

* Uses Iterable Dataset for Replay Buffer

* Buffer is populated by agent as training is carried out, updating the
dataset

* Applied autopep8 fixes

* * Updated line length from 120 to 110

* Update pl_examples/domain_templates/dqn.py

simplify get_device method

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pl_examples/domain_templates/dqn.py

Re-ordered imports

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Clean up

* added module docstring

* renamed variables to be more descriptive

* Added missing docstrings and type annotations

* Added gym to example requirements

* Added note to changelog

* update types

* rename script

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* another rename

Co-authored-by: Donal Byrne <Donal.Byrne@xperi.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Adrian W盲lchli <adrian.waelchli@students.unibe.ch>
Co-authored-by: Jeremy Jordan <13970565+jeremyjordan@users.noreply.github.com>
Co-authored-by: Martin.B <51887611+bmartinn@users.noreply.github.com>
Co-authored-by: Tyler Yep <tyep@stanford.edu>
Co-authored-by: Shunta Komatsu <59395084+skmatz@users.noreply.github.com>
Co-authored-by: Jack Pertschuk <jackpertschuk@gmail.com>
@Borda Borda modified the milestones: v0.7., v0.7.x Apr 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda approved these changes

@ethanwharris ethanwharris ethanwharris approved these changes

@jeremyjordan jeremyjordan jeremyjordan approved these changes

Assignees
No one assigned
Labels
feature Is an improvement or enhancement
Projects
None yet
Milestone
pl:0.7.x
Development

Successfully merging this pull request may close these issues.

Disabling validation with val_percent_check=0.0 not working
4 participants
@awaelchli @Borda @ethanwharris @jeremyjordan