Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐞 Fix issues when validation and test split modes set to none #1703

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

willyfh
Copy link
Contributor

@willyfh willyfh commented Feb 1, 2024

📝 Description

This PR is based on the close PR #1628:

  • when val_split_mode is 'NONE' , skip the validation stage by setting config.trainer.limit_val_batches to 0.0

  • when test_split_mode is 'NONE', skip the test stage by including a conditional check datamodule.test_split_mode == TestSplitMode.NONE. To address a comment in the closed PR, a warning is provided to check the length of the test set that is being skipped.

  • 🛠️ Fixes 🐞 v1 - Issues when validation and test split modes are set to 'none' #1629

✨ Changes

Select what type of change your PR is:

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🔨 Refactor (non-breaking change which refactors the code base)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

  • 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
  • 📚 I have made the necessary updates to the documentation (if applicable).
  • 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

Signed-off-by: Willy Fitra Hendria <willyfitrahendria@gmail.com>
Signed-off-by: Willy Fitra Hendria <willyfitrahendria@gmail.com>
@willyfh willyfh marked this pull request as ready for review February 1, 2024 14:23
Copy link
Collaborator

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few questions regarding this fix. I am not clear on the assumed intention behind passing split mode as None. From what I understand it is only to split the datamodule. But it should not restrict users if they pass test or validation dataset separately. But I don't have a lot of familiarity with data modules. Maybe @djdameln can chime in.

@@ -671,7 +675,12 @@ def train(
self.trainer.validate(model, val_dataloaders, None, verbose=False, datamodule=datamodule)
else:
self.trainer.fit(model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
self.trainer.test(model, test_dataloaders, ckpt_path=ckpt_path, datamodule=datamodule)

if datamodule is not None and datamodule.test_split_mode == TestSplitMode.NONE:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if datamodule is not None and test_dataloaders is not None

Returns:
DictConfig | ListConfig: Updated configurable parameters in DictConfig object.
"""
if config.data.init_args.val_split_mode == ValSplitMode.NONE and config.trainer.limit_val_batches != 0.0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure that if val_split_mode is None, that validation dataset is not provided separately?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐞 v1 - Issues when validation and test split modes are set to 'none'
3 participants