Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor PreProcessor and fix Visualizer denormalization issue. #570

Merged
merged 28 commits into from
Sep 23, 2022

Conversation

samet-akcay
Copy link
Contributor

Description

Changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which refactors the code base)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My code follows the pre-commit style and check guidelines of this project.
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing tests pass locally with my changes

@github-actions github-actions bot added Data Post-Processing The components that are related to post-processing Pre-Processing labels Sep 21, 2022
@github-actions github-actions bot added the Tests label Sep 21, 2022
Copy link
Collaborator

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks I don't have anything to add here

logger = logging.getLogger(__name__)


def get_transforms(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also move this function to data/utils?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is still one of the most important components of pre-processing sub-package. If we move it to data/utils, then pre_processing would become quite a weak package that we could consider removing in the future

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case we could ask ourselves if we really need the pre-processing module

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that's the question, I think

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm merging this one, we could discuss this later on

Copy link
Contributor

@djdameln djdameln left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the meeting, we could probably deprecate the PreProcessor class now that it's reduced to a wrapper around the get_transforms function. It would require some changes to the datamodule, so let's address this once the datamodule refactor has been merged.

@samet-akcay samet-akcay merged commit 5b3fc2b into main Sep 23, 2022
@samet-akcay samet-akcay deleted the fix/sa/custom-data-normalization branch September 23, 2022 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Data Post-Processing The components that are related to post-processing Pre-Processing Tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow custom Data Normalization and consider it in Post-processing
3 participants