Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SkipEval exception #1200

Merged
merged 13 commits into from
Jun 13, 2024
Merged

SkipEval exception #1200

merged 13 commits into from
Jun 13, 2024

Conversation

sfc-gh-pmardziel
Copy link
Contributor

@sfc-gh-pmardziel sfc-gh-pmardziel commented Jun 12, 2024 •
edited
Loading

Items to add to release announcement:

  • SkipEval: Added exception to be raised by a feedback function to indicate that it should be skipped and not aggregated. This is for use cases were the feedback function is evaluated multiple times on different data and the determination of whether an evaluation on one piece of data should be skipped is done in the feedback implementation itself. Note the related existing feature if_exists to skip the entire feedback function for a whole record.

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jun 12, 2024
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Jun 12, 2024
sfc-gh-chu
sfc-gh-chu approved these changes Jun 13, 2024
View reviewed changes
Copy link
Contributor

@sfc-gh-chu sfc-gh-chu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm w/ some small feedback

trulens_eval/trulens_eval/feedback/feedback.py Show resolved Hide resolved
trulens_eval/trulens_eval/feedback/feedback.py Outdated Show resolved Hide resolved
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 13, 2024
@sfc-gh-pmardziel sfc-gh-pmardziel merged commit 4a58aec into main Jun 13, 2024
9 checks passed
@sfc-gh-jreini sfc-gh-jreini mentioned this pull request Jun 21, 2024
Merged
sfc-gh-dhuang pushed a commit that referenced this pull request Jun 28, 2024
@sfc-gh-pmardziel @sfc-gh-chu
SkipEval exception (#1200)
1a8daf6 
* skipeval exception

* fix skip logic

* nits

* adding test cases for SkipEval

* finalize tests

* add new functionality to docs

* do typo

* unrelated typo

* add black to docs requirements

* fix bad links in docs

* minor

* add percent skipped warning

---------

Co-authored-by: corey hu <corey.hu@snowflake.com>
sfc-gh-dhuang pushed a commit that referenced this pull request Jul 1, 2024
@sfc-gh-pmardziel @sfc-gh-chu @sfc-gh-dhuang
SkipEval exception (#1200)
3abbd6b 
* skipeval exception

* fix skip logic

* nits

* adding test cases for SkipEval

* finalize tests

* add new functionality to docs

* do typo

* unrelated typo

* add black to docs requirements

* fix bad links in docs

* minor

* add percent skipped warning

---------

Co-authored-by: corey hu <corey.hu@snowflake.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sfc-gh-chu sfc-gh-chu sfc-gh-chu approved these changes

Assignees
No one assigned
Labels
lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sfc-gh-pmardziel @sfc-gh-chu