Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEAT: Adding Human in the Loop Scorer #202

Merged
merged 23 commits into from
May 14, 2024
Merged

FEAT: Adding Human in the Loop Scorer #202

merged 23 commits into from
May 14, 2024

Conversation

rlundeen2
Copy link
Contributor

@rlundeen2 rlundeen2 commented May 13, 2024 •
edited
Loading

Description

  • Updating memory documentation for how users can import/export memory using DBeaver
  • Added HiTL scorer, which can take input from STDIN or a csv
  • Added code to import CSVs from a TextTarget to allow for importing prompts.
  • Added validation for these scenarios so Literals are explicitly checked

Tests and Documentation

Scoring and memory jupytext run

@rlundeen2 rlundeen2 changed the title Users/rlundeen/5 10 hitl scorer FEAT: Adding Human in the Loop Scorer May 13, 2024
romanlutz
romanlutz approved these changes May 14, 2024
View reviewed changes
pyrit/score/hitl_scorer.py Outdated Show resolved Hide resolved
pyrit/prompt_target/text_target.py Show resolved Hide resolved
doc/code/memory/memory.py Show resolved Hide resolved
@rlundeen2 rlundeen2 merged commit 2b1e4a9 into Azure:main May 14, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romanlutz romanlutz romanlutz approved these changes

@rdheekonda rdheekonda rdheekonda approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@rlundeen2 @romanlutz @rdheekonda