Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

have stream generate ValidationOutcome #675

Merged
merged 1 commit into from
Mar 25, 2024
Merged

Conversation

zsimjee
Copy link
Collaborator

@zsimjee zsimjee commented Mar 25, 2024

Problem

Streaming output originally looked like a string.

Raw LLM Output = xyz
Validated Output = xyz

This formatting is unusable for several reasons. It also doesn't support streaming into pydantic models.

Changes

  1. Change the StreamRunner to return Generator[ValidationOutcome, None, None] instead of Generator[str, None, None]
  2. Make sure that validation outcome is loaded with the parsed and validated run output, without doing a json.dumps
  3. Split up run.py into 4 files for readability

Note: validation_passed is poorly calculated. We need to interop with validation better and expose why validation fails per unit.

Copy link
Collaborator

@ShreyaR ShreyaR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! left a small nit, feel free to merge once fixed

docs/how_to_guides/streaming.ipynb Show resolved Hide resolved
@zsimjee zsimjee merged commit 5e7b298 into main Mar 25, 2024
36 checks passed
@ShreyaR ShreyaR deleted the bug/streaming-interface-hotfix branch March 25, 2024 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants