Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter to set PRNG seed #47

Open
jemc opened this issue Jun 26, 2024 · 0 comments
Open

Add parameter to set PRNG seed #47

jemc opened this issue Jun 26, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@jemc
Copy link
Collaborator

jemc commented Jun 26, 2024

For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.

Current expectations of support across LLM providers:

  • OpenAI has this as a documented feature
  • VertexAI Gemini doesn't appear to have it, but it may be an undocumented feature
  • Anthropic Claude doesn't appear to have it.
  • Local LLMs should be able to easily support it.

For LLMs which do not support this feature, if the optional seed parameter is given by the user, Kurt should throw a KurtFeatureIncompatibleError (an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant