Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0003: Smart Routing #5

Open
mkrueger12 opened this issue Dec 23, 2023 · 2 comments
Open

0003: Smart Routing #5

mkrueger12 opened this issue Dec 23, 2023 · 2 comments
Assignees
@mkrueger12
Labels
enhancement New feature or request

Comments

@mkrueger12
Copy link
Contributor

Add a pool that is the "smart" routing pool. Effectively a pool that classifies prompts and sends them to the optimal endpoint. Prompts could be classified via any number of categories.

For example, say that a prompt comes in that is best answered by a specialized, legal endpoint. The smart pool classifies and routes it to the legal endpoint.

Likely involves traditional ML.
@mkrueger12 mkrueger12 self-assigned this Dec 23, 2023
@roma-glushko
Copy link
Member

This is a great feature to have! I would consider it post-MVP but we surely could think of it right now.

@roma-glushko roma-glushko added the enhancement New feature or request label Dec 27, 2023
@mkrueger12
Copy link
Contributor Author

I agree on the timing as well. For sure post-MVP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mkrueger12 mkrueger12

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

0003-smart-routing EinStack/geps
2 participants
@roma-glushko @mkrueger12