Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This does not currently work.
This pr attempts to use input/output mapping to deal with LLMs in a more generic way. What I've discovered is that there's simply too much going on with both our existing implementation as well as in individual LLM clients. I think that overhauling to this level is too big a change for the minor version update with regards to the risk and testing surface area involved.
This PR is missing a few key parts:
Fundamentally, I do think this approach can work, and is a better mapping solution than the one that we currently have since in some ways it's "looser". It does not include recreating clients or trying to type-match into them, which I think is a huge benefit from a maintainability perspective.
All that being said, I would propose a difference in deliverables for 0.4.3 that I think are still achievable, help with debug scenarios, and keep backwards compatability without being too huge a change. However, I only think these changes are appropriate if we do necessary pruning on 0.5.0.
0.4.3
__call__
llm_api
callable optional. When it is optional, we pass through information to a LiteLLM client we create internallynaked_llm_call
function onguard
. This function would have inputs styled the same way we style__call__
inputs, but purely uses our internal mapping to make a call to the LLM without guarding. This would help users debug and translate from their call pattern to the guard-style call pattern.0.5.0
messages
for BaseCallable. For everything else, pass through args as is