Added subclass of OpenAI class to support custom endpoints #560
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new class to support for providing a custom base_url to the OpenAI client, and a couple of small updates to remove the checks on the model via the "retrieve" api (since it's not always available), sets tiktoken to use gpt4 by default (with the option to override it), and enables passing temperature as a named parameter to the __call__method to override the config.
Usage is basically the same as the regular OpenAI model wrapper, with the addition of the base_url parameter. Eg:
The environment variables checked to set the API key was also updated, so as not to conflict with any OpenAI configurations in place, and sound a little more generic, along with a new env var
INFERENCE_BASE_URL
to act as an alternative to the constructor parameter.