feature(dspy): added compatibility with ANY llamaindex LLM #1225
Conversation
|
@jerryjliu I'm sure this could use a little polishing but what's your thought on this |
| @@ -0,0 +1,335 @@ | |||
| """ | |||
| this code implements a wrapper around the llama_index library to emulate a dspy llm | |||
There was a problem hiding this comment.
feel free to add these comments into documentation for the LlamaIndexWrapper LM
| from llama_index.core.llms import LLM | ||
|
|
||
|
|
||
| def LlamaIndexOpenAIClientWrapper(llm: LLM): |
There was a problem hiding this comment.
To clarify, does this support compatibility with all Llamaindex-LLM integrations? It seems to me like the definition here is only for OpenAI clients, which would leave this change redundant to just using DSPy.OpenAI. If it is not just isolated to OpenAI, lets refactor this function to LlamaIndexLLMWrapper
There was a problem hiding this comment.
This is supposed to support all llamaindex llms. However the way i use this is by wrapping the llamaindex llm to look like an openai client object, then i pass this to a dspy azure openai llm.
Llamaindex llm.chat and llm.complete are used for the actual calls
| ) | ||
|
|
||
|
|
||
| class DspyLlamaIndexWrapper(LM): |
There was a problem hiding this comment.
refactor name to LlamaIndexWrapper. and please add to imports in lm.py as done with other modules
|
|
||
|
|
||
| class DspyLlamaIndexWrapper(LM): | ||
| """Wrapper around Azure's API for OpenAI. |
There was a problem hiding this comment.
is the azure openai documentation still relevant here? please refactor if not
|
|
||
| self.history: list[dict[str, Any]] = [] | ||
|
|
||
| def _openai_client(self): |
There was a problem hiding this comment.
refactor if the scope of this LM is not limited to just OpenAI
|
|
||
| ## ======== test ========= | ||
|
|
||
| if __name__ == '__main__': |
There was a problem hiding this comment.
remove and add to documentation for the LM
|
Hi @FarisHijazi , left some comments here. I mainly want to clarify whether this PR supports all LLM integrations with LlamaIndex as specified or only limited to OpenAI? seems like one could just use DSPy.OpenAI instead? Additionally, can you add the comments and example code to documentation for this LM as done with other LMs in our documentation here? |
|
this is dope passing it over internally to help take a look very soon! |
I've responded to this in one of the comments: supports all llamaindex llms not just openai I'll get onto addressing the comments within a few days hopefully, mind the messy code it was a quick hack, but now that i have your initial approvals I can invest time in polishing (also waiting for @jerryjliu 's feedback before I start coding) |
hello and thank you for the lovely tool
I've been struggling to get it to work with different LLM providers like Anthropic or some unorthodox LLMs that need to use the huggingface transformers modules, and I saw that dspy implements most of these but not all of them, while llama-index already has everything I've ever needed implemented.
this code implements a wrapper around the llama_index library to emulate a dspy llm
this allows the llama_index library to be used in the dspy framework since dspy has limited support for LLMs
This code is a slightly modified copy of dspy/dsp/modules/azure_openai.py
The way this works is simply by creating a dummy openai client that wraps around any llama_index LLM object and implements llm.complete and llm.chat
tested with python 3.12
dspy==0.1.4
dspy-ai==2.4.9
llama-index==0.10.35
llama-index-llms-openai==0.1.18