feature(dspy): dspy-integration-with-langfuse

README.md

@@ -75,12 +75,18 @@ Or open our intro notebook in Google Colab: [<img align="center" src="https://co
 
 By default, DSPy installs the latest `openai` from pip. However, if you install old version before OpenAI changed their API `openai~=0.28.1`, the library will use that just fine. Both are supported.
 
-For the optional (alphabetically sorted) [Chromadb](https://github.com/chroma-core/chroma), [Groq](https://github.com/groq/groq-python), [Marqo](https://github.com/marqo-ai/marqo), [Milvus](https://github.com/milvus-io/milvus), [MongoDB](https://www.mongodb.com), [MyScaleDB](https://github.com/myscale/myscaledb), Pinecone, [Qdrant](https://github.com/qdrant/qdrant), [Snowflake](https://github.com/snowflakedb/snowpark-python), or [Weaviate](https://github.com/weaviate/weaviate) retrieval integration(s), include the extra(s) below:
+For the optional (alphabetically sorted) [Chromadb](https://github.com/chroma-core/chroma), [Groq](https://github.com/groq/groq-python), [Marqo](https://github.com/marqo-ai/marqo), [Milvus](https://github.com/milvus-io/milvus), [MongoDB](https://www.mongodb.com), [MyScaleDB](https://github.com/myscale/myscaledb), Pinecone, [Qdrant](https://github.com/qdrant/qdrant), [Snowflake](https://github.com/snowflakedb/snowpark-python), or [Weaviate](https://github.com/weaviate/weaviate) , [Langfuse](https://langfuse.com/) retrieval integration(s), include the extra(s) below:
 
 ```
-pip install dspy-ai[chromadb] # or [groq] or [marqo] or [milvus] or [mongodb] or [myscale] or [pinecone] or [qdrant] or [snowflake] or [weaviate]
+pip install dspy-ai[chromadb] # or [groq] or [marqo] or [milvus] or [mongodb] or [myscale] or [pinecone] or [qdrant] or [snowflake] or [weaviate] or [langfuse]
 ```
 
+langfuse is now supported(openAI、AzureOpenAI and Ollama)!
+
+Before you configure langfuse, please manually deploy the langfuse server or use Langfuse Cloud. You will get the corresponding configuration after you create a new project.
+When please configure the relevant environment variables in the project, they are `LANGFUSE_SECRET_KEY`、`LANGFUSE_PUBLIC_KEY` and `LANGFUSE_HOST`.
+Just write the environment variables and langfuse will automatically read them.[langfuse details](https://langfuse.com/docs/deployment/self-host) .
+
 ## 2) Documentation
 
 The DSPy documentation is divided into **tutorials** (step-by-step illustration of solving a task in DSPy), **guides** (how to use specific parts of the API), and **examples** (self-contained programs that illustrate usage).

dsp/modules/azure_openai.py

@@ -2,9 +2,20 @@
 import json
 import logging
 from typing import Any, Literal, Optional, cast
-
 import backoff
-import openai
+
+try:
+    """
+    langfuse has made it compatible for us.
+    If there is any error in the langfuse configuration, it will turn to request the real address(openai or azure endpoint)
+    """
+    import langfuse
+    from langfuse.openai import openai
+    logging.info(f"You are using Langfuse,version{langfuse.__version__}")
+except:
+    import openai
+    logging.info(f"You are using openai,version{openai.version.__version__}")
+
 
 from dsp.modules.cache_utils import CacheMemory, NotebookCacheMemory, cache_turn_on
 from dsp.modules.lm import LM

dsp/modules/gpt3.py

@@ -2,9 +2,19 @@
 import json
 import logging
 from typing import Any, Literal, Optional, cast
-
 import backoff
-import openai
+
+try:
+    """
+    langfuse has made it compatible for us.
+    If there is any error in the langfuse configuration, it will turn to request the real address(openai or azure endpoint)
+    """
+    import langfuse
+    from langfuse.openai import openai
+    logging.info(f"You are using Langfuse,version{langfuse.__version__}")
+except:
+    import openai
+    logging.info(f"You are using openai,version{openai.version.__version__}")
 
 from dsp.modules.cache_utils import CacheMemory, NotebookCacheMemory, cache_turn_on
 from dsp.modules.lm import LM

dsp/modules/ollama.py

@@ -1,11 +1,19 @@
 import datetime
 import hashlib
+import uuid
 from typing import Any, Literal
 
 import requests
 
 from dsp.modules.lm import LM
 
+try:
+    from langfuse import Langfuse
+    # If you need higher performance, set the "thread" value
+    langfuse = Langfuse(max_retries=2)
+except:
+    LANGFUSE = False
+
 
 def post_request_metadata(model_name, prompt):
     """Creates a serialized request object for the Ollama API."""
@@ -134,7 +142,14 @@ def basic_request(self, prompt: str, **kwargs):
             "raw_kwargs": raw_kwargs,
         }
         self.history.append(history)
-
+        if LANGFUSE:
+            langfuse.trace(
+                name="Ollama request",
+                user_id=str(uuid.uuid4()),
+                metadata={**settings_dict['options'], **request_info["usage"]},
+                input=prompt,
+                output=request_info['choices']
+            )
         return request_info
 
     def request(self, prompt: str, **kwargs):

setup.py

@@ -38,7 +38,8 @@
         "google-vertex-ai": ["google-cloud-aiplatform==1.43.0"],
         "myscale":["clickhouse-connect"],
         "groq": ["groq~=0.8.0"],
-    },	
+        "langfuse": ["langfuse~=2.36.1"]
+    },
     classifiers=[	
         "Development Status :: 3 - Alpha",	
         "Intended Audience :: Science/Research",