New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prompt playground #355

Open

zilto wants to merge 4 commits into main from feat/playground

Collaborator

zilto commented Sep 6, 2024

Allows to load traces from tracked Burr application with OpenLLMetry traces.

You can load previous chat interactions and try new prompts with multiple LLM providers via LiteLLM.

To launch the app

OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... streamlit run burr/integrations/playground/app.py

Features:

3 providers at once
load interactions directly from local Burr tracking
load a specific interaction via the URL params
the playground interactions themselves are logged using Burr under burr-playground

Limitations:

streamlit makes it very inconvenient to call LLMs asynchronously
currently no streamlined way to support generic API key input

zilto added 4 commits

September 6, 2024 11:47


          added playground

cb4ffc9


          added generic streamlit components; removed streamlit integration dep…

e08509e

… on hamilton


          fixed improper filtering

bc790ef


          fix bug for Anthropic that doesn't return a 'role'

ec2f69c

zilto requested a review from elijahbenizzy

September 24, 2024 14:08

Contributor

elijahbenizzy commented Sep 25, 2024 •

edited

Loading

OK, looking good. A few minor UI points after playing around:

Model not selected doesn't run it -- maybe we should be able to schose an arbitrary number of them? Or just grey it out if it's not selected?
Launch button is ina. weird place, I think it should be below the provider/model selection
Selecting a model resets everything -- it shouldn't reset the other ones -- we shoud have a reset button
We should have a progress button if it's running
The third one never seems to work for me...
Tab to link to the burr UI (iframe?) -- we should be able to see the trace it came from + the trace it led to
See what I'm working with:

elijahbenizzy reviewed

View reviewed changes

Contributor

elijahbenizzy left a comment

Some thoughts here -- a few nits on the structure then feedback on the UI

burr/integrations/playground/app.py



		@st.cache_data
		def instrument(provider: str):

Contributor

elijahbenizzy Sep 24, 2024

You can replace this all with init_instruments(), right?

burr/integrations/playground/app.py

+                      msg = f"Couldn't instrument {provider}. Try installing `opentelemetry-instrumenation-{provider}"
+                  if msg:
+                      print(msg)

Contributor

elijahbenizzy Sep 24, 2024

logger.exception

burr/integrations/playground/app.py



		@action(reads=["history"], writes=["history"])
		def generate_answer(

Contributor

elijahbenizzy Sep 24, 2024

I'm not sure the value of a single-node burr app, I think it might confuse people.

The standard pattern is to break this into two -- one that processes the input, and one that outputs the result of querying the LLM.

We also could have one per model we're evaluating, but that's a bit more complex.

burr/integrations/playground/app.py

		@@ -0,0 +1,303 @@
		import litellm

Contributor

elijahbenizzy Sep 24, 2024

Add README saying this is experimental + a bit of instructions. Also could be a tab on the app?

burr/integrations/playground/app.py

		@@ -0,0 +1,303 @@
		import litellm

Contributor

elijahbenizzy Sep 24, 2024

I'm not sure this is an integration -- it's more a "tool"? Maybe it should live somewhere else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet