Skip to content

Latest commit

 

History

History
263 lines (192 loc) · 8.29 KB

README.md

File metadata and controls

263 lines (192 loc) · 8.29 KB
Raw

Helicone

Twitter

Open-source observability platform for LLMs

Helicone is an open-source observability platform for Language Learning Models (LLMs). It offers the following features:

  • 📝 Logs all of your requests to OpenAI (and other providers) in a user-friendly UI

  • 💾 Caching, custom rate limits, and retries

  • 📊 Track costs and latencies by users and custom properties

  • 🎮 Every log is a playground: iterate on prompts and chat conversations in a UI

  • 🚀 Share results and collaborate with your friends or teammates

  • 👍👎 APIs to log feedback and evaluate results

  • 📲 Sessions to group and visualize multi-step LLM interactions.

Quick Start ⚡️ Just add a Header

Get your API key by signing up here.

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

👉 Then view your logs at Helicone.

Resources

Local Setup 💻

Helicone's cloud offering is deployed on Cloudflare and ensures the lowest latency add-on to your API requests.

To get started locally, Helicone is comprised of five services:

  • Web: Frontend Platform (NextJS)
  • Worker: Proxy Logging (Cloudflare Workers)
  • Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
  • Supabase: Application Database and Auth
  • ClickHouse: Analytics Database
  • Minio: Object Storage for logs.

If you have any questions, contact help@helicone.ai or join discord.

Install Wrangler and Yarn

nvm install 18.18.0
nvm use 18.18.0
npm install -g wrangler
npm install -g yarn

Install Supabase CLI

brew install supabase/tap/supabase

Install and setup ClickHouse

# This will start clickhouse locally
python3 clickhouse/ch_hcone.py --start

Install and setup MinIO

# Install minio globally
python3 -m pip install minio

# Start minio
python3 minio_hcone.py --restart

# Minio Dashboard will be available at http://localhost:9001
# Default credentials:
# Username: minioadmin
# Password: minioadmin

Run all services

cd web

# start supabase to collect logs metadata
supabase start

# start frontend
yarn
yarn dev:local

# start workers for proxying requests
# in another terminal
cd worker
yarn
chmod +x run_all_workers.sh
./run_all_workers.sh

# start jawn (for serving the FE, collecting logs and handling API requests)
cd valhalla/jawn
cp .env.example .env
yarn && yarn dev

# Make your request to local host
curl --request POST \
  --url http://127.0.0.1:8787/v1/chat/completions \
  --header 'Authorization: Bearer <KEY>' \
  --data '{
	"model": "gpt-4o-mini",
	"messages": [
		{
			"role": "user",
			"content": "What is the UNIX Epoch?"
		}
	],
	"temperature": 1,
	"max_tokens": 7
}'

# Now you can go to localhost:3000 and create an account and see your request.
# When creating an account on localhost, you will automatically be signed in.

Setup .env file

Make sure your .env file is in web/.env. Here is an example:

NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=""
STRIPE_SECRET_KEY=""
NEXT_PUBLIC_HELICONE_BILLING_PORTAL_LINK=""
NEXT_PUBLIC_HELICONE_CONTACT_LINK="https://calendly.com/d/x5d-9q9-v7x/helicone-discovery-call"
STRIPE_PRICE_ID=""
STRIPE_STARTER_PRICE_ID=""
STRIPE_ENTERPRISE_PRODUCT_ID=""
STRIPE_STARTER_PRODUCT_ID=""
DATABASE_URL="postgresql://postgres:postgres@localhost:54322/postgres"
NEXT_PUBLIC_SUPABASE_ANON_KEY="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0"
NEXT_PUBLIC_SUPABASE_URL="http://localhost:54321"
SUPABASE_URL="http://localhost:54321"
SUPABASE_SERVICE_KEY="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZx35lJzdJsyH-qQwv8Hdp7fsn3W0YpN81IU"
NEXT_PUBLIC_HELICONE_JAWN_SERVICE="http://localhost:8585"

Community 🌍

Learn this repo with Onboard AI

learnthisrepo.com/helicone

Supported Projects

Name Docs
nextjs-chat-app Docs
langchain Docs
langchainjs Docs
ModelFusion Docs

Contributing

We are extremely open to contributors on documentation, integrations, and feature requests.

Update Cost Data

  1. Add new cost data to the costs/src/ directory. If provider folder exists, add to its index.ts. If not, create a new folder with the provider name and an index.ts and export a cost object

    Example:

    File name: costs/src/anthropic/index.ts

    export const costs: ModelRow[] = [
  {
    model: {
      operator: "equals",
      value: "claude-instant-1",
    },
    cost: {
      prompt_token: 0.00000163,
      completion_token: 0.0000551,
    },
  },
];

    We can match in 3 ways:

    • equals: The model name must be exactly the same as the value
    • startsWith: The model name must start with the value
    • includes: The model name must include the value

    Use what is most appropriate for the model

    cost object is the cost per token for prompt and completion

  2. Import the new cost data into src/providers/mappings.ts and add it to the providers array

    Example:

    File name: src/providers/mappings.ts

    import { costs as anthropicCosts } from "./providers/anthropic";

// 1. Add the pattern for the API so it is a valid gateway.
const anthropicPattern = /^https:\/\/api\.anthropic\.com/;

// 2. Add Anthropic pattern, provider tag, and costs array from the generated list
export const providers: {
  pattern: RegExp;
  provider: string;
  costs?: ModelRow[];
}[] = [
  // ...
  {
    pattern: anthropicPattern,
    provider: "ANTHROPIC",
    costs: anthropicCosts,
  },
  // ...
];

  3. Run yarn test -- -u in the cost/ directory to update the snapshot tests

  4. Run yarn copy in the cost/ directory to copy the cost data into other directories

Contributors

License

Helicone is licensed under the Apache v2.0 License.