langgraph_self_reflective_rag.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5afcaed0-3d55-4e1f-95d3-c32c751c29d8",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Self Reflective RAG for a Business\n",
    "\n",
    "Self Reflective RAG is an advanced and state-of-the-art strategy that unites (1) [query analysis](https://blog.langchain.dev/query-construction/) with (2) [active / self-corrective RAG](https://blog.langchain.dev/agentic-rag-with-langgraph/).\n",
    "\n",
    "The implementation is inspired by this [paper](https://arxiv.org/abs/2403.14403). \n",
    "I have chosen LangGraph to build the RAG and Gradio to build the chatting interface. Skip to [chatting interface](#setup-chatting-interface).\n",
    "\n",
    "The architecture involves following data sources/ Routing:\n",
    "* URL and pdf for the vector store\n",
    "* SQL database\n",
    "* Web Search using\n",
    "* Fallback conversational LLM\n",
    "  \n",
    "The Self-Reflection loop includes:\n",
    "* Grading retrieved documents -> re-retrieve or change the data source if document is not relevant\n",
    "* Hallucination checker -> re-generates the response if hallucination is found\n",
    "* Answer checker -> checks whether the generated answer addresses the user query or not otherwise generates again\n",
    "  \n",
    "Additionally, the LangSmith is used to trace the RAG for evaluation and debugging\n",
    "\n",
    "![flow_chart.svg](flow_chart.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a85501ca-eb89-4795-aeab-cdab050ead6b",
   "metadata": {},
   "source": [
    "# Setup Environment  \n",
    "\n",
    "install all the required dependencies  \n",
    "environment variables required: `OPENAI_API_KEY`, `TAVILY_API_KEY`, `LANGCHAIN_API_KEY`, and `PINECONE_API_KEY`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "id": "53d1a740-9fea-4a6e-8f95-fb9dbf1c80a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "! pip install -U \\\n",
    "    langchain_community tiktoken langchain-openai\\\n",
    "        langchainhub chromadb langchain langgraph\\\n",
    "        tavily-python pypdf pinecone-notebooks \\\n",
    "        langchain-pinecone gradio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "222f204d-956f-4128-b597-2c698120edda",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "018c9e9f-8199-4f33-b5b4-7adfb66d6219",
   "metadata": {},
   "source": [
    "### Tracing\n",
    "\n",
    "Using [LangSmith](https://docs.smith.langchain.com/) for tracing: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "08edba00-988a-478b-96fc-ae0199cbef49",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Tracing \n",
    "\n",
    "# Hard coded example not recommended, rather setup these as environment variables\n",
    "# os.environ[\"LANGCHAIN_TRACING_V2\"]=\"true\"\n",
    "# os.environ[\"LANGCHAIN_ENDPOINT\"]=\"https://api.smith.langchain.com\"\n",
    "# os.environ[\"LANGCHAIN_PROJECT\"]=\"Test\"\n",
    "# os.environ[\"LANGCHAIN_API_KEY\"] = \"<your-api-key>\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2877f6",
   "metadata": {},
   "source": [
    "# Setup Pinecone Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7e9e01d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# setup the pinecone api key\n",
    "\n",
    "# pinecone_api_key = os.environ.get(\"PINECONE_API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b32b8890",
   "metadata": {},
   "source": [
    "### Create an Index\n",
    "uncomment to create an index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8faffe8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import time\n",
    "\n",
    "# from pinecone import Pinecone, ServerlessSpec\n",
    "# pc = Pinecone(api_key=pinecone_api_key)\n",
    "\n",
    "# index_name = \"self-reflective-rag\n",
    "\n",
    "# existing_indexes = [index_info[\"name\"] for index_info in pc.list_indexes()]\n",
    "\n",
    "# if index_name not in existing_indexes:\n",
    "#     pc.create_index(\n",
    "#         name=index_name,\n",
    "#         dimension=1536,\n",
    "#         metric=\"cosine\",\n",
    "#         spec=ServerlessSpec(cloud=\"aws\", region=\"us-east-1\"),\n",
    "#     )\n",
    "#     while not pc.describe_index(index_name).status[\"ready\"]:\n",
    "#         print(\"sleeping\")\n",
    "#         time.sleep(1)\n",
    "\n",
    "# index = pc.Index(index_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c67b4cb9",
   "metadata": {},
   "source": [
    "### Setup Multi Source Index  \n",
    "\n",
    "* Using `WebBaseLoader` to parse `urls` to create knowledge source. \n",
    "* Using `PyPDFLoader` to parse the pdfs.\n",
    "* `Embedding` is to the `OpenAIEmbeddings`\n",
    "* Using `RecursiveCharacterTextSplitter` to create chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95fd5c71",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Build Index\n",
    "\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "from langchain.document_loaders import PyPDFLoader\n",
    "from langchain_pinecone import PineconeVectorStore\n",
    "\n",
    "\n",
    "# Set embeddings\n",
    "embd = OpenAIEmbeddings()\n",
    "\n",
    "# Docs to index\n",
    "\n",
    "# urls to index\n",
    "urls = [\n",
    "    \"https://lilianweng.github.io/posts/2023-06-23-agent/\",\n",
    "    \"https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/\",\n",
    "    \"https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/\",\n",
    "]\n",
    "\n",
    "# pdf to index\n",
    "pdf_loader = PyPDFLoader(\"Magic_SEO.pdf\")\n",
    "\n",
    "# Load\n",
    "docs = [WebBaseLoader(url).load() for url in urls]\n",
    "pdf_pages = pdf_loader.load()\n",
    "\n",
    "docs_list = [item for sublist in docs for item in sublist] + pdf_pages\n",
    "\n",
    "\n",
    "# Split\n",
    "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
    "    chunk_size=500, \n",
    "    chunk_overlap=20, \n",
    "    separators=[\"\\n\\n\", \"\\n\", \"(?<=\\. )\", \" \", \"\"]\n",
    ")\n",
    "doc_splits = text_splitter.split_documents(docs_list)\n",
    "\n",
    "# Add to vectorstore\n",
    "index_name = \"self-reflective-rag\" \n",
    "vectorstore = PineconeVectorStore.from_documents(\n",
    "    documents=doc_splits, \n",
    "    embedding=embd, \n",
    "    index_name=index_name\n",
    ")\n",
    "\n",
    "# Setup Pinecone as the retriever\n",
    "retriever = vectorstore.as_retriever(search_kwargs={\"k\": 3})\n",
    "\n",
    "# Test the retriever\n",
    "retriever.invoke(\"What is Agent?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ce884cc",
   "metadata": {},
   "source": [
    "### SQL Agent\n",
    "\n",
    "To query the SQL database, we can use the the SQL Agent by LangChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "id": "9e8e1c9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The customer with the name Shannon Garcia bought the product \"Stumps\".'"
      ]
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.utilities import SQLDatabase\n",
    "from langchain_community.agent_toolkits import create_sql_agent\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "\n",
    "db = SQLDatabase.from_uri(\"sqlite:///db.sqlite3\")\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "\n",
    "\n",
    "sql_agent = create_sql_agent(llm, db=db, agent_type=\"openai-tools\") \n",
    "sql_agent.invoke({\"input\": \"Which product did the customer with name Shannon Garcia buy?\"})[\"output\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f52b427-750c-40f8-8893-e9caab3afd8d",
   "metadata": {},
   "source": [
    "# Setup Chains \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa8339a9",
   "metadata": {},
   "source": [
    "### Router\n",
    "\n",
    "Router is responsible for selecting an appropriate knowledge "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4dec9d98-f3dc-4b7f-abc0-9d01c754f2be",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "datasource='web_search'\n",
      "datasource='vectorstore'\n",
      "datasource='conversation'\n",
      "datasource='vectorstore'\n",
      "datasource='sql_database'\n",
      "datasource='sql_database'\n"
     ]
    }
   ],
   "source": [
    "### Router\n",
    "\n",
    "from typing import Literal\n",
    "\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.pydantic_v1 import BaseModel, Field\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "\n",
    "# Data model\n",
    "class RouteQuery(BaseModel):\n",
    "    \"\"\"Route a user query to the most relevant datasource.\"\"\"\n",
    "\n",
    "    datasource: Literal[\"vectorstore\", \"web_search\", \"conversation\", \"sql_database\"] = Field(\n",
    "        ...,\n",
    "        description=\"Given a user question choose to route it to sql database, web search, conversation or a vectorstore.\",\n",
    "    )\n",
    "\n",
    "\n",
    "# LLM with function call\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "structured_llm_router = llm.with_structured_output(RouteQuery)\n",
    "\n",
    "# Prompt\n",
    "system = \"\"\"You are an expert at routing a user question to a vectorstore, conversation or web search.\n",
    "The vectorstore contains documents related to agents, prompt engineering, Magic SEO and adversarial attacks.\n",
    "Use the vectorstore for questions on these topics only. \n",
    "\n",
    "The SQL database contains data related to customers, orders, products and shipment information.\n",
    "Use the SQL database for questions on these topics.\n",
    "If a user uses informal conversation, then use conversation. Otherwise, use web-search.\"\"\"\n",
    "route_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")\n",
    "# Router chain\n",
    "question_router = route_prompt | structured_llm_router\n",
    "\n",
    "# Run test\n",
    "print(question_router.invoke({\"question\": \"Who will the Bears draft first in the NFL draft?\"}))\n",
    "print(question_router.invoke({\"question\": \"What are the types of agent memory?\"}))\n",
    "print(question_router.invoke({\"question\": \"How are you?\"}))\n",
    "print(question_router.invoke({\"question\": \"What services do you offer??\"}))\n",
    "print(question_router.invoke({\"question\": \"Which product did the customer with name Shannon Garcia buy?\"}))\n",
    "print(question_router.invoke({\"question\": \"What are the products did Shlok Taneja buy?\"}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a207368",
   "metadata": {},
   "source": [
    "### Retrieved Document Grader\n",
    "\n",
    "The Grader is responsible to grade the retrieved documents as relevant or not"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "856801cb-f42a-44e7-956f-47845e3664ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Retrieval Grader\n",
    "\n",
    "# Data model\n",
    "class GradeDocuments(BaseModel):\n",
    "    \"\"\"Binary score for relevance check on retrieved documents.\"\"\"\n",
    "\n",
    "    binary_score: str = Field(\n",
    "        description=\"Documents are relevant to the question, 'yes' or 'no'\"\n",
    "    )\n",
    "\n",
    "\n",
    "# LLM with function call\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "structured_llm_grader = llm.with_structured_output(GradeDocuments)\n",
    "\n",
    "# Prompt\n",
    "system = \"\"\"You are a grader assessing relevance of a retrieved document to a user question. \\n \n",
    "    If the document contains keyword(s) or semantic meaning related to the user question, grade it as relevant. \\n\n",
    "    It does not need to be a stringent test. The goal is to filter out erroneous retrievals. \\n\n",
    "    Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question.\"\"\"\n",
    "grade_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"Retrieved document: \\n\\n {document} \\n\\n User question: {question}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Retriever grader chain\n",
    "retrieval_grader = grade_prompt | structured_llm_grader\n",
    "\n",
    "# Run test\n",
    "question = \"What is agent\"\n",
    "docs = retriever.invoke(question)\n",
    "doc_txt = docs[0].page_content\n",
    "res_retgr = retrieval_grader.invoke({\"question\": question, \"document\": doc_txt})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "56341691",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "binary_score='no'\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'no'"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(res_retgr)\n",
    "res_retgr.binary_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a2e205c",
   "metadata": {},
   "source": [
    "### Generator\n",
    "\n",
    "Responsible for generating the final query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "2272333e-50b2-42ab-b472-e1055a3b94a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "An agent is a software program that performs tasks on behalf of a user or another program. It can be used in various fields such as AI, SEO, and video generation. Agents can automate processes, gather information, and make decisions based on predefined rules.\n"
     ]
    }
   ],
   "source": [
    "### Generate\n",
    "\n",
    "from langchain import hub\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "# Prompt\n",
    "prompt = hub.pull(\"rlm/rag-prompt\")\n",
    "\n",
    "# LLM\n",
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "\n",
    "# Post-processing\n",
    "def format_docs(docs):\n",
    "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
    "\n",
    "\n",
    "# Generate Chain\n",
    "rag_chain = prompt | llm | StrOutputParser()\n",
    "\n",
    "# Test Run\n",
    "generation = rag_chain.invoke({\"context\": format_docs(docs), \"question\": question})\n",
    "print(generation)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e28bbe1",
   "metadata": {},
   "source": [
    "### Fallback Conversation Chain\n",
    "\n",
    "To carry out casual conversation, the fallback chain can be used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2710f7d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "### Fallback Conversation \n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "system = \"\"\"You are a highly knowledgeable assistant of the company called CyberPun \n",
    "    Your name is PunAssist. If a user ask you anything other than CyberPun and greetings you must not reply to that\n",
    "    and remind them that you are just an assistant and they should only ask something related to CyberPun.\n",
    "    \"\"\"\n",
    "    \n",
    "conv_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"{question}\")\n",
    "    ]\n",
    ")\n",
    "\n",
    "# LLM\n",
    "conv_llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "\n",
    "# Conversation Chain\n",
    "conversation_chain = conv_prompt | conv_llm | StrOutputParser()\n",
    "\n",
    "# Test Run\n",
    "conv_test1 = conversation_chain.invoke({\"question\":\"What is an Agent\"})\n",
    "print(conv_test1)\n",
    "conv_test2 = conversation_chain.invoke({\"question\":\"For which company do you work for?\"})\n",
    "print(conv_test2)\n",
    "conv_test3 = conversation_chain.invoke({\"question\":\"Do you know how many people work at CyberPun?\"})\n",
    "print(conv_test3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f97d6f2",
   "metadata": {},
   "source": [
    "### Hallucination Grader\n",
    "\n",
    "Checks whether the generated response is grounded or the LLM is producing gibberish"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "f0c08d14-77a0-4eed-b882-2d636abb22a3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "binary_score='yes'\n"
     ]
    }
   ],
   "source": [
    "### Hallucination Grader\n",
    "\n",
    "\n",
    "# Data model\n",
    "class GradeHallucinations(BaseModel):\n",
    "    \"\"\"Binary score for hallucination present in generation answer.\"\"\"\n",
    "\n",
    "    binary_score: str = Field(\n",
    "        description=\"Answer is grounded in the facts, 'yes' or 'no'\"\n",
    "    )\n",
    "\n",
    "\n",
    "# LLM with function call\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "structured_llm_grader = llm.with_structured_output(GradeHallucinations)\n",
    "\n",
    "# Prompt\n",
    "system = \"\"\"You are a grader assessing whether an LLM generation is grounded in / supported by a set of retrieved facts. \\n \n",
    "     Give a binary score 'yes' or 'no'. 'Yes' means that the answer is grounded in / supported by the set of facts.\"\"\"\n",
    "hallucination_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"Set of facts: \\n\\n {documents} \\n\\n LLM generation: {generation}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Hallucination chain\n",
    "hallucination_grader = hallucination_prompt | structured_llm_grader\n",
    "\n",
    "# Run test\n",
    "print(hallucination_grader.invoke({\"documents\": docs, \"generation\": generation}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b7d1a05",
   "metadata": {},
   "source": [
    "### Answer Grader\n",
    "\n",
    "Grades based on whether the generated answer addresses the asked question or not"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "ded99680-437a-4c9d-b860-619c88949d84",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "binary_score='yes'\n"
     ]
    }
   ],
   "source": [
    "### Answer Grader\n",
    "\n",
    "\n",
    "# Data model\n",
    "class GradeAnswer(BaseModel):\n",
    "    \"\"\"Binary score to assess answer addresses question.\"\"\"\n",
    "\n",
    "    binary_score: str = Field(\n",
    "        description=\"Answer addresses the question, 'yes' or 'no'\"\n",
    "    )\n",
    "\n",
    "\n",
    "# LLM with function call\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "structured_llm_grader = llm.with_structured_output(GradeAnswer)\n",
    "\n",
    "# Prompt\n",
    "system = \"\"\"You are a grader assessing whether an answer addresses / resolves a question \\n \n",
    "     Give a binary score 'yes' or 'no'. Yes' means that the answer resolves the question.\"\"\"\n",
    "answer_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"User question: \\n\\n {question} \\n\\n LLM generation: {generation}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Answer grader chain\n",
    "answer_grader = answer_prompt | structured_llm_grader\n",
    "\n",
    "# Run test\n",
    "print(answer_grader.invoke({\"question\": question, \"generation\": generation}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26ee97da",
   "metadata": {},
   "source": [
    "### Question Re-Writer\n",
    "\n",
    "used when the LLM is unable to produce good results due to either bad or difficult input query "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "9d75f1d7-a47a-4577-bb0d-84b504b0867e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'What is the definition of an agent?'"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### Question Re-writer\n",
    "\n",
    "# LLM\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "\n",
    "# Prompt\n",
    "system = \"\"\"You a question re-writer that converts an input question to a better version that is optimized \\n \n",
    "     for vectorstore retrieval. Look at the input and try to reason about the underlying semantic intent / meaning.\"\"\"\n",
    "re_write_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\n",
    "            \"human\",\n",
    "            \"Here is the initial question: \\n\\n {question} \\n Formulate an improved question.\",\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Question rewriter chain\n",
    "question_rewriter = re_write_prompt | llm | StrOutputParser()\n",
    "\n",
    "# Run test\n",
    "question_rewriter.invoke({\"question\": question})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d07c0b31-b919-4498-869f-9673125c2473",
   "metadata": {},
   "source": [
    "### Web Search\n",
    "\n",
    "adds web searching capabilities "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "01d829bb-1074-4976-b650-ead41dcb9788",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Search\n",
    "\n",
    "from langchain_community.tools.tavily_search import TavilySearchResults\n",
    "\n",
    "# web search tool\n",
    "web_search_tool = TavilySearchResults(k=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "679b6687",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'url': 'https://www.geeksforgeeks.org/agents-artificial-intelligence/',\n",
       " 'content': 'Here are a few:\\nCharacteristics of an Agent\\nTypes of Agents\\nAgents can be grouped into five classes based on their degree of perceived intelligence and capability :\\nSimple Reflex Agents\\nSimple reflex agents ignore the rest of the percept history and act only on the basis of the current percept. Data Structures and Algorithms\\nML & Data Science\\nWeb Development\\nLanguages\\nInterview Corner\\nCS Subjects\\nJobs\\nPractice\\nContests\\nAgents in Artificial Intelligence\\nIn artificial intelligence, an agent is a computer program or system that is designed to perceive its environment, make decisions and take actions to achieve a specific goal or set of goals. Uses of Agents\\nAgents are used in a wide range of applications in artificial intelligence, including:\\nOverall, agents are a versatile and powerful tool in artificial intelligence that can help solve a wide range of problems in different fields.\\n MAS can be classified into different types based on their characteristics, such as whether the agents have the same or different goals, whether the agents are cooperative or competitive, and whether the agents are homogeneous or heterogeneous.\\n Interaction of Agents with the Environment\\nStructure of an AI Agent\\nTo understand the structure of Intelligent Agents, we should be familiar with Architecture and Agent programs.'}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_search = web_search_tool.invoke({\"query\":\"Define what is an agent in Artificial Intelligence\"})\n",
    "test_search[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efbbff0e-8843-45bb-b2ff-137bef707ef4",
   "metadata": {},
   "source": [
    "# Setup Graph \n",
    "\n",
    "Capture the flow in as a graph.\n",
    "\n",
    "### Graph state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "e723fcdb-06e6-402d-912e-899795b78408",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List\n",
    "\n",
    "from typing_extensions import TypedDict\n",
    "\n",
    "\n",
    "class GraphState(TypedDict):\n",
    "    \"\"\"\n",
    "    Represents the state of our graph.\n",
    "\n",
    "    Attributes:\n",
    "        question: question\n",
    "        generation: LLM generation\n",
    "        documents: list of documents\n",
    "    \"\"\"\n",
    "\n",
    "    question: str\n",
    "    generation: str\n",
    "    documents: List[str]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e2d6c0d-42e8-4399-9751-e315be16607a",
   "metadata": {},
   "source": [
    "### Graph Flow \n",
    "\n",
    "Each node has a corresponding straight forward function, and each edge has a conditional function \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "id": "b76b5ec3-0720-443d-85b1-c0e79659ca0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.schema import Document\n",
    "\n",
    "\n",
    "def transform_query(state):\n",
    "    \"\"\"\n",
    "    Transform the query to produce a better question.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): Updates question key with a re-phrased question\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---TRANSFORM QUERY---\")\n",
    "    question = state[\"question\"]\n",
    "    documents = state[\"documents\"]\n",
    "\n",
    "    # Re-write question\n",
    "    better_question = question_rewriter.invoke({\"question\": question})\n",
    "    return {\"documents\": documents, \"question\": better_question}\n",
    "\n",
    "def retrieve(state):\n",
    "    \"\"\"\n",
    "    Retrieve documents\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): New key added to state, documents, that contains retrieved documents\n",
    "    \"\"\"\n",
    "    print(\"---RETRIEVE---\")\n",
    "    question = state[\"question\"]\n",
    "\n",
    "    # Retrieval\n",
    "    documents = retriever.invoke(question)\n",
    "    return {\"documents\": documents, \"question\": question}\n",
    "\n",
    "def conversation(state):\n",
    "    \"\"\"\n",
    "    Acts as a fall back conversation chain. \n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): New key added to state, generation, that contains LLM generation\n",
    "    \"\"\"\n",
    "    print(\"---CONVERSATION---\")\n",
    "    question = state[\"question\"]\n",
    "    \n",
    "    # Conversation\n",
    "    conversation = conversation_chain.invoke({\"question\": question})\n",
    "    return { \"question\": question, \"generation\": conversation}\n",
    "    \n",
    "def query_sql(state):\n",
    "    \"\"\"\n",
    "    Queries SQL database on the question. \n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): New key added to state, generation, that contains LLM generation from SQL \n",
    "    \"\"\"\n",
    "    print(\"---SQL DATABASE---\")\n",
    "    question = state[\"question\"]\n",
    "    \n",
    "    # Conversation\n",
    "    sql_results = sql_agent.invoke({\"input\": question})[\"output\"]\n",
    "    return { \"question\": question, \"generation\": sql_results}\n",
    "    \n",
    "    \n",
    "def grade_documents(state):\n",
    "    \"\"\"\n",
    "    Determines whether the retrieved documents are relevant to the question.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): Updates documents key with only filtered relevant documents\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---CHECK DOCUMENT RELEVANCE TO QUESTION---\")\n",
    "    question = state[\"question\"]\n",
    "    documents = state[\"documents\"]\n",
    "\n",
    "    # Score each doc\n",
    "    filtered_docs = []\n",
    "    for d in documents:\n",
    "        score = retrieval_grader.invoke(\n",
    "            {\"question\": question, \"document\": d.page_content}\n",
    "        )\n",
    "        grade = score.binary_score\n",
    "        if grade == \"yes\":\n",
    "            print(\"---GRADE: DOCUMENT RELEVANT---\")\n",
    "            filtered_docs.append(d)\n",
    "        else:\n",
    "            print(\"---GRADE: DOCUMENT NOT RELEVANT---\")\n",
    "            continue\n",
    "    return {\"documents\": filtered_docs, \"question\": question}\n",
    "\n",
    "\n",
    "def web_search(state):\n",
    "    \"\"\"\n",
    "    Web search based on the re-phrased question.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): Updates documents key with appended web results\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---WEB SEARCH---\")\n",
    "    question = state[\"question\"]\n",
    "\n",
    "    # Web search\n",
    "    docs = web_search_tool.invoke({\"query\": question})\n",
    "    web_results = \"\\n\".join([d[\"content\"] for d in docs])\n",
    "    web_results = Document(page_content=web_results)\n",
    "\n",
    "    return {\"documents\": web_results, \"question\": question}\n",
    "\n",
    "\n",
    "def generate(state):\n",
    "    \"\"\"\n",
    "    Generate answer\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        state (dict): New key added to state, generation, that contains LLM generation\n",
    "    \"\"\"\n",
    "    print(\"---GENERATE---\")\n",
    "    question = state[\"question\"]\n",
    "    documents = state[\"documents\"]\n",
    "\n",
    "    # RAG generation\n",
    "    generation = rag_chain.invoke({\"context\": documents, \"question\": question})\n",
    "    return {\"documents\": documents, \"question\": question, \"generation\": generation}\n",
    "\n",
    "\n",
    "### Edges ###\n",
    "def route_question(state):\n",
    "    \"\"\"\n",
    "    Route question to web search, conversation or RAG.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        str: Next node to call\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---ROUTE QUESTION---\")\n",
    "    question = state[\"question\"]\n",
    "    source = question_router.invoke({\"question\": question})\n",
    "    if source.datasource == \"web_search\":\n",
    "        print(\"---ROUTE QUESTION TO WEB SEARCH---\")\n",
    "        return \"web_search\"\n",
    "    elif source.datasource == \"vectorstore\":\n",
    "        print(\"---ROUTE QUESTION TO RAG---\")\n",
    "        return \"vectorstore\"\n",
    "    elif source.datasource == \"sql_database\":\n",
    "        print(\"--ROUTE QUESTION TO SQL DATABASE--\")\n",
    "        return \"sql_database\"\n",
    "    elif source.datasource == \"conversation\":\n",
    "        print(\"--ROUTE QUESTION TO CONVERSATION\")\n",
    "        return \"conversation\"\n",
    "\n",
    "\n",
    "def decide_to_generate(state):\n",
    "    \"\"\"\n",
    "    Determines whether to generate an answer, or re-generate a question.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        str: Binary decision for next node to call\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---ASSESS GRADED DOCUMENTS---\")\n",
    "    state[\"question\"]\n",
    "    filtered_documents = state[\"documents\"]\n",
    "\n",
    "    if not filtered_documents:\n",
    "        # All documents have been filtered check_relevance\n",
    "        # We will re-generate a new query\n",
    "        print(\n",
    "            \"---DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, TRANSFORM QUERY---\"\n",
    "        )\n",
    "        return \"transform_query\"\n",
    "    else:\n",
    "        # We have relevant documents, so generate answer\n",
    "        print(\"---DECISION: GENERATE---\")\n",
    "        return \"generate\"\n",
    "\n",
    "\n",
    "def grade_generation_v_documents_and_question(state):\n",
    "    \"\"\"\n",
    "    Determines whether the generation is grounded in the document and answers question.\n",
    "\n",
    "    Args:\n",
    "        state (dict): The current graph state\n",
    "\n",
    "    Returns:\n",
    "        str: Decision for next node to call\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"---CHECK HALLUCINATIONS---\")\n",
    "    question = state[\"question\"]\n",
    "    documents = state[\"documents\"]\n",
    "    generation = state[\"generation\"]\n",
    "\n",
    "    score = hallucination_grader.invoke(\n",
    "        {\"documents\": documents, \"generation\": generation}\n",
    "    )\n",
    "    grade = score.binary_score\n",
    "\n",
    "    # Check hallucination\n",
    "    if grade == \"yes\":\n",
    "        print(\"---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---\")\n",
    "        # Check question-answering\n",
    "        print(\"---GRADE GENERATION vs QUESTION---\")\n",
    "        score = answer_grader.invoke({\"question\": question, \"generation\": generation})\n",
    "        grade = score.binary_score\n",
    "        if grade == \"yes\":\n",
    "            print(\"---DECISION: GENERATION ADDRESSES QUESTION---\")\n",
    "            return \"useful\"\n",
    "        else:\n",
    "            print(\"---DECISION: GENERATION DOES NOT ADDRESS QUESTION---\")\n",
    "            return \"not useful\"\n",
    "    else:\n",
    "        print(\"---DECISION: GENERATION IS NOT GROUNDED IN DOCUMENTS, RE-TRY---\")\n",
    "        return \"not supported\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ab01f36-5628-49ab-bfd3-84bb6f1a1b0f",
   "metadata": {},
   "source": [
    "## Build Graph"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33afa774",
   "metadata": {},
   "source": [
    "### Setup memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "id": "2e40a6dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup memory for the RAG\n",
    "\n",
    "from langgraph.checkpoint.sqlite import SqliteSaver\n",
    "\n",
    "memory = SqliteSaver.from_conn_string(\":memory:\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81084507",
   "metadata": {},
   "source": [
    "### Build the Graph\n",
    "\n",
    "* first create nodes as per the chosen architecture - individual processes\n",
    "* implement edges and conditional edges - implements execution flow based on the conditions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "id": "67854e07-9293-4c3c-bf9a-bc9a605570ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langgraph.graph import END, StateGraph, START\n",
    "\n",
    "workflow = StateGraph(GraphState)\n",
    "\n",
    "# Define the nodes\n",
    "workflow.add_node(\"web_search\", web_search)  # web search\n",
    "workflow.add_node(\"retrieve\", retrieve)  # retrieve\n",
    "workflow.add_node(\"conversation\", conversation)  # fallback conversation\n",
    "workflow.add_node(\"query_sql\", query_sql)  # query SQL database\n",
    "workflow.add_node(\"grade_documents\", grade_documents)  # grade documents\n",
    "workflow.add_node(\"generate\", generate)  # generate\n",
    "workflow.add_node(\"transform_query\", transform_query)  # transform_query\n",
    "\n",
    "# Build graph\n",
    "workflow.add_conditional_edges(\n",
    "    START,\n",
    "    route_question,\n",
    "    {\n",
    "        \"conversation\": \"conversation\",\n",
    "        \"web_search\": \"web_search\",\n",
    "        \"vectorstore\": \"retrieve\",\n",
    "        \"sql_database\": \"query_sql\",\n",
    "    },\n",
    ")\n",
    "\n",
    "workflow.add_edge(\"query_sql\", END)\n",
    "workflow.add_edge(\"conversation\", END)\n",
    "workflow.add_edge(\"web_search\", \"generate\")\n",
    "workflow.add_edge(\"retrieve\", \"grade_documents\")\n",
    "workflow.add_conditional_edges(\n",
    "    \"grade_documents\",\n",
    "    decide_to_generate,\n",
    "    {\n",
    "        \"transform_query\": \"transform_query\",\n",
    "        \"generate\": \"generate\",\n",
    "    },\n",
    ")\n",
    "workflow.add_edge(\"transform_query\", \"retrieve\")\n",
    "workflow.add_conditional_edges(\n",
    "    \"generate\",\n",
    "    grade_generation_v_documents_and_question,\n",
    "    {\n",
    "        \"not supported\": \"generate\",\n",
    "        \"useful\": END,\n",
    "        \"not useful\": \"transform_query\",\n",
    "    },\n",
    ")\n",
    "\n",
    "# Compile\n",
    "app = workflow.compile(checkpointer=memory)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ed9600d",
   "metadata": {},
   "source": [
    "### Test the Graph before implementing chat interface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "29acc541-d726-4b75-84d1-a215845fe88a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---ROUTE QUESTION---\n",
      "---ROUTE QUESTION TO WEB SEARCH---\n",
      "---WEB SEARCH---\n",
      "\"Node 'web_search':\"\n",
      "'\\n---\\n'\n",
      "---GENERATE---\n",
      "---CHECK HALLUCINATIONS---\n",
      "---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---\n",
      "---GRADE GENERATION vs QUESTION---\n",
      "---DECISION: GENERATION ADDRESSES QUESTION---\n",
      "\"Node 'generate':\"\n",
      "'\\n---\\n'\n",
      "('Caleb Williams from USC was the player expected to be drafted first by the '\n",
      " 'Chicago Bears in the 2024 NFL draft. He was selected with the No. 1 pick and '\n",
      " 'was considered the top prospect in a draft class filled with talented '\n",
      " 'quarterbacks.')\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "\n",
    "# Run\n",
    "inputs = {\n",
    "    \"question\": \"What player at the Bears expected to draft first in the 2024 NFL draft?\"\n",
    "}\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "for output in app.stream(inputs, thread):\n",
    "    for key, value in output.items():\n",
    "        # Node\n",
    "        pprint(f\"Node '{key}':\")\n",
    "        # Optional: print full state at each node\n",
    "        # pprint.pprint(value[\"keys\"], indent=2, width=80, depth=None)\n",
    "    pprint(\"\\n---\\n\")\n",
    "\n",
    "# Final generation\n",
    "pprint(value[\"generation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19ac1f6f-2d84-488f-8a0e-7ee2a46b0f71",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run\n",
    "inputs = {\"question\": \"For which company do you work for?\"}\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "for output in app.stream(inputs, thread):\n",
    "    for key, value in output.items():\n",
    "        # Node\n",
    "        pprint(f\"Node '{key}':\")\n",
    "        \n",
    "    pprint(\"\\n---\\n\")\n",
    "\n",
    "# Final generation\n",
    "pprint(value[\"generation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49eb778e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run\n",
    "inputs = {\"question\": \"What is CyberPun?\"}\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "for output in app.stream(inputs, thread):\n",
    "    for key, value in output.items():\n",
    "        # Node\n",
    "        pprint(f\"Node '{key}':\")\n",
    "        \n",
    "    pprint(\"\\n---\\n\")\n",
    "\n",
    "# Final generation\n",
    "pprint(value[\"generation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "cbff6240",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---ROUTE QUESTION---\n",
      "--ROUTE QUESTION TO SQL DATABASE--\n",
      "---SQL DATABASE---\n",
      "\"Node 'query_sql':\"\n",
      "'\\n---\\n'\n",
      "{'input': 'Which product did the customer with name Shannon Garcia buy?',\n",
      " 'output': 'The customer with the name Shannon Garcia bought the product '\n",
      "           '\"Stumps\".'}\n"
     ]
    }
   ],
   "source": [
    "# Run\n",
    "inputs = {\"question\": \"Which product did the customer with name Shannon Garcia buy?\"}\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "for output in app.stream(inputs, thread):\n",
    "    for key, value in output.items():\n",
    "        # Node\n",
    "        pprint(f\"Node '{key}':\")\n",
    "        \n",
    "    pprint(\"\\n---\\n\")\n",
    "\n",
    "# Final generation\n",
    "pprint(value[\"generation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96205a41",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run\n",
    "inputs = {\"question\": \"My name is Taha. What is my name? What is your name?\"}\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "for output in app.stream(inputs, thread):\n",
    "    for key, value in output.items():\n",
    "        # Node\n",
    "        pprint(f\"Node '{key}':\")\n",
    "        \n",
    "    pprint(\"\\n---\\n\")\n",
    "\n",
    "# Final generation\n",
    "pprint(value[\"generation\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "id": "58d69504",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import time\n",
    "\n",
    "def inference(inputs, history, *args, **kwargs):\n",
    "    \"\"\"\n",
    "    Inference Generator to support streaming\n",
    "\n",
    "    Args:\n",
    "        inputs (str): The input query for inference\n",
    "        history (list[list] | list[tuple]): The chat history; internally managed by the gradio app\n",
    "        args: additional arguments\n",
    "        kwargs: additional keyword arguments\n",
    "\n",
    "    Yields:\n",
    "      str: A string containing a portion of the generated text, simulating a gradual generation process.\n",
    "    \"\"\"\n",
    "    # thread is required for memory checkpoint\n",
    "    thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "    \n",
    "    # input the graph\n",
    "    inputs = {\"question\": inputs}\n",
    "    \n",
    "    # the output can be streamed but due to the print statements, I am using invoke\n",
    "    output = app.invoke(inputs, thread)\n",
    "    \n",
    "    output_generation = output[\"generation\"].split(\" \")\n",
    "    generated_text = \"\"\n",
    "    \n",
    "    for i in  range(len(output_generation)):\n",
    "        time.sleep(0.05)\n",
    "        generated_text = ' '.join(output_generation[:i+1])\n",
    "        \n",
    "        yield generated_text\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9862711e",
   "metadata": {},
   "source": [
    "# Setup Chatting Interface\n",
    "\n",
    "Using Gradio app to build a simple and easy to use UI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "id": "87918235",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7906\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7906/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 244,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---ROUTE QUESTION---\n",
      "--ROUTE QUESTION TO SQL DATABASE--\n",
      "---SQL DATABASE---\n"
     ]
    }
   ],
   "source": [
    "import gradio as gr\n",
    "\n",
    "chat_interface = gr.ChatInterface(\n",
    "    inference,\n",
    "    chatbot=gr.Chatbot(height=300),\n",
    "    textbox=gr.Textbox(placeholder=\"message PunAssist\", container=False, scale=7),\n",
    "    title=\"PunAssist\",\n",
    "    description=\"Ask me anything CyberPun. About company, Services, Projects, Contact Information, Products, Customers, Shipment Details, Orders and more\",\n",
    "    undo_btn=None,\n",
    "    clear_btn=\"Clear\",\n",
    ")\n",
    "\n",
    "demo = gr.TabbedInterface([chat_interface], [\"Chat\"])\n",
    "demo.launch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55e08d84",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9420f142",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a31107e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d4c6559",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf766c02",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}