-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
meta-llama-3.1 client samples (#3304)
* meta-llama-3.1 client samples * black formatting * Update langchain.ipynb --------- Co-authored-by: shail2208 <59747407+shail2208@users.noreply.github.com>
- Loading branch information
Showing
4 changed files
with
926 additions
and
0 deletions.
There are no files selected for viewing
232 changes: 232 additions & 0 deletions
232
sdk/python/foundation-models/meta-llama-3.1/langchain.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,232 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Use LangChain with Meta-Llama-3.1-405B-Instruct for Azure AI\n", | ||
| "\n", | ||
| "You can use Meta-Llama-3.1-405B-Instruct - or any of the Llama 3.1 models - as a serverless API endpoint through Azure AI Studio and Azure ML with `langchain` to create more sophisticated intelligent applications. Models available on Azure as a serverless API endpoint:\n", | ||
| "\n", | ||
| "- `Meta-Llama-3.1-405B-Instruct`\n", | ||
| "- `Meta-Llama-3.1-70B-Instruct`\n", | ||
| "- `Meta-Llama-3.1-8B-Instruct`\n", | ||
| "\n", | ||
| "The above models are supported by cross-regional support for any region in the US.\n", | ||
| "\n", | ||
| "> Review the [documentation](https://aka.ms/meta-llama-3.1-405B-instruct-azure-ai-studio-docs) for Meta-Llama-3.1-405B-Instruct and other Llama 3.1 models through AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Prerequisites\n", | ||
| "\n", | ||
| "Before we start, there are certain steps we need to take to deploy the models:\n", | ||
| "\n", | ||
| "* Register for a valid Azure account with subscription \n", | ||
| "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", | ||
| "* Create a project and resource group\n", | ||
| "* Select `Meta-Llama-3.1-405B-Instruct`.\n", | ||
| "\n", | ||
| " > Notice that some models may not be available in all the regions in Azure AI and Azure Machine Learning. On those cases, you can create a workspace or project in the region where the models are available and then consume it with a connection from a different one. To learn more about using connections see [Consume models with connections](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deployments-connections)\n", | ||
| "\n", | ||
| "* Deploy with \"Pay-as-you-go\"\n", | ||
| "\n", | ||
| "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference.\n", | ||
| "\n", | ||
| "For more information, you should consult Azure's official documentation [here](https://aka.ms/meta-llama-3.1-405B-instruct-azure-ai-studio-docs) for model deployment and inference.\n", | ||
| "\n", | ||
| "To complete this tutorial, you will need to:\n", | ||
| "\n", | ||
| "* Install `langchain` and `langchain_community`\n", | ||
| "\n", | ||
| " ```bash\n", | ||
| " pip install langchain langchain_community\n", | ||
| " ```" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Example\n", | ||
| "\n", | ||
| "The following example demonstrate how to create a chain that uses `Meta-Llama-3.1-405B-Instruct` model deployed through Azure AI and Azure ML. The chain has been configured with a `ConversationBufferMemory`. This example has been adapted from [LangChain official documentation](https://python.langchain.com/docs/modules/memory/adding_memory)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%pip install langchain langchain_community" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "imports" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from langchain.chains import LLMChain\n", | ||
| "from langchain.memory import ConversationBufferMemory\n", | ||
| "from langchain.prompts import (\n", | ||
| " ChatPromptTemplate,\n", | ||
| " HumanMessagePromptTemplate,\n", | ||
| " MessagesPlaceholder,\n", | ||
| ")\n", | ||
| "from langchain.schema import SystemMessage\n", | ||
| "from langchain_community.chat_models.azureml_endpoint import (\n", | ||
| " AzureMLChatOnlineEndpoint,\n", | ||
| " AzureMLEndpointApiType,\n", | ||
| " LlamaChatContentFormatter,\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Let's create an instance of our `AzureMLChatOnlineEndpoint` model. This class allow us to get access to any model deployed in Azure AI or Azure ML. For completion models use class `langchain_community.llms.azureml_endpoint.AzureMLOnlineEndpoint` with `LlamaContentFormatter` as the `content_formatter`.\n", | ||
| "\n", | ||
| "- `endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL.\n", | ||
| "- `api_key`: Use your API key." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "chat_client" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "chat_model = AzureMLChatOnlineEndpoint(\n", | ||
| " endpoint_url=\"https://<endpoint-name>.<region>.inference.ai.azure.com/v1/chat/completions\",\n", | ||
| " endpoint_api_type=AzureMLEndpointApiType.serverless,\n", | ||
| " endpoint_api_key=\"<key>\",\n", | ||
| " content_formatter=LlamaChatContentFormatter(),\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "In the below prompt, we have two input keys: one for the actual input (`human_input`), and another for the input from the `Memory` class (`chat_history`)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "prompt-template" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "prompt = ChatPromptTemplate.from_messages(\n", | ||
| " [\n", | ||
| " SystemMessage(\n", | ||
| " content=\"You are a chatbot having a conversation with a human. You love making references to pop culture in your answers.\"\n", | ||
| " ),\n", | ||
| " MessagesPlaceholder(variable_name=\"chat_history\"),\n", | ||
| " HumanMessagePromptTemplate.from_template(\"{human_input}\"),\n", | ||
| " ]\n", | ||
| ")\n", | ||
| "\n", | ||
| "memory = ConversationBufferMemory(memory_key=\"chat_history\", return_messages=True)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We create the chain as follows:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "chain" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "chat_llm_chain = LLMChain(\n", | ||
| " llm=chat_model,\n", | ||
| " prompt=prompt,\n", | ||
| " memory=memory,\n", | ||
| " verbose=True,\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We can see how it works:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "chat_invoke" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "chat_llm_chain.predict(human_input=\"Hi there my friend\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "name": "chat_invoke_2" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "chat_llm_chain.predict(\n", | ||
| " human_input=\"Who is the most renowned architect in the world? Provide a short poem that describes their work in the style of Shakespeare with Iambic pentimeter and a rhythm pattern of ABABCC\"\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Aditional resources\n", | ||
| "\n", | ||
| "Here are some additional reference: \n", | ||
| "\n", | ||
| "* [Plan and manage costs (marketplace)](https://learn.microsoft.com/azure/ai-studio/how-to/costs-plan-manage#monitor-costs-for-models-offered-through-the-azure-marketplace)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "jupyter", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.10.11" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
Oops, something went wrong.