fix: Support for Meta models provided by Amazon SageMaker

[JGalego]

Refactors AWSMeta to work with Amazon SageMaker endpoints.

Fixes #1009

Example

import os

import boto3
import dspy

from sagemaker.jumpstart.model import JumpStartModel

# 0a. Get AWS region
region_name = boto3.session.Session().region_name

# 0b. Deploy Llama3 70B Instruct model via SageMaker JumpStart SDK
# https://aws.amazon.com/blogs/machine-learning/meta-llama-3-models-are-now-available-in-amazon-sagemaker-jumpstart/
# https://aws.amazon.com/blogs/machine-learning/deploy-and-fine-tune-foundation-models-in-amazon-sagemaker-jumpstart-with-two-lines-of-code/
model = JumpStartModel(
    model_id="meta-textgeneration-llama-3-70b-instruct",
    model_version="2.0.2",
    instance_type="ml.g5.48xlarge",
    region=region_name
)
if os.environ.get('ENDPOINT_NAME'):
    predictor = model.deploy(
        accept_eula=input("Accept EULA? [y/n]") == "y"
    )

# 1. Initialize SageMaker provider
sagemaker = dspy.Sagemaker(
    region_name=region_name
)

# 2. Configure Meta model
lm_sagemaker = dspy.AWSMeta(
    aws_provider=sagemaker,
    model=os.environ.get('ENDPOINT_NAME', predictor.endpoint_name)
)
dspy.configure(lm=lm_sagemaker)

# 3. Create Q&A CoT
qa = dspy.ChainOfThought('question -> answer')

# 4. Run a query
response = qa(question="Who was Albert Einstein?")
print(response)

# 999. Clean up everything 🧹
predictor.delete_endpoint()

Output

Prediction(
    rationale="Here is the completed response:\n\nQuestion: Who was Albert Einstein?\nReasoning: Let's think step by step in order to identify the famous physicist. We know that Albert Einstein was a renowned German-born physicist who is widely regarded as one of the most influential scientists of the 20th century.",
    answer='Albert Einstein was a renowned German-born physicist who is widely regarded as one of the most influential scientists of the 20th century.'
)

[JGalego]

Published in blog post form on AWS Community: https://community.aws/content/2im2sOOlZMSY5rCz2qZ1We3OPna/make-programs-not-prompts-dspy-pipelines-with-llama-3-on-amazon-sagemaker-jumpstart

[arnavsinghvi11]

Thanks @JGalego for the PR and the wonderful blog post!