anthropic[patch]: add vertex and bedrock support, streamResponseChunk…

[tofuliang]

Add support for vertex and bedrock, optimize stream response chunks

This pull request adds support for vertex and bedrock, and optimizes the *_streamResponseChunks method for better efficiency.

Changes:

• Added support for vertex and bedrock:
  • Added necessary configurations and logic to handle vertex and bedrock requests.
• Optimized *_streamResponseChunks method:
  • Simplified the logic for handling response chunks, making it more concise and efficient.

Question:

• Should the extractToken method return tool_use related tokens?
  • Currently, the extractToken method returns tool_use related tokens as new tokens to handleLLMNewToken. This might be problematic because these tokens are not actual language model outputs for human.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 26, 2024 10:47pm
langchainjs-docs	✅ Ready (Inspect)	Visit Preview		Jul 26, 2024 10:47pm

[afirstenberg]

Making this a required, rather than an optional, dependency adds a library for an edge case. (It also suggests that we can't include this in the google-vertex package, since we're not using this library there.)

[bracesproul]

I just spoke async with the rest of the LangChain team, and we decided not to include Bedrock/Vertex support via the Anthropic package. For Anthropic via Bedrock, the new recommended approach is via ChatBedrockConverse in the @langchain/aws package. As for Vertex, any changes should be pushed to the @langchain/google-common package, which will then be used in @langchain/google-vertexai.

I pushed up a commit removing these additions, but still keeping the other refactors you made. Thanks for cleaning that up!

[danny-avila]

After this update (0.2.10), anthropic will stream with bound tools but it will not stream tool chunks.

Related issues:

langchain-ai/langgraphjs#253 (closed but still ongoing issue)
#6238