Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@langchain/aws Bug: ChatBedrockConverse fails when calling multiple tools simultaneously #6173

Closed
5 tasks done
tinque opened this issue Jul 22, 2024 · 2 comments · Fixed by #6175
Closed
5 tasks done
Assignees
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@tinque
Copy link
Contributor

tinque commented Jul 22, 2024

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain.js documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain.js rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I use the legacy createToolCallingAgent and AgentExecutor from "langchain/agents", but I encounter the same problem with the LangGraph agent.

import { ChatBedrockConverse } from "@langchain/aws";
import { tool } from "@langchain/core/tools";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { createToolCallingAgent, AgentExecutor } from "langchain/agents";
import { z } from "zod";

const model = new ChatBedrockConverse({
  model: "anthropic.claude-3-sonnet-20240229-v1:0",
  region: "us-east-1",
});

const weatherTool = tool(
  ({ city, state }) => `The weather in ${city}, ${state} is 72°F and sunny`,
  {
    name: "weather_tool",
    description: "Get the weather for a city",
    schema: z.object({
      city: z.string().describe("The city to get the weather for"),
      state: z.string().describe("The state to get the weather for").optional(),
    }),
  }
);

const tools = [weatherTool];

const modelWithTools = model.bindTools(tools);


const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant"],
  ["placeholder", "{chat_history}"],
  ["human", "{input}"],
  ["placeholder", "{agent_scratchpad}"],
]);

const agent = await createToolCallingAgent({ llm: model, tools, prompt });

const agentExecutor = new AgentExecutor({
  agent,
  tools,
});

const res = await agentExecutor.invoke({
  input: "What's the weather in New York and Paris?",
})
console.log(res);

Error Message and Stack Trace (if applicable)

/Users/quentin/contrib/repro_bug/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1159
  const exception = new ValidationException({
                    ^

ValidationException: The number of toolResult blocks at messages.2.content exceeds the number of toolUse blocks of previous turn.
    at de_ValidationExceptionRes (/Users/quentin/contrib/repro_bug/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1159:21)
    at de_CommandError (/Users/quentin/contrib/repro_bug/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1008:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/quentin/contrib/repro_bug/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /Users/quentin/contrib/repro_bug/node_modules/@smithy/core/dist-cjs/index.js:165:18
    at async /Users/quentin/contrib/repro_bug/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /Users/quentin/contrib/repro_bug/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
    at async ChatBedrockConverse._streamResponseChunks (file:///Users/quentin/contrib/repro_bug/node_modules/@langchain/aws/dist/chat_models.js:247:26)
    at async ChatBedrockConverse._streamIterator (file:///Users/quentin/contrib/repro_bug/node_modules/@langchain/core/dist/language_models/chat_models.js:89:34)
    at async ChatBedrockConverse.transform (file:///Users/quentin/contrib/repro_bug/node_modules/@langchain/core/dist/runnables/base.js:387:9) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'd92de5a1-fa53-42dd-88fd-d0438fafe27c',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

Node.js v20.12.0

### Description

When the LLM calls more than one tool simultaneously, the following error is encountered:

ValidationException: The number of toolResult blocks at messages.2.content exceeds the number of toolUse blocks of previous turn.



The issue seems to originate from the fact that each `ToolMessage` is converted into a `user` type message with a single `toolResult` block (see the relevant code [here](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-aws/src/common.ts#L170)). However, Bedrock expects a single message containing multiple `toolResult` blocks in the content.

For more details, refer to the AWS documentation:
- [API_runtime_Message](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Message.html)
- [API_runtime_ContentBlock](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ContentBlock.html)


### System Info

langchain@0.2.10
platform mac
node v20.12.0
npm 10.5.0
@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label Jul 22, 2024
@bracesproul
Copy link
Collaborator

Thank you for flagging this! I'll take a look tomorrow and update you with my findings.

@tinque
Copy link
Contributor Author

tinque commented Jul 23, 2024

I've proposed a fix, but I find it a bit messy... I don't have any other ideas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants