Generative AI agents are designed to interact with their environment to achieve specific objectives, such as automating repetitive tasks and augmenting human capabilities. By orchestrating multistep workflows that adapt to evolving goals in real time, these agents increase productivity, reduce errors, and deliver more personalized experiences. To manage these complex workflows effectively, agents rely on an orchestration strategy that coordinates interactions with various tools, knowledge sources, and other agents. This orchestration allows agents to analyze data, interpret context, sequence tasks, and adapt to shifting requirements, making sure that workflows remain efficient, accurate, and resilient.
Amazon Bedrock Agents streamlines the development of generative AI applications by offering a fully managed solution that uses foundation models (FMs) and augmenting tools to autonomously run tasks and achieve objectives through orchestrated, multistep workflows. Using the default orchestration strategy, reasoning and action (ReAct), users can quickly build and deploy agentic solutions. ReAct is a general problem-solving approach that uses the FM’s planning capabilities to dynamically adjust actions at each step. Although ReAct offers flexibility by allowing agents to continually reevaluate their decisions based on shifting requirements, its iterative approach can lead to higher latency when many tools are involved.
For greater orchestration control, Amazon Bedrock Agents has launched the custom orchestrator feature, which users can use to fine-tune agent behavior and manage tool interactions at each workflow step. This customization allows organizations to tailor agent functionality to their specific operational needs, improving precision, adaptability, and efficiency. In this post, we explore how custom orchestrators work and demonstrate their application with the default Bedrock Agent’s ReAct and reasoning without observation (ReWoo) examples.
Custom orchestrator overview
Implemented by users as an AWS Lambda function, the Amazon Bedrock Agents custom orchestrator offers granular control over task planning, completion, and verification. Unlike the default ReAct orchestration method, which prioritizes decision transparency and step-by-step reasoning, the custom orchestrator gives users the ability to define strategies that are better aligned with specific use case requirements. In ReAct, FM and tool invocations follow a sequential, step-by-step process, where each action depends on the outcome of the previous one. This structured, linear approach offers transparency, making it easier to trace the reasoning behind each action and decision while also promoting consistency through predictable workflows. Although ReAct’s design provides incremental adaptability by allowing agents to reassess actions at each step, its sequential structure may introduce delays when rapid parallel actions are required or when workflows demand instant responsiveness across multiple steps. This makes ReAct less suited to scenarios where speed and rapid sequential processing are paramount, such as in complex, high-volume workflows.
The custom orchestrator offers an alternative, more flexible approach, which users can use to define orchestration strategies that are more closely aligned with their specific requirements. With real-time adjustments and precise control over FM and tool interactions, users can create workflows that provide the optimal balance of performance, accuracy, and resilience. After a custom orchestrator is created, it can be reused across multiple agents by updating a single reference when configuring new agents.
Key benefits of the custom orchestrator include:
- Full control over orchestration strategies – Tailor agent workflows for optimal performance across various metrics, such as accuracy, speed, and resilience. Use Amazon Bedrock Agents built-in integrations with action groups, knowledge bases, and guardrails to streamline interactions.
- Real-time adjustments – Dynamically adjust agent actions based on the current context, tool outputs, or evolving user requirements so the agent adapts efficiently and effectively to new information.
- Reusability and consistency – After an orchestration strategy is created, it can be implemented across all relevant agents, saving time and promoting consistency.
In this post, we compare the invocations of an Amazon Bedrock agent with the default ReAct prompts with the invocations of an Amazon Bedrock agent with a custom orchestration implementing the ReWoo strategy. First, we examine the underlying contracts and state management principles that drive its adaptability.
Custom orchestrator workflow management
The custom orchestrator enables dynamic decision-making and adaptable workflow management through contract-based interactions between Amazon Bedrock Agents and AWS Lambda. The Lambda function acts as the orchestration engine, processing contextual inputs—such as state, conversation history, session parameters, and user requests—to generate instructions and define the state for subsequent actions. Upon receiving user input, Amazon Bedrock Agents uses the custom orchestrator logic and the Amazon Bedrock Converse API to manage interactions between the underlying FM and various tools, such as action groups, knowledge bases, and guardrails.
The following diagram illustrates the flow of interactions between the user, Amazon Bedrock Agents, and the custom orchestrator, which manages the workflow:
The custom orchestrator workflow includes the following steps:
- User input – The process begins when the user submits a request or query. This input is sent to Amazon Bedrock Agents, initiating the workflow.
- Custom orchestrator initiation – Amazon Bedrock Agents passes the user input to the custom orchestrator, which initiates the orchestration process in the
STARTstate. The orchestrator guides the workflow through intermediate steps to process the input. - Tool interactions – Amazon Bedrock Agents interacts with various tools to manage the request:
- Knowledge bases – Provide relevant context or information based on user input.
- Action groups – Invoke predefined action groups, which include:
- Lambda functions for custom logic
- Return of control (RoC) functions to sequence steps
- Code interpreter (CI) functions for code execution
- Guardrails – Makes sure responses comply with predefined criteria or safety standards.
- Converse API – Manages conversation flow and processes natural language responses between Amazon Bedrock Agents and the FM.
- Session attributes – Manage session-specific data, such as long-term memory, session attributes, and knowledge base configurations, personalizing and maintaining context across interactions.
- Custom orchestrator workflow – As Amazon Bedrock Agents interacts with various tools, the custom orchestrator tracks progress through states, adjusting the workflow as necessary. After the workflow reaches completion, the orchestrator signals it using the
FINISHaction event. - Final output – Amazon Bedrock Agents generates and delivers the final output to the user, completing the interaction.
This workflow highlights how Amazon Bedrock Agents, guided by the custom orchestrator, coordinates various steps and manages the flow of information to fulfill the user request. Through state transitions, the orchestrator makes sure that each action follows a structured sequence, enabling dynamic and flexible control over the workflow. Next, we explore how state transitions and contract-based interactions structure customizable workflow management.
State and event management
State management is central to guiding the progression of interactions and determining the next steps in the workflow. States represent specific stages or conditions, allowing the orchestration engine to track and manage actions. These states make sure that the workflow proceeds in an orderly manner, with each action dependent on the current state. States are passed in the request schema from Amazon Bedrock Agents to the customer orchestrator handled through the Lambda function. In contrast, events are actions that drive state transitions or invoke further actions. Events are passed in the response schema from AWS Lambda to Amazon Bedrock Agents.
Each interaction between the agent and the custom orchestrator starts with a “START” state and ends with a “FINISH” event. During the orchestration, the custom orchestrator Lambda can receive “START”, “MODEL_INVOKED”, “TOOL_INVOKED”, “APPLY_GUARDRAILS_INVOKED”, or a custom defined state as input and will output “FINISHED”, “INVOKE_MODEL”, “INVOKE_TOOL”, “APPLY_GUARDRAILS”, or a custom defined event. The flow between states and events is shown in the following figure.
Each state transition occurs in response to specific events, allowing the workflow to adapt dynamically based on input and context. For example, when a FINISH event response is received, the orchestrator is signaling that workflow is complete. The custom orchestrator Lambda function then streams the output back to Amazon Bedrock Agents, which streams it to the user. This mechanism provides a smooth and responsive interaction, enabling effective orchestration of tasks. The requests and response contract-based interactions are handled through JSON events as detailed here.
By using these contract-based interactions, Amazon Bedrock Agents and the custom orchestrator Lambda function collaborate effectively to process contextual inputs, manage state transitions, and produce accurate, tailored responses. This flexible architecture is critical for handling complex workflows that require real-time adjustments and precise control over the agent’s behavior.
Custom orchestrator workflow patterns: ReAct and ReWoo
To illustrate the power and flexibility of the custom orchestrator, the next section examines two orchestration strategies—default Bedrock Agent’s ReAct and ReWoo—and explores how each addresses trade-offs in agent workflows. To further explore the flexibility and potential of the custom orchestrator, consider a restaurant example use case. In this use case, we have an Amazon Bedrock Agent that has one action group that can connect to three APIs: create reservation, update existing reservation, and delete reservation. The agent also connects with a knowledge base that indexes the different menus for the food served in this restaurant. The following diagram shows the agent architecture.
Default orchestrator: ReAct
The default Amazon Bedrock Agents ReAct approach is an iterative decision-making process where the model analyzes each step, deciding on the next action based on the information gathered at each stage, as shown in the following figure.
This method provides transparency and allows for a clear, step-by-step breakdown of actions, making it well-suited for workflows that benefit from incremental adjustments. Although effective in dynamic environments where real-time reevaluation is advantageous, ReAct’s sequential structure can introduce latency when a complex plan is required. For instance, considering the restaurant assistant example, when asking simple queries such as “What do you serve for dinner?” or “Can you make a reservation for two people, at 7pm tonight?” the agent plan will consist of a single action that doesn’t have a much higher latency. However, when considering a more complex query such as “What do you serve for dinner? Can you make a reservation for four people, at 9pm tonight.” The agent plan will have multiple steps. At each step the results are observed, and the plan is adapted as shown in the following diagram. Notice that the plan is implicit, and the thought provides the next step. After each step, a new model invocation is done to determine the next step or to provide the final answer.
ReWoo
The ReWoo technique optimizes performance by generating a complete task plan up front and executing it without checking intermediate outputs, as shown in the following flow diagram.
This approach minimizes model calls, significantly reducing response times for queries that require interaction with multiple tools. For tasks where speed is prioritized over iterative adjustments—or where the intermediate reasoning steps should remain hidden for security reasons—ReWoo offers clear advantages over the default ReAct strategy.
A key source of agent latency is the number of FM calls required to complete a task. Although the default ReAct strategy requires at least N+1 calls for N steps, ReWoo reduces this to at most two calls to the model for any number of tools, cutting down model invocations and, consequently, response time. For example, for a task that takes 9 seconds with three model invocations with ReAct, the difference would be marginal with ReWoo because the task would still take two model invocations. However, as the complexity scales, the latency difference becomes bigger. For instance, a task taking 18 seconds with six model invocations could take only 9 seconds and two model invocations with ReWoo—a difference that scales with the complexity of the workflow.
When analyzing the query “What do you serve for dinner? Can you make a reservation for four people, at 9pm tonight,” with ReWoo the agent will create a plan to access the knowledge base for the dinner menu information and the action group to create a new dinner reservation without validating intermediate steps as shown in the following video clip.
When running this query with an agent using Anthropic’s Claude Sonnet 3.5 v2, we observed a 50–70% latency reduction for the complex query. You can find the implementation of this solution in our GitHub repository amazon-bedrock-samples.
It’s important to notice that although ReWoo has advantages for speed, it does have a more complex prompt, and you need to build a parser for the output, which makes it a more difficult strategy to implement. This is one reason why you should weigh speed, accuracy, and complexity of solution when creating a new orchestration strategy.
Conclusion
In this post, we explored how Amazon Bedrock Agents simplifies the orchestration of generative AI workflows, particularly with the introduction of the custom orchestrator feature. You can use the custom orchestrator to fine-tune and optimize agentic workflows that align more closely with specific business and operational needs. We outlined the feature’s key benefits, including full control over orchestration, real-time adjustments, and reusability, followed by a breakdown of how it manages state transitions and contract-based interactions between Amazon Bedrock Agents and AWS Lambda.
We then dove deeper into the default ReAct and a custom ReWoo orchestration strategies, and discussed the trade-offs between flexibility and performance. Through the detailed workflow management, state events, and contract interactions applied to a custom ReWoo implementation, we highlighted how the custom orchestrator adapts to dynamic conditions, and you can therefore build more efficient and accurate AI applications. We also illustrated examples of simplified ReAct and ReWoo orchestration strategies and the trade-offs between flexibility and performance.
To learn more about custom orchestrator techniques and get started with end-to-end examples, refer to our GitHub repository.
About the Authors
Kyle T. Blocksom is a Sr. Solutions Architect with AWS based in Southern California. Kyle’s passion is to bring people together and leverage technology to deliver solutions that customers love. Outside of work, he enjoys surfing, eating, wrestling with his dog, and spoiling his niece and nephew.
Maira Ladeira Tanke is a Tech Lead Amazon Bedrock for Generative AI Agents at AWS. With a background in machine learning, she has over 10 years of experience architecting and building AI applications with customers across industries. As a technical lead, she helps customers accelerate their achievement of business value through generative AI solutions on Amazon Bedrock. In her free time, Maira enjoys traveling, playing with her cat, and spending time with her family someplace warm.
Mark Roy is a Principal Machine Learning Architect for AWS, helping customers design and build generative AI solutions. His focus since early 2023 has been leading solution architecture efforts for the launch of Amazon Bedrock, the flagship generative AI offering from AWS for builders. Mark’s work covers a wide range of use cases, with a primary interest in generative AI, agents, and scaling ML across the enterprise. He has helped companies in insurance, financial services, media and entertainment, healthcare, utilities, and manufacturing. Prior to joining AWS, Mark was an architect, developer, and technology leader for over 25 years, including 19 years in financial services. Mark holds six AWS certifications, including the ML Specialty Certification.
John Baker is a Principal SDE at AWS where he works on Amazon Bedrock and specifically Amazon Bedrock Agents. He has been with Amazon for more than 10 years and has worked across AWS, Alexa, and Amazon.com. In his spare time, John enjoys skiing and other outdoor activities throughout the Pacific Northwest.
Sudip Dutta is a senior Software Developer engineer leading the development of Amazon Bedrock Agents custom orchestrator. With more than 17 year of experience developing distributed systems and architectures he has worked at AWS for the past 6 years focusing on ML and AI services such as Bedrock and Lex. On his free time Sudip enjoys hiking in the forest of pacific northwest or reading mystery novels!
Source link
lol
