Building a Multi-Agent Conversational AI System with Amazon Bedrock

As AI systems become more sophisticated, we're moving beyond the "one model handles everything" approach. Today, I'll share how I built a conversational AI system that uses specialized agents for different tasks - all powered by Amazon Bedrock. The Problem with Generic AI Assistants Have you ever had a conversation with an AI chatbot that kept forgetting details or mixed up information from different topics? It's frustrating, right? Generic AI assistants try to handle everything—from booking cabs to tracking orders to answering random questions—in one conversational flow. This often leads to context confusion and poor user experience. Enter the Multi-Agent Architecture To solve this, I created a system that uses specialized AI agents that focus on specific tasks: A main coordinator agent that handles general queries and routes requests A cab booking specialist agent An order tracking specialist agent System Architecture Here's how the system works: The user interacts with the main agent initially Based on intent detection, the conversation gets transferred to a specialized agent The specialized agent handles its specific task until completion The user can switch between agents or return to the main menu Building the System with Amazon Bedrock Let's dive into the implementation. I used Amazon Bedrock with Claude 3.5 Sonnet as the underlying LLM. import logging import json import boto3 import requests import datetime import random from botocore.exceptions import ClientError # Configure logging logger = logging.getLogger(name) logging.basicConfig(level=logging.INFO) # Create the Bedrock client def get_bedrock_client(): return boto3.client( service_name='bedrock-runtime', aws_access_key_id="YOUR_ACCESS_KEY", aws_secret_access_key="YOUR_SECRET_KEY", region_name="REGION" ) Tool Configuration The system uses function calling (or "tools" in Amazon Bedrock terminology) to perform actions: tool_config = { "tools": [ { "toolSpec": { "name": "book_cab", "description": "Book a cab between locations", "inputSchema": { "json": { "type": "object", "properties": { "pickup": { "type": "string", "description": "Pickup location (e.g., Andheri, Bandra, Dadar)" }, "destination": { "type": "string", "description": "Drop-off location (e.g., Airport, Powai, Worli)" }, "time": { "type": "string", "description": "Pickup time in HH:MM format" }, "passengers": { "type": "integer", "description": "Number of passengers" } }, "required": ["pickup", "destination"] } } } }, { "toolSpec": { "name": "track_order", "description": "Track the status of an order", "inputSchema": { "json": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order ID to track (e.g., ORD12345)" } }, "required": ["order_id"] } } } } ] } Intent Detection The system needs to identify what the user wants to do: def detect_intent(text): text = text.lower() if any(keyword in text for keyword in ['book', 'cab', 'taxi', 'ride', 'uber', 'ola']): return 'cab_booking' elif any(keyword in text for keyword in ['order', 'track', 'package', 'delivery', 'shipment']): return 'order_tracking' else: return 'unknown' The Cab Booking Agent Let's look at how the cab booking agent is implemented: def cab_booking_agent(bedrock_client, model_id, tool_config, conversation_history=None): print("Transferring to Cab Booking Agent...") print("-" * 50) # Start with a clean conversation history if conversation_history: messages = clean_conversation_history(conversation_history) else: messages = [{ "role": "user", "content": [{ "text": """You are a specialized cab booking agent. Focus only on helping users book cabs. Ask for the follow

Apr 10, 2025 - 10:21

Building a Multi-Agent Conversational AI System with Amazon Bedrock

As AI systems become more sophisticated, we're moving beyond the "one model handles everything" approach. Today, I'll share how I built a conversational AI system that uses specialized agents for different tasks - all powered by Amazon Bedrock.

The Problem with Generic AI Assistants

Have you ever had a conversation with an AI chatbot that kept forgetting details or mixed up information from different topics? It's frustrating, right?

Generic AI assistants try to handle everything—from booking cabs to tracking orders to answering random questions—in one conversational flow. This often leads to context confusion and poor user experience.

Enter the Multi-Agent Architecture

To solve this, I created a system that uses specialized AI agents that focus on specific tasks:

A main coordinator agent that handles general queries and routes requests
A cab booking specialist agent
An order tracking specialist agent

System Architecture

Here's how the system works:

The user interacts with the main agent initially
Based on intent detection, the conversation gets transferred to a specialized agent
The specialized agent handles its specific task until completion
The user can switch between agents or return to the main menu

Building the System with Amazon Bedrock

Let's dive into the implementation. I used Amazon Bedrock with Claude 3.5 Sonnet as the underlying LLM.

import logging
import json
import boto3
import requests
import datetime
import random
from botocore.exceptions import ClientError

# Configure logging
logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

# Create the Bedrock client
def get_bedrock_client():
    return boto3.client(
        service_name='bedrock-runtime',
        aws_access_key_id="YOUR_ACCESS_KEY",
        aws_secret_access_key="YOUR_SECRET_KEY",
        region_name="REGION"
    )

Tool Configuration
The system uses function calling (or "tools" in Amazon Bedrock terminology) to perform actions:

tool_config = {
    "tools": [
        {
            "toolSpec": {
                "name": "book_cab",
                "description": "Book a cab between locations",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "pickup": {
                                "type": "string",
                                "description": "Pickup location (e.g., Andheri, Bandra, Dadar)"
                            },
                            "destination": {
                                "type": "string",
                                "description": "Drop-off location (e.g., Airport, Powai, Worli)"
                            },
                            "time": {
                                "type": "string",
                                "description": "Pickup time in HH:MM format"
                            },
                            "passengers": {
                                "type": "integer",
                                "description": "Number of passengers"
                            }
                        },
                        "required": ["pickup", "destination"]
                    }
                }
            }
        },
        {
            "toolSpec": {
                "name": "track_order",
                "description": "Track the status of an order",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "order_id": {
                                "type": "string",
                                "description": "The order ID to track (e.g., ORD12345)"
                            }
                        },
                        "required": ["order_id"]
                    }
                }
            }
        }
    ]
}

Intent Detection
The system needs to identify what the user wants to do:

def detect_intent(text):
    text = text.lower()
    if any(keyword in text for keyword in ['book', 'cab', 'taxi', 'ride', 'uber', 'ola']):
        return 'cab_booking'
    elif any(keyword in text for keyword in ['order', 'track', 'package', 'delivery', 'shipment']):
        return 'order_tracking'
    else:
        return 'unknown'

The Cab Booking Agent
Let's look at how the cab booking agent is implemented:

def cab_booking_agent(bedrock_client, model_id, tool_config, conversation_history=None):
    print("Transferring to Cab Booking Agent...")
    print("-" * 50)

    # Start with a clean conversation history
    if conversation_history:
        messages = clean_conversation_history(conversation_history)
    else:
        messages = [{
            "role": "user",
            "content": [{
                "text": """You are a specialized cab booking agent. Focus only on helping users book cabs.
                Ask for the following information if not provided: pickup location, destination, pickup time, 
                and number of passengers. Be conversational but efficient."""
            }]
        }]

    # Add specialized prompt to make agent more focused
    specialized_prompt = {
        "role": "user",
        "content": [{
            "text": """You are now a specialized cab booking agent. Your responses should be direct and to the point.
            Just ask directly for any missing information needed to book a cab."""
        }]
    }
    messages.append(specialized_prompt)

    # Main conversation loop
    booking_complete = False
    first_interaction = True
    returning_to_main = False
    new_intent = None

    while not returning_to_main and not new_intent:
        try:
            # Get user input
            if first_interaction and conversation_history:
                user_input = ""
                first_interaction = False
            else:
                print("Cab Booking Agent: ", end="")
                user_input = input()

            # Check for exit commands or intent switching
            if user_input.lower() in ["exit", "quit", "bye", "cancel", "back", "return", "main menu"]:
                print("Cab Booking Agent: Returning to main menu.")
                returning_to_main = True
                break

            # Process conversation with model
            if user_input:
                messages.append({
                    "role": "user",
                    "content": [{"text": user_input}]
                })

            # Get model response
            response = bedrock_client.converse(
                modelId=model_id,
                messages=messages,
                toolConfig=tool_config
            )

            # Rest of conversation handling
            # ...

Handling Tool Calls
When the model decides to book a cab, it uses the tool:

if stop_reason == 'tool_use':
    has_tool_use = True
    tool_requests = response['output']['message']['content']

    for tool_request in tool_requests:
        if 'toolUse' in tool_request and tool_request['toolUse']['name'] == 'book_cab':
            tool = tool_request['toolUse']
            print("Processing your cab booking request...")

            try:
                booking_details = book_cab(
                    tool['input']['pickup'],
                    tool['input']['destination'],
                    tool['input'].get('time', '15:00'),
                    tool['input'].get('passengers', 1)
                )

                tool_result = {
                    "toolUseId": tool['toolUseId'],
                    "content": [{"json": booking_details}]
                }

                # Mark booking as complete
                booking_complete = True

            except CabNotFoundError as err:
                tool_result = {
                    "toolUseId": tool['toolUseId'],
                    "content": [{"text": f"I couldn't find cabs from {tool['input']['pickup']} to {tool['input']['destination']}. Please check the locations and try again."}],
                    "status": 'error'
                }

            # Send tool result back to model
            tool_result_message = {
                "role": "user",
                "content": [{"toolResult": tool_result}]
            }
            messages.append(tool_result_message)

Main Application Loop
The heart of the system is the main loop that coordinates between agents:

def main():
    # Setup
    model_id = "YOUR MODEL ID"
    bedrock_client = boto3.client(service_name='bedrock-runtime', region_name="ap-south-1")

    print("I can help you with cab booking and order tracking.")

    # Initialize base conversation
    base_messages = [{
        "role": "user",
        "content": [{
            "text": """You are a helpful travel and shopping assistant. You can help with cab booking and order tracking.
            Keep your responses friendly and concise."""
        }]
    }]

    try:
        current_intent = None
        base_messages_copy = None

        while True:
            if current_intent is None:
                # When in main menu
                user_input = input("You: ")

                if user_input.lower() in ["exit", "quit", "bye"]:
                    print("Assistant: Goodbye! Have a great day!")
                    break

                # Add user input to conversation
                base_messages.append({
                    "role": "user",
                    "content": [{"text": user_input}]
                })

                # Detect intent from user input
                intent = detect_intent(user_input)
                base_messages_copy = base_messages.copy()

                if intent == 'cab_booking':
                    # Transfer to cab booking agent
                    current_intent = 'cab_booking'
                    next_intent = cab_booking_agent(bedrock_client, model_id, tool_config, base_messages)

                    if next_intent:
                        current_intent = next_intent
                    else:
                        current_intent = None  # Return to main menu

                elif intent == 'order_tracking':
                    # Transfer to order tracking agent
                    current_intent = 'order_tracking'
                    next_intent = order_tracking_agent(bedrock_client, model_id, tool_config, base_messages)

                    if next_intent:
                        current_intent = next_intent
                    else:
                        current_intent = None  # Return to main menu

                else:
                    # General conversation
                    response = bedrock_client.converse(
                        modelId=model_id,
                        messages=base_messages,
                        toolConfig=tool_config
                    )

                    # Display response
                    # ...

Key Features That Make This System Special

Context Isolation: Each specialized agent maintains its own conversation state, preventing context confusion.
Seamless Transitions: Users can move between different agents without losing context.
Proactive Intent Detection: The system identifies when a user wants to switch topics and transfers them to the appropriate agent.
Persistent Memory: The system remembers key information across transfers.
Error Handling: Robust error handling for API failures, invalid inputs, and edge cases.

Conclusion

The multi-agent approach represents the next evolution in conversational AI. By using specialized agents, we can create more focused, helpful, and reliable conversational experiences.

The code shared here is just a starting point. You could expand this system with more specialized agents, better natural language understanding, and integration with real backend systems.

What specialized agents would you build for your business? Let me know in the comments!