Multi-Agent System - CopilotKit

The Travel Planner example demonstrates a complete multi-agent system where multiple specialized agents work together to help users plan trips. It showcases agent coordination, human-in-the-loop workflows, shared state management, and real-time UI updates.

Overview

The Travel Planner is a full-featured application that demonstrates:

Multi-agent architecture with LangGraph
Real-time map integration using Leaflet
Google Maps API integration for place search
Human-in-the-loop workflows for trip approvals
Shared state management between agents and UI
Streaming intermediate state for progress updates
Context-aware chat suggestions

Live Demo: https://copilotkit.ai/examples/travel-planner Source Code: GitHub

What You’ll Learn

This example teaches you how to:

Build multi-agent systems with LangGraph
Implement agent routing and node coordination
Set up human-in-the-loop interrupts
Share state between agents and React UI
Stream intermediate agent state to the frontend
Integrate external APIs (Google Maps) into agents
Create context-aware chat suggestions
Build generative UI for approval workflows

Prerequisites

Node.js 18 or higher
Python 3.11+ with uv package manager
npm, yarn, or pnpm
OpenAI API Key
Google Maps API Key

Alternatively, you can use Copilot Cloud to avoid running the agent locally.

Installation

Clone the Repository

git clone https://github.com/CopilotKit/CopilotKit.git
cd CopilotKit/examples/v1/travel

Install Frontend Dependencies

pnpm install

Using other package managers

# Using yarn
yarn install

# Using npm
npm install

Install Agent Dependencies

pnpm install:agent

This installs the Python dependencies for the LangGraph agent using uv.

Configure Environment Variables

Create a .env file in the project root:

OPENAI_API_KEY=your_openai_api_key
GOOGLE_MAPS_API_KEY=your_google_maps_api_key

Using Copilot Cloud instead:

NEXT_PUBLIC_CPK_PUBLIC_API_KEY=your_copilotkit_api_key

Start the Development Server

pnpm dev

This starts both the Next.js frontend (port 3000) and the Python agent (port 8000).

Using other package managers

# Using yarn
yarn dev

# Using npm
npm run dev

Open the Application

Navigate to http://localhost:3000 in your browser.

Architecture Overview

The Travel Planner uses a multi-node agent graph with specialized nodes for different tasks:

Agent Nodes

chat_node - Main conversational interface
trips_node - Handles trip CRUD operations (requires approval)
search_node - Searches for places using Google Maps API
perform_trips_node - Executes approved trip changes

Key Implementation Details

1. Agent Graph Definition

File: agent/src/agent.py

from langchain_core.messages import AIMessage, ToolMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import END, START, StateGraph

from src.chat import chat_node
from src.search import search_node
from src.state import AgentState
from src.trips import perform_trips_node, trips_node

def route(state: AgentState):
    """Route after the chat node based on tool calls."""
    messages = state.get("messages", [])
    if messages and isinstance(messages[-1], AIMessage):
        ai_message = messages[-1]
        
        if ai_message.tool_calls:
            tool_name = ai_message.tool_calls[0]["name"]
            
            # Route to appropriate node based on tool
            if tool_name in ["add_trips", "update_trips", "delete_trips", "select_trip"]:
                return "trips_node"
            if tool_name in ["search_for_places"]:
                return "search_node"
                
    if messages and isinstance(messages[-1], ToolMessage):
        return "chat_node"
        
    return END

# Build the graph
graph_builder = StateGraph(AgentState)

# Add nodes
graph_builder.add_node("chat_node", chat_node)
graph_builder.add_node("trips_node", trips_node)
graph_builder.add_node("search_node", search_node)
graph_builder.add_node("perform_trips_node", perform_trips_node)

# Add edges
graph_builder.add_conditional_edges(
    "chat_node", 
    route, 
    ["search_node", "chat_node", "trips_node", END]
)
graph_builder.add_edge(START, "chat_node")
graph_builder.add_edge("search_node", "chat_node")
graph_builder.add_edge("perform_trips_node", "chat_node")
graph_builder.add_edge("trips_node", "perform_trips_node")

# Compile with interrupt for human-in-the-loop
graph = graph_builder.compile(
    interrupt_after=["trips_node"],  # Wait for approval after trip changes
    checkpointer=MemorySaver(),      # Enable state persistence
)

Key features:

Conditional routing based on tool calls
Human-in-the-loop via interrupt_after on trips_node
State persistence with MemorySaver checkpointer
Modular design with separate nodes for each concern

2. Shared State with useAgent

File: lib/hooks/use-trips.tsx

import { useAgent } from "@copilotkit/react";
import { Trip, AgentState, defaultTrips } from "@/lib/types";

export const TripsProvider = ({ children }: { children: ReactNode }) => {
  const { agent } = useAgent({ agentId: "travel" });
  const state = agent.state as AgentState;

  // Helper function to update trips
  const updateTrip = (id: string, updatedTrip: Trip) => {
    agent.setState({
      ...state,
      trips: state.trips.map((trip) => 
        trip.id === id ? updatedTrip : trip
      ),
    });
  };

  return (
    <TripsContext.Provider value={{ trips: state.trips, updateTrip, ... }}>
      {children}
    </TripsContext.Provider>
  );
};

Key concepts:

useAgent provides bidirectional state sync
Agent updates automatically trigger UI re-renders
UI updates are reflected in agent state
Type-safe state management with TypeScript

3. Human-in-the-Loop Approvals

File: lib/hooks/use-trips.tsx

import { useHumanInTheLoop } from "@copilotkit/react";
import { z } from "zod";
import { AddTrips, EditTrips, DeleteTrips } from "@/components/humanInTheLoop";

// Action for adding trips
useHumanInTheLoop({
  name: "add_trips",
  description: "Add some trips",
  parameters: z.object({
    trips: z.array(z.object({
      name: z.string(),
      location: z.string(),
      dates: z.string(),
    })).describe("The trips to add"),
  }),
  render: AddTrips,  // Shows approval UI and waits for user
});

// Action for updating trips
useHumanInTheLoop({
  name: "update_trips",
  description: "Update some trips",
  parameters: z.object({
    trips: z.array(z.object({
      id: z.string(),
      name: z.string(),
      location: z.string(),
      dates: z.string(),
    })).describe("The trips to update"),
  }),
  render: (props) =>
    EditTrips({
      ...props,
      trips: state.trips,
      selectedTripId: state.selected_trip_id as string,
    }),
});

// Action for deleting trips
useHumanInTheLoop({
  name: "delete_trips",
  description: "Delete some trips",
  parameters: z.object({
    trip_ids: z.array(z.string()).describe("The ids of the trips to delete"),
  }),
  render: (props) => DeleteTrips({ ...props, trips: state.trips }),
});

File: components/humanInTheLoop/AddTrips.tsx

import { ToolCallStatus } from "@copilotkit/react";

interface AddTripsProps {
  args: { trips: Array<{ name: string; location: string }> };
  status: ToolCallStatus;
  respond: (result: any) => void;
}

export function AddTrips(props: AddTripsProps) {
  const { args, status, respond } = props;

  return (
    <div className="bg-white p-4 rounded-lg border shadow-sm">
      <h3 className="font-medium mb-3">Add New Trips</h3>
      
      <div className="space-y-2 mb-4">
        {args.trips?.map((trip, index) => (
          <div key={index} className="p-2 bg-gray-50 rounded">
            <p className="font-medium">{trip.name}</p>
            <p className="text-sm text-gray-600">{trip.location}</p>
          </div>
        ))}
      </div>

      {status !== ToolCallStatus.Complete && (
        <div className="flex gap-2">
          <button
            className="flex-1 bg-green-600 text-white px-4 py-2 rounded"
            onClick={() => respond({ approved: true, trips: args.trips })}
          >
            Approve
          </button>
          <button
            className="flex-1 bg-gray-200 px-4 py-2 rounded"
            onClick={() => respond({ approved: false })}
          >
            Reject
          </button>
        </div>
      )}
    </div>
  );
}

4. Streaming Search Progress

Backend - File: agent/src/search.py

from copilotkit.langchain import copilotkit_customize_config, copilotkit_emit_state

async def search_node(state: AgentState, config: RunnableConfig):
    # Configure intermediate state emission
    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[{
            "state_key": "search_progress",
            "tool": "search_for_places",
            "tool_argument": "search_progress",
        }],
    )
    
    # Perform search and emit progress
    state["search_progress"] = [
        {"step": "Searching Google Maps", "status": "in_progress"},
    ]
    await copilotkit_emit_state(config, state)
    
    # ... perform actual search ...
    
    state["search_progress"] = [
        {"step": "Searching Google Maps", "status": "complete"},
        {"step": "Processing results", "status": "in_progress"},
    ]
    await copilotkit_emit_state(config, state)
    
    return state

Frontend - File: lib/hooks/use-trips.tsx

import { useRenderActivityMessage } from "@copilotkit/react";
import { SearchProgress } from "@/components/SearchProgress";
import { AgentState } from "@/lib/types";

useRenderActivityMessage({
  agentId: "travel",
  render: ({ activity }) => {
    const state = activity.state as AgentState;
    if (state?.search_progress && state.search_progress.length > 0) {
      return <SearchProgress progress={state.search_progress} />;
    }
    return null;
  },
});

5. Context-Aware Chat Suggestions

File: lib/hooks/use-trips.tsx

import { useConfigureSuggestions } from "@copilotkit/react";

useConfigureSuggestions(
  {
    instructions: `Offer the user actionable suggestions based on their last message, 
                   current trips, and selected trip.
                   Selected trip: ${state.selected_trip_id}
                   Trips: ${JSON.stringify(state.trips)}`,
    minSuggestions: 1,
    maxSuggestions: 2,
  },
  [state.trips]  // Re-generate when trips change
);

This provides intelligent, context-aware suggestions like:

“Add a restaurant to your Paris trip”
“Find hotels near the Eiffel Tower”
“What’s the weather like in your destination?“

6. Google Maps Integration

File: agent/src/search.py

import googlemaps
from typing import List, Dict

def search_for_places(query: str, location: str) -> List[Dict]:
    """Search for places using Google Maps API."""
    gmaps = googlemaps.Client(key=os.getenv("GOOGLE_MAPS_API_KEY"))
    
    # Perform place search
    places_result = gmaps.places(
        query=query,
        location=location,
        radius=5000
    )
    
    results = []
    for place in places_result.get("results", [])[:5]:
        results.append({
            "name": place["name"],
            "address": place.get("formatted_address", ""),
            "rating": place.get("rating", 0),
            "lat": place["geometry"]["location"]["lat"],
            "lng": place["geometry"]["location"]["lng"],
        })
    
    return results

Expected Behavior

When you run the application, you should be able to:

View the map with existing trip markers
Chat with the AI to plan trips
Add trips - AI suggests trips, you approve them
Search for places - See real-time search progress
Update trips - Modify trip details with AI assistance
Delete trips - Remove trips with confirmation
See suggestions - Get context-aware action suggestions

Try It Yourself

Example interactions to try:

User: "I want to plan a trip to Paris"
AI: Creates a trip and shows approval UI

User: "Find some good restaurants near the Eiffel Tower"
AI: Searches Google Maps, shows progress, adds restaurants

User: "Add a museum to my Paris trip"
AI: Searches for museums, suggests options

User: "Change the dates of my Paris trip to next week"
AI: Shows edit approval with updated dates

User: "Delete the restaurant with low ratings"
AI: Shows delete confirmation

Advanced Features

LangGraph Studio

For debugging and visualizing your agent:

Install LangGraph Studio
Load the agent/ directory
Visualize the agent graph and state transitions
Debug tool calls and routing logic

Checkpointing

The agent uses checkpointing for state persistence:

graph = graph_builder.compile(
    interrupt_after=["trips_node"],
    checkpointer=MemorySaver(),  # In-memory checkpointing
)

For production, use a persistent checkpointer like PostgresSaver.

Multi-Turn Conversations

The agent maintains conversation context across multiple turns, enabling natural follow-up questions:

User: "Plan a trip to Tokyo"
AI: [Creates trip]

User: "Add some temples"
AI: [Searches for temples in Tokyo - knows context]

User: "Make it 5 days instead"
AI: [Updates Tokyo trip duration - remembers which trip]

Troubleshooting

Agent Connection Issues

If you see “I’m having trouble connecting to my tools”:

Make sure the LangGraph agent is running on port 8000
Check that your OpenAI API key is set correctly
Verify both frontend and agent servers started successfully
Check the console for any Python errors

Google Maps Not Working

Verify:

Your Google Maps API key is valid
The Places API is enabled in Google Cloud Console
The API key has the correct restrictions/permissions
The key is properly set in the .env file

Human-in-the-Loop Not Showing

Check:

The agent graph is compiled with interrupt_after=["trips_node"]
The render function is used in useHumanInTheLoop
The action names match between agent and frontend
The approval components are properly imported

State Not Syncing

Ensure:

useAgent is properly initialized with the agent ID
The agent name matches in both frontend and backend
State types are consistent between TypeScript and Python
The CopilotKit provider wraps the application

Next Steps

Human-in-the-Loop

Deep dive into approval workflows

Shared State

Learn more about state management patterns

LangGraph Guide

Complete guide to building LangGraph agents

useAgent

Reference documentation for useAgent

​Overview

​What You’ll Learn

​Prerequisites

​Installation

​Architecture Overview

​Agent Nodes

​Key Implementation Details

​1. Agent Graph Definition

​2. Shared State with useAgent

​3. Human-in-the-Loop Approvals

​4. Streaming Search Progress

​5. Context-Aware Chat Suggestions

​6. Google Maps Integration

​Expected Behavior

​Try It Yourself

​Advanced Features

​LangGraph Studio

​Checkpointing

​Multi-Turn Conversations

​Troubleshooting

​Next Steps

Human-in-the-Loop

Shared State

LangGraph Guide

useAgent

Overview

What You’ll Learn

Prerequisites

Installation

Architecture Overview

Agent Nodes

Key Implementation Details

1. Agent Graph Definition

2. Shared State with useAgent

3. Human-in-the-Loop Approvals

4. Streaming Search Progress

5. Context-Aware Chat Suggestions

6. Google Maps Integration

Expected Behavior

Try It Yourself

Advanced Features

LangGraph Studio

Checkpointing

Multi-Turn Conversations

Troubleshooting

Next Steps