Multi-agent Systems (MASs)

MASs are a team of agents that work together to achieve a common goal. You should orchestrate more than one agent in a way that they efficiently do what you need to do. Orchestration refers to defining the flow of agents. Which agents run, in what order, and how do they decide what happens next? Intelligence of an LLM is used to plan, reason, and decide on what steps to take based on that. MAS is actually an abstract concept so the flow of a MAS is actually defined when creating its agents.

Types of MASs

If you already read agents guide, you know that the resources (tools, context, etc.) that an agent can use are different for each type of the MAS. If an agent belongs to a MAS, it will have the resources of that MAS type. For example, sender’s email address is available in the context of an email MAS but not in a storefront MAS. Similarly Add to Cart may be available in the context of a storefront MAS but doesn’t make sense in an email MAS since it is not possible to add to cart from an email.

There are four main types of MASs we currently support:

• Storefront MASs: conversational These MASs are used to answer inquiries coming from the chatbot in the storefront. You can create more than one storefront MAS but you have to assign one of them to the chatbot. When assigned to the chatbot, this MAS will run once with every incoming message.
• Email MASs: conversational These MASs are used to answer inquiries coming from the email. You can create more than one email MAS but every email integration has to be assigned to one of them. When assigned to the email integration, this MAS will run once with every incoming email.
• Content MASs: background Blog/content generator that works on a schedule – e.g., weekly SEO posts
• Conversation Insights MASs: background Post‑conversation summaries and insights.

What does conversational and background mean?

They are two labels for MASs. They don’t affect the resources of the MAS but they are important to understand the purpose of the MAS. It is always conversations that come to mind first when we think of LLMs and AI Agents. However, an Agent or a MAS can be used for many other tasks thanks to the tools it can use. For example, an Agent in Content MAS can generate content and share it as a blog post using the PublishBlogPostTool. In this case, the MAS doesn’t produce a text answer but takes an action as the final output.

• Conversational MASs: These MASs are used to answer inquiries coming from any channel. Currently email and storefront messenger is supported. If a MAS is conversational, it’s purpose is to do what the customer asks and send a text response as a result. You can use Inbox in the dashboard to monitor the conversations.
• Background MASs: These MASs that runs in the background. They don’t send a response to the customer. They are used to perform tasks like data analysis, data processing, etc. Content MAS or Conversation Insights MAS are some examples of background MASs.

How to orchestrate agents to form a MAS?

There are three main parameters of an Agent that controls the flow of a MAS:

• Entry Agent: This is a very basic configuration for an Agent. You can select the agent that will be the entry point of the MAS. Then the agent will be the first one to run. Remember that there can be only one entry agent per MAS.
• Handoffs: This is the mechanism to delegate a task to a sub-agent. You can define agents that the current agent can delegate to. Then starting from the entry agent, the MAS will run the agents in the order of the handoffs.
• Handoff Description: Every agent can have a description of what it can do. This description is used to decide which agent to delegate to. So a handoff description informs the parent agent about the capabilities of the child agent. (e.g. “Use me when the customer asks for order cancellation”)

A Basic Storefront MAS Example

The tree below shows a basic Storefront MAS. It has a triage agent which breasts the initial query and transfers the control to one of the three agents: Sales Agent, Orders Agent, or Discounts Agent. Notice that there is an Order Cancellation Agent that Orders Agent can delagate to. This is a good practice because cancelling orders is a critical task and should be performed by a specialized agent. If possible, a more intelligent model should be used.

The following diagram is from the dashboard, when you define your agents, you can see the auto-generated flow of the agents on the Home page of the MAS.

Triage Agent:
- Handoff Description: ""
- Instructions: "You're the triage agent, your job is to determine which agents to be used based on the customer query. If the query is about sales you need to use Sales Agent. If the query is about shipping, orders etc. You need to use Orders agent. If the query is about discount you need to use Discount Agent."
- Handoff Agents: [Orders Agent, Discounts Agent, Sales Agent]
- Entry Agent: True
- Model: Express
- Tool Choice: Required (Remember that Agents see handoffs as tools so if you want an agent to always use a handoff, you need to select "Required")

Sales Agent:
- Handoff Description: "Use me when the customer asks for product details"
- Instructions: "You are the pre-sales assistant for `Shop Name` , your job is to help customers to find the relevant item by making a pre-sales dialogue based on store's category, always try to swap-domain specific dialogues. If a customer wants to learn about product details, you need to ask which products they are interested in. Then you call `Get Product Details` tool by retrieving product's name or ID. You can also use `Get Related Knowledge Source` tool to retrive store specific answers. For example, if user is asking about the return policy you can answer them."
- Handoff Agents: []
- Entry Agent: False
- Model: Core
- Tool Choice: Auto

Orders Agent:
- Handoff Description: "Use this handoff when a query is about orders or order details. Orders Agent can: `Get Customer Orders`, `Get Order Details` or delegate to `Order Cancellation Agent`"
- Instructions: "Use this agent to respond support queries. You can use `Get Customer Orders` tool to list the customer orders. When a customer wants to track their orders you need to ask their order address first, then call `Get Order Details` tool to learn the status of their orders."
- Handoff Agents: [Order Cancellation Agent]
- Entry Agent: False
- Model: Express
- Tool Choice: Auto

Discount Agent:
- Handoff Description: "Use this agent when a customer wants discount or finds prices too high. You can call `Create Dicount Code` tool to create one time discount and use `Send OTP Email` tool to send the verification code to the customer."
- Instructions: "Use this agent to respond support queries. You can use `Get Customer Orders` tool to list the customer orders. When a customer wants to track their orders you need to ask their order address first, then call `Get Order Details` tool to learn the status of their orders."
- Handoff Agents: []
- Entry Agent: False
- Model: Express
- Tool Choice: Auto

Order Cancellation Agent:
- Handoff Description: "Use this agent when the customer asked to cancel an order."
- Instructions: "Use this agent to cancel an order. You can use `Cancel Order` tool to cancel the order. Most imporantly check if the customer asking to cancel an order is the owner of the order. If the `Customer Logged In` parameter is true and `Customer Email` is the same as the order's email, then you can cancel the order. If not, send an OTP email to order's email using `Send OTP Email` tool to verify the customer's identity."
- Handoff Agents: []
- Entry Agent: False
- Model: Express
- Tool Choice: Auto

This pattern is great when the task is open-ended and you want to rely on the intelligence of an LLM. The most important tactics here are:

1. Invest in good prompts. Make it clear what tools are available, how to use them, and what parameters it must operate within.
2. Monitor your MAS and iterate on it. See where things go wrong, and iterate on your prompts.
3. Have specialized agents that excel in one task, rather than having a general purpose agent that is expected to be good at anything.

Best Practices for Multi-agent Systems

• Single Responsibility: One topic per agent.
• Clear Handoff Descriptions: No ambiguity in the handoff descriptions.
• Short Pipelines: Avoid >3 deep nested handoffs especially for conversational MASs
• Monitor & Iterate: Use the dashboard to monitor and refine the MAS.

---