April 7, 2025

With the adoption of agents, LLM applications are changing from question-and-answer chatbots to dynamic systems. Agentic workflows give LLMs decision-making power to not only call APIs, but also delegate subtasks to other LLM agents.

Agentic workflows come with their own downsides, however. Adding agents to your system design may drive up your costs and drive down your quality if you’re not careful.

By breaking down your tasks into specialized agents, which we’ll call sub-agents, you can build more accurate systems and lower the risk of misalignment with goals. Here are the tactics you should be using when designing an agentic LLM system.

Design your system with a supervisor and specialist roles

Think of your agentic system as a coordinated team where each member has a different strength. Set up a clear relationship between a supervisor and other agents that know about each others’ specializations.

  1. Supervisor Agent Implement a supervisor agent to understand your goals and a definition of done. Give it decision-making capability to delegate to sub-agents based on which tasks are suited to which sub-agent.
  2. Task decomposition Break down your high-level goals into smaller, manageable tasks. For example, rather than making a single LLM call to generate an entire marketing strategy document, assign one sub-agent to create an outline, another to research market conditions, and a third one to refine the plan. Instruct the supervisor to call one sub-agent after the other and check the work after each one has finished its task.
  3. Specialized roles Tailor each sub-agent to a specific area of expertise and a single responsibility. This allows you to optimize their prompts and select the best model for each use case. For example, use a faster, more cost-effective model for simple steps, or provide tool access to only a sub-agent that would need to search the web.
  4. Clear communication Your supervisor and sub-agents need a defined handoff process between them. The supervisor should coordinate and determine when each step or goal has been achieved, acting as a layer of quality control to the workflow.

Give each sub-agent just enough capabilities to get the job done

Agents are only as effective as the tools they can access. They should have no more power than they need. Safeguards will make them more reliable.

Tool Implementation

OpenAI’s Agents SDK provides the following tools out of the box: