Your three-agent demo works first try. Then a run dies at step 7 of 9, 40s in.
Nobody can say what state it was in. You restart and pay for every token twice.
The stack you picked shaped that failure. Save this guide to pick the next one on purpose.
It covers what each option gives you, five questions that decide, and an honest verdict.
What you're choosing between
All three options can run agents that hand work to each other.
They differ in what they make you write down, and in what they hide. That second part decides how your first outage goes.
Option A: a graph you can pause and resume
LangGraph calls itself "a low-level orchestration framework and runtime" for stateful agents. You define nodes, edges, and one shared state object.
Its docs list durable execution and human-in-the-loop as core features. You can use it on its own, and the repo is MIT licensed.
Option B: roles and crews, wrapped in Flows
CrewAI is built around agents with roles, goals, and tools. A crew is a team of agents, and a Flow is the event-driven backbone that runs it.
Its README calls it "a standalone Python framework." It's MIT licensed, and you can reach a working crew fast.
Option C: a loop you own end to end
A custom loop is a plain while loop. It calls a model, runs a tool, saves state, and checks a budget.
You write every line. Nothing is hidden, and nothing is free. That's the right trade when the job is small enough to read in one sitting.
The five questions that decide
Answer these for your workflow before you read any feature list.
1. Where does state live, and can you resume it?
The graph option saves state as checkpoints per thread. Its persistence docs offer in-memory, SQLite, and Postgres savers, and point to Postgres for production.
The crew option's Flows have a persist decorator, with SQLite as the default backend. In your own loop, state lives wherever you put it.
2. Can you see why a run did what it did?
The graph option points you to its sister product for traces. The crew option's README puts tracing and observability in its paid suite. Your own loop logs what you tell it to log, and nothing more.
3. What happens when step 7 of 9 fails?
This is where checkpoints pay off. If you can't resume from the last good step, every failure costs a full rerun.
4. Can you cap cost per task?
None of the three sets a dollar budget for you. You still need a hard cap per task and a kill switch that someone owns.
5. How hard is it to leave?
Prompts and tools port easily. State shape, graph wiring, and crew config don't, so count those as switching cost. Keep framework types out of your business logic and the bill stays small.
Write the answers down as a short decision record. A decision record might look like this.
decision: multi-agent-stack
state: resumable from last good step, stored in Postgres
traces: every tool call, cost, and handoff, kept 90 days
failure: resume, then escalate to a named owner after 2 retries
budget: hard cap per task, kill switch owned by ops
exit: prompts and tools in plain modules, no framework types in themAn honest verdict per option
There's no winner across the board. Here's how we'd call it.
Pick the graph when state and recovery matter most
Choose it for long-running work with approvals, pauses, and resumes. The graph forces you to name every state and every transition, which production needs anyway.
The cost is a steeper start. Simple flows feel heavy, and the best tracing story runs through a paid product. Budget a few days for your team to think in nodes and edges.
Pick the crew when roles map cleanly to the job
Choose it when the work splits into roles with distinct tools, like researcher, writer, and reviewer. Its processes run tasks in order or through a manager agent that delegates.
The cost is that role-play can hide control flow. Watch for crews where the roles are labels without distinct data or tests.
Pick your own loop when the job is narrow
Choose it when one or two agents do a bounded job and you want every line visible. It's often the fastest path to something you can test and audit.
The cost is that you rebuild what the frameworks hand you. Checkpoints, retries, and traces all become your code to maintain.
Mixing them is fine
You can run a crew inside one node of a graph, or call a framework from your own loop. Keep the boundary typed, so you can swap either side later.
Start with the smallest option that passes your test set. Add a second agent only when it beats one agent on the same tests.
What a custom loop needs on day one
If you roll your own, treat these as 5 rules, not suggestions. Skip one, and you'll add it after the first bad night.
MAX_STEPS = 12
BUDGET_USD = 0.50
def run_task(task, store, llm, tools):
state = store.load(task.id) or {"step": 0, "spent": 0.0, "log": []}
while state["step"] < MAX_STEPS:
if state["spent"] >= BUDGET_USD:
return escalate(task, state, reason="budget")
reply = llm.next_action(task, state["log"])
state["spent"] += reply.cost_usd
if reply.done:
return finish(task, state, reply.result)
result = tools.call(reply.tool, reply.args) # allowlisted tools only
state["log"].append({"tool": reply.tool, "result": result})
state["step"] += 1
store.save(task.id, state) # checkpoint after every step
return escalate(task, state, reason="max_steps")The guards are a step cap, a dollar budget, a tool allowlist, a checkpoint after every step, and an escalation path. Whichever stack you pick, check that it gives you all five.
Test them on purpose. Kill the process mid-run, feed a tool error, and set the budget to one cent. If the task resumes, escalates, and stops where it should, the loop is ready for real work.
Store state somewhere plain. A Postgres table keyed by task ID is enough for a long time.
Keep the log append-only, so a reviewer can replay what the agent saw at each step. When something breaks at 3am, that log is the first thing anyone opens. Make sure it answers what the agent saw, what it did, and what it cost.
The control plane sits above any of them
The framework runs the agents. Something else has to answer three questions: who approved this, what did it cost, and what's stuck?
That is the job of a control plane, and it should outlive any one framework. One open-source example is Mission Control, our agent control plane (GitHub).
Its README lists adapters for both frameworks above, plus several coding agents and SDK workflows. It also says adapter depth varies by runtime.
The project is alpha. Pin a version and expect changes.
A control plane also gives you a fair way to compare stacks. Run the same tasks through two runtimes and read cost and failure counts side by side. The framework debate turns into numbers from your own work.
In practice, you can change frameworks without changing how you operate.
- Tasks move through inbox, assignment, execution, and review on one board.
- Token use and cost views show spend per task across runtimes.
- A review step needs an approval record before a task can reach done.
- Roles, API keys, and audits sit in one place.
Our own agent-ops write-up shows the pattern. Read it next to the README, so you can see the board and the habits around it.
For a non-agent reference, the historical B-Pay case study covers a payments, team management, and invoicing platform built on Solana. Whatever does the work, a system that moves money needs an owner and a trail.
FAQ
Is LangGraph better than CrewAI?
It's better when you need durable state, pauses, and resumes. The crew framework is faster to start when the work splits into clear roles. Run both against the same test set before you commit.
Can I use a graph runtime without the rest of its ecosystem?
Yes, the graph option's docs say it works on its own. You define the graph and state yourself, and bring whatever model client you like. Tracing and hosting are separate choices you can make later.
When should I build a custom agent loop instead of a framework?
Build your own loop when one or two agents do a narrow job and you want every line visible. Add a step cap, a budget, an allowlist, checkpoints, and an escalation path from day one. If you find yourself rebuilding a graph runtime, switch to a framework. Keep your tools in plain modules, so the move costs days, not weeks.
How many agents should a multi-agent system start with?
Start with one. Add a second only when it beats the single agent on the same test set, and give each role its own tools and checks. Roles that share everything are one agent with extra cost.
Do I need a control plane for multi-agent systems?
You need one once more than one person depends on the agents' output. A control plane gives you tasks, approvals, cost, and audit in one place. Mission Control is our open-source take on it.
Pick the stack with us
Our multi-agent orchestration work starts with the smallest version that could work. Our AI agent development work builds agents around the five guards above.
Try the five questions on your workflow first. Then book a call and bring your answers.
We'll come back with a stack call and the guards it needs.
