A team wires its CRM, billing, and ticket tools into an MCP server in week one. Now the model can call all of them.
It still can't answer "what's our refund policy?" That answer lives in a PDF nobody indexed.
RAG helps a model know your material. An MCP server lets it reach your tools. Below are three questions that set the order, a sequence to copy, and what breaks when you build backwards.
What Each One Solves
RAG: knowing your material
Retrieval-augmented generation finds the right passages in your sources and puts them in front of the model before it answers. The output is an answer with a link back to the document it came from. Nothing in your systems changes.
Our RAG development page lists three fit signals. Answers must use changing or private material, and users need citations. Document owners and access rules must be nameable.
MCP: reaching your tools
The Model Context Protocol specification is an open protocol for connecting LLM apps to "external data sources and tools."
Servers can offer three things. Resources are "context and data." Prompts are templates, and tools are "functions for the AI model to execute."
The spec says tools are model-controlled, so the model can "discover and invoke tools automatically." That's the power and the risk. A tool call can query a database or call an API that writes.
So the short version is this:
- RAG answers "what do we know about this?" and returns text with citations.
- MCP tools answer "what can the model do about this?" and return results from real systems.
They overlap at one point
An MCP server can expose a search over your documents as a tool or a resource. So "RAG or MCP" is partly a false choice. What matters is which problem you solve first: finding the right knowledge, or giving governed access to actions.
The Three Questions That Decide the Order
Ask these in order. The first "yes" usually tells you where to start.
1. Is the bottleneck knowing or doing?
Look at the failures you see today. If people get wrong or stale answers, or spend time hunting through docs, you have a knowledge problem. Start with RAG.
If people know the answer but spend time clicking through tools to act on it, you have an access problem. Start with tools. If both are true, start with knowledge, because tools without context make confident mistakes.
2. How many clients need the same access?
An MCP server earns its place when several clients need governed access to the same tools. That might be an IDE, a chat app, and an internal agent. Our MCP server development page says it plainly: for one stable client, a direct API integration is smaller, and we'll tell you so.
If you have one app and one model, skip the protocol layer for now. Call the API directly and keep the code small.
3. Can you name the owner and the access rules?
RAG needs document owners, freshness targets, and access rules enforced at query time. MCP needs identity, delegated permissions, and an audit trail carried through every call.
Whichever one you can govern today is the one you can ship safely. If neither is clear yet, that's scoping work, not build work.
A Typical Sequence
When a team needs both, this is the order we recommend. Each step uses what the step before it produced.
- Retrieval over the sources people trust. Index the docs, policies, and records people already check by hand. Wire citations and abstention, so the system says "I don't know" when sources don't cover it.
- A test set. Collect the questions people ask, the expected citations, and the known bad answers. Run it on every change.
- Read-only tools. Expose lookups like "get account status" or "list open tickets." Nothing changes data yet, so a bad call costs a wasted request, not a bad change.
- Retrieval behind MCP, if more than one client needs it. Put search behind a resource or tool, so every client gets the same access rules.
- Action tools, one at a time. Add each action with a confirmation step and a log, since the spec says there "SHOULD always be a human in the loop with the ability to deny tool invocations."
- An agent, if the work branches. Once tools and knowledge are both governed, an agent can combine them. Our AI agent development page covers that step.
You won't always need all six. You can stop at step two or three, and that's a fine result. A cited answer that people trust is worth shipping on its own.
Failure Modes of Building It Backwards
Tools before knowledge
The model can act, but it doesn't know your rules. It issues a refund the policy doesn't allow, or updates the wrong record because it guessed which customer you meant. Every tool call is correct to the protocol and wrong for the business.
One service account with everything enabled
Teams rush the MCP server and give it one credential with full access, so every client inherits every permission.
The spec lists what servers MUST do. They must "Validate all tool inputs" and "Implement proper access controls." They must also "Rate limit tool invocations" and "Sanitize tool outputs."
Skip those, and you've built a fast path around your own auth.
RAG over sources nobody owns
The other way round fails too. You index a shared drive full of drafts and duplicates, and retrieval surfaces the wrong version that nobody owns. Our RAG page lists unowned, duplicated, or stale material as a non-fit for this reason.
A protocol layer for one client
You build a full MCP server for a single internal app. You now maintain schemas, transports, and versioning for one caller. A direct API call would have shipped faster and been easier to change.
Cost and Ownership
Here's a planning sketch you can copy into your own doc. It puts the order decision next to who owns each piece.
# build-order.yaml: fill this in before you scope either build
bottleneck: knowing # knowing | doing | both
clients_needing_tools: 1 # MCP pays off when this is 2 or more
knowledge:
sources: [policies, help-center, contracts]
owner: "Head of Support"
freshness_target: "update within 24h of a policy change"
access_rule: "role-based, enforced at query time"
tools:
read_only: [get_account, list_tickets]
writes: [] # add one at a time, each with a confirmation step
identity: "per-user delegation, no shared admin key"
audit_log: required
first_build: rag # rag | mcp | direct-apiOwnership drives cost more than code does. RAG keeps costing you in ingestion, freshness, and test upkeep. An MCP server keeps costing you in schemas, permission changes, and the audit trail.
Our MCP page maps to the Architecture Sprint ($3K to $5K), because the schemas and permissions need to be settled before code. RAG maps to a Production Build ($10K to $40K), scored on your questions before it goes live.
Once you run agents on top of either, you need a place to dispatch work and watch it. Mission Control is our open source, self-hosted control plane for that.
The historical Tool Suite case study splits lookup tools, like snapshots, from tools that move tokens, like airdrops and royalty payments.
FAQ
Should I build an MCP server or RAG first?
Build RAG first if people need correct answers from your material. Build an MCP server first only if the knowledge is already clear and several clients need governed access to the same tools. If you have one client, a direct API call usually beats both.
Can an MCP server do RAG?
Yes, an MCP server can expose document search as a tool or a resource. That lets every connected client use the same retrieval with the same access rules. You still need the retrieval itself to be good, with owned sources and a test set.
Is MCP secure enough for actions that change data?
MCP can carry actions that change data safely, but the protocol alone doesn't make them safe. The spec says servers MUST validate inputs, enforce access controls, and rate limit tool calls. You still need per-user identity, a confirmation step for sensitive actions, and a log.
Do I need an agent to use MCP or RAG?
No, you don't need an agent for either. A chat app can use RAG to answer with citations, and a client can call MCP tools on a user's request. An agent becomes useful when the work branches and needs both knowledge and tools.
Pick Your First Build
Run the three questions with the person who owns the workflow, then fill in the planning sketch. If the answer is "knowing," start with retrieval. If it's "doing" across several clients, start with a governed MCP server.
Want a second opinion on the order? Send us the workflow, and we'll reply with a fit call.
