Neither yet
One AI application and one or two direct integrations, with owners who can review every connection themselves.
Based on the current stable MCP specification
MCP vs LLM gateway
An MCP gateway controls what AI applications may reach and do in your systems. An LLM gateway controls what leaves your organisation for a language model and which models may be used. An AI control plane is a name for governing both from one place. None of them decides your business permissions.
Enterprise architects, security and IT leads
You are deciding where to put controls for AI applications and agents.
A clear split of what each gateway controls and what stays with your organisation.
Two directions, two control points.
Data travelling to a model and actions taken in your systems are different risks with different owners. Put a control point where each risk enters, and keep business authorisation in the system that owns the decision.
Why the names get mixed up
Vendors and articles use "AI gateway", "MCP gateway", "LLM gateway" and "AI control plane" loosely, and sometimes for one product. For an architecture decision the question is simpler: which direction of traffic does the control sit on, what can it see, and which decision can it actually enforce.
Side by side
| Aspect | MCP gateway | LLM gateway | AI control plane |
|---|---|---|---|
| Direction | From AI applications and agents into your systems: the tools and resources that MCP servers expose. | From your applications and people out to language models: prompts, context and the responses that come back. | Both directions, governed from one place. |
| Typical controls | A catalogue of approved servers, routing, checks of the caller's token for each server, rate limits and a record of every tool call. | Approved models and providers, checks for sensitive data before it leaves, cost and usage limits, and a record of prompts and responses. | Shared policy, identity, inventory and evidence across both gateways. |
| Standard behind it | Not part of MCP. The specification defines hosts, clients and servers, and an authorisation model for servers reached over HTTP; a gateway sits in front of servers by architecture choice. | No standard. An industry pattern built on the APIs of model providers. | No standard. Mostly a product category name for a combined offer. |
| What it cannot decide | Whether this person may change this record, or which change needs a second approver. | Whether a model's answer is correct, or what an agent may do with it in your systems. | Business rules that live in your systems of record. |
TechOne position
One AI application and one or two direct integrations, with owners who can review every connection themselves.
Several teams call external models and you must control which data leaves the company, the cost, or which models are allowed.
Several AI applications or agents reach the same systems and you need one catalogue of approved servers, shared routing and a record of tool calls.
Agents read company data, send it to a model and act on the answer in your systems. This is the usual enterprise case.
Illustrative flow, not a customer case
A person or an agent asks for something in an AI application.
The LLM gateway checks what is sent to the model and which model may receive it.
The model proposes a tool call.
The MCP gateway routes the call to an approved MCP server and records it.
The MCP server and the system of record check the business permission and, where required, the approval for this exact action.
The outcome is recorded and, after a timeout or error, reconciled before anything is retried.
TechOne position
Where to go next
Next guide
Next guide
Next guide
Next guide
Primary sources
The source list distinguishes core protocol requirements, optional extensions and external implementation guidance. The surrounding enterprise control model is a TechOne architecture recommendation.
Direct answers
The mechanics are similar: routing, identity checks, rate limits and logging. An MCP gateway also understands MCP, so it knows which servers and tools exist and which calls agents made. An existing API gateway can host part of that role.
No. The specification defines hosts, clients and servers, and a client can connect to a server directly. A gateway is an architecture choice for when several hosts or servers need shared routing, policy or evidence.
Some products offer both. Architecturally they stay two control points: they see different data, protect against different risks and usually have different owners.
It can detect some patterns in what is sent to and returned from a model. It cannot know whether an action in your systems is allowed; that decision belongs to the MCP server and the system of record.
TechOne position: a platform or security team usually operates the gateways, while business owners keep the permissions and approvals in their own systems. Write both down before the first production use.
With one use case. List the systems the AI touches, the data that leaves your organisation and the actions with consequences. The control points follow from that list.