TechOneDigital Request an MCP architecture review

Based on the current stable MCP specification

MCP vs LLM gateway

MCP gateway, LLM gateway or AI control plane: what each one controls.

An MCP gateway controls what AI applications may reach and do in your systems. An LLM gateway controls what leaves your organisation for a language model and which models may be used. An AI control plane is a name for governing both from one place. None of them decides your business permissions.

For

Enterprise architects, security and IT leads

Use when

You are deciding where to put controls for AI applications and agents.

Outcome

A clear split of what each gateway controls and what stays with your organisation.

Technical guide by , Founder & Technology Consultant.

TechOne architecture position

Two directions, two control points.

Data travelling to a model and actions taken in your systems are different risks with different owners. Put a control point where each risk enters, and keep business authorisation in the system that owns the decision.

Why the names get mixed up

Three names, often used for the same box on a slide.

Vendors and articles use "AI gateway", "MCP gateway", "LLM gateway" and "AI control plane" loosely, and sometimes for one product. For an architecture decision the question is simpler: which direction of traffic does the control sit on, what can it see, and which decision can it actually enforce.

Side by side

What each control point sees and decides.

AspectMCP gatewayLLM gatewayAI control plane
DirectionFrom AI applications and agents into your systems: the tools and resources that MCP servers expose.From your applications and people out to language models: prompts, context and the responses that come back.Both directions, governed from one place.
Typical controlsA catalogue of approved servers, routing, checks of the caller's token for each server, rate limits and a record of every tool call.Approved models and providers, checks for sensitive data before it leaves, cost and usage limits, and a record of prompts and responses.Shared policy, identity, inventory and evidence across both gateways.
Standard behind itNot part of MCP. The specification defines hosts, clients and servers, and an authorisation model for servers reached over HTTP; a gateway sits in front of servers by architecture choice.No standard. An industry pattern built on the APIs of model providers.No standard. Mostly a product category name for a combined offer.
What it cannot decideWhether this person may change this record, or which change needs a second approver.Whether a model's answer is correct, or what an agent may do with it in your systems.Business rules that live in your systems of record.

TechOne position

When you need one, both or neither

Neither yet

One AI application and one or two direct integrations, with owners who can review every connection themselves.

LLM gateway first

Several teams call external models and you must control which data leaves the company, the cost, or which models are allowed.

MCP gateway first

Several AI applications or agents reach the same systems and you need one catalogue of approved servers, shared routing and a record of tool calls.

Both

Agents read company data, send it to a model and act on the answer in your systems. This is the usual enterprise case.

Illustrative flow, not a customer case

How the two control points work together

  1. 01

    A person or an agent asks for something in an AI application.

  2. 02

    The LLM gateway checks what is sent to the model and which model may receive it.

  3. 03

    The model proposes a tool call.

  4. 04

    The MCP gateway routes the call to an approved MCP server and records it.

  5. 05

    The MCP server and the system of record check the business permission and, where required, the approval for this exact action.

  6. 06

    The outcome is recorded and, after a timeout or error, reconciled before anything is retried.

TechOne position

What gateways handle, and what stays your decision.

The gateways handle

  • Which servers and models may be used at all.
  • Routing, identity checks and rate limits.
  • Checks of data before it leaves for a model.
  • A record of prompts, responses and tool calls.

Your organisation still decides

  • Which operations each system exposes to AI at all.
  • Who may perform which action on which record.
  • Which changes need a human approval, and from whom.
  • What evidence is kept of each action and refusal.
  • How a failed or uncertain change is detected and reversed.
  • Who owns each gateway, each MCP server and each model contract.

Where to go next

Continue with the MCP decisions.

Next guide

What is MCP

The protocol basics: roles, tools, resources and prompts, local and remote servers.

Commercial route

Decide where your AI control points belong.

Bring one use case. We map the systems the AI touches, the data that leaves your organisation and the actions with consequences, then recommend the control points, not a product.

Request an MCP architecture reviewExplore the complete MCP architecture

Primary sources

Protocol claims are grounded in official sources.

The source list distinguishes core protocol requirements, optional extensions and external implementation guidance. The surrounding enterprise control model is a TechOne architecture recommendation.

  1. Model Context Protocol specificationRoles, server and client features, and security principles; no gateway is part of the protocol.
  2. MCP architectureHosts enforce security policy and consent; each client connects to exactly one server.
  3. MCP authorizationAuthorisation for servers reached over HTTP, with access tokens issued for a specific server.
  4. Streamable HTTP transportThe HTTP transport a gateway sits on for remote servers.

Direct answers

Questions architects ask when they compare gateways.

Is an MCP gateway the same as an API gateway?

The mechanics are similar: routing, identity checks, rate limits and logging. An MCP gateway also understands MCP, so it knows which servers and tools exist and which calls agents made. An existing API gateway can host part of that role.

Does MCP require a gateway?

No. The specification defines hosts, clients and servers, and a client can connect to a server directly. A gateway is an architecture choice for when several hosts or servers need shared routing, policy or evidence.

Can one product be both an MCP gateway and an LLM gateway?

Some products offer both. Architecturally they stay two control points: they see different data, protect against different risks and usually have different owners.

Does an LLM gateway stop prompt injection?

It can detect some patterns in what is sent to and returned from a model. It cannot know whether an action in your systems is allowed; that decision belongs to the MCP server and the system of record.

Who should own each gateway?

TechOne position: a platform or security team usually operates the gateways, while business owners keep the permissions and approvals in their own systems. Write both down before the first production use.

Where should we start?

With one use case. List the systems the AI touches, the data that leaves your organisation and the actions with consequences. The control points follow from that list.