MCP Gateway Architecture Patterns for Enterprise Scale

An MCP gateway is a stack of patterns. It's a stack of patterns, and picking the wrong one, or picking one when you needed three working together, is how enterprises end up with agents that can see tools they shouldn't and audit logs that don't actually cover anything. Since Anthropic released the Model Context Protocol in November 2024, adoption has moved fast, with SDK downloads approaching half a billion, the TypeScript and Python SDKs each crossing a billion total downloads, and companies including Block, Bloomberg, and Amazon among hundreds of Fortune 500 firms running it in production. The spec itself got donated to the Agentic AI Foundation under the Linux Foundation, which gives the protocol vendor-neutral governance. But that governance covers the spec, not what any given company builds to run it. And MCP, by design, leaves out rate limiting, audit logging, and access control. Those gaps don't close themselves. Gateway architecture is what closes them, or doesn't.

How the 2026-07-28 spec revision reshapes gateway architecture decisions

Anyone designing an MCP gateway today needs to build against the July 28, 2026 spec revision. It's the biggest change since MCP shipped, and the core shift is structural: the protocol dropped its stateful session model in favor of a stateless core.

That matters more than it sounds like on paper. The old model needed sticky routing or a shared session store to keep track of which client was talking to which server across requests. That's a bad fit for horizontal scaling, full stop. SEP-2575 gets rid of the initialize/initialized handshake altogether; protocol version, client info, and capabilities now travel in _meta on every single request instead of being negotiated once at the start. SEP-2567 goes further and removes the Mcp-Session-Id header, along with the whole concept of a protocol-level session. There's also a new server/discover method, so clients can pull a server's capabilities up front if they want, as an optional alternative to the initialization flow.

The practical payoff: a plain HTTP load balancer, the kind that's been sitting in infrastructure stacks for a decade, can now spread MCP traffic across a gateway cluster without sticky sessions or an external store tracking state. Requests also carry Mcp-Method and Mcp-Name headers now, so a load balancer can route on the header alone, without parsing the request body first. And the auth surface has caught up too. OAuth 2.1 with PKCE, resource indicators under RFC 8707, and Protected Resource Metadata under RFC 9728 (discoverable at /.well-known/oauth-protected-resource) are all part of the spec now. Any gateway built before this revision needs its auth layer updated, not patched around.

The centralized hub-and-spoke pattern's limits

The simplest pattern, and usually the first one enterprises reach for, is hub-and-spoke. Every agent, no matter where it's hosted, routes all MCP traffic through one central gateway cluster.

That centralization buys real ground. Agents authenticate to the gateway once, the gateway authenticates onward to each MCP server, and credentials stop being scattered across a dozen agent configs. Policy enforcement, access lists, rate limits, routing rules, all live in one place instead of being reimplemented per team. Audit logging turns from a scavenger hunt into an actual timeline, because every tool call passes through the same chokepoint. And if an agent gets compromised, revoking its access is one change at the gateway, not a hunt through every integration it ever touched.

Single gateway means single point of configuration drift, which is the catch in the name itself. Without a registry tracking every MCP server in play, an unregistered server becomes an identity the gateway has no record of, and no visibility into. Without a registry tracking every MCP server in play, an unregistered server becomes an identity the gateway has no record of, and no visibility into, and that is the default state unless someone builds the registry deliberately.

The pattern also runs into walls at scale. A single hub trying to serve agents spread across multiple clouds or regions adds latency for anyone far from the hub, and turns the hub into a single point of failure for the whole operation. Large organizations where business units run their own MCP servers tend to push back on routing everything through one central team's checkpoint too, and that resistance is organizational as much as technical. One thing the spec change does fix: before the stateless core, scaling a hub cluster horizontally meant wrestling with sticky routing. Now it doesn't.

The federation pattern: one governed tool catalog across distributed infrastructure

Federation exists for a specific kind of mess: APIs scattered across multiple API gateways, business units, cloud providers, and regions, where no agent should be expected to know how to route to each one individually.

The mechanics are straightforward once you see them. The MCP gateway presents agents with one governed tool catalog, a single endpoint, while quietly routing each request to whichever upstream gateway or MCP server actually handles it. Agents never see the distributed backend. They see one door. The gateway becomes the source of truth for what tools exist, and tool namespacing keeps two backend systems from colliding when they happen to expose a tool with the same name.

A deployment called Jarvis Registry federates across AWS AgentCore, Azure AI Foundry, Cloudflare MCP Server Portals, and self-hosted MCP and A2A servers, all behind a single MCP-compatible endpoint, confirmed in production. That's federation doing its actual job, hiding real infrastructure sprawl behind one governed surface.

Agent identity and RBAC: why "which tool" is less important than "which agent"

MCP has no native concept of role-based access control. If an agent can connect to a server, it can see every tool that server exposes, full stop. A finance agent pointed at the wrong server can see development tools. A customer support agent can end up staring at database administration endpoints it was never meant to touch.

The behavior pattern in practice resembles how people install phone apps: grab access first, worry about whether it's needed later. Nobody sits down and applies least-privilege by default, so most MCP deployments start over-provisioned and stay that way until something forces a cleanup.

Least-privilege in this context means a support agent has zero visibility into deployment tools, and a read-only analytics agent has zero access to anything that writes data, regardless of what the underlying server technically makes available. Getting there requires solving identity at two separate layers, and they're not the same problem even though they get conflated constantly.

First is tool-surface identity: what can this specific agent even see? Virtual MCPs, endpoints built per use case with a curated, narrow tool surface, are the fix here. Second is agent machine identity: who is this agent, actually, what authority does it carry, and what has it done so far? That's a different problem, solved with agent bundles carrying machine-to-machine OAuth tokens, workload identity federation, and credentials that rotate independently of any human's login. Skipping the second layer means even a perfectly scoped tool surface doesn't tell you which agent misused it.

Sidecar and in-process patterns: where latency is the governing constraint

Centralized gateways solve governance problems and create a physics problem. Every tool call routed through a remote gateway adds a network hop, and at real concurrency and throughput, those hops stack up fast.

The numbers make the gap concrete. In one independent benchmark at concurrency of 1, Bifrost added 840 microseconds of overhead, Docker added 1,134 microseconds, and IBM ContextForge added 23,058 microseconds. That's a 27x spread between the fastest and slowest option tested, a different category of system depending on which one gets picked.

Two patterns exist specifically to cut that overhead out. The in-process pattern runs gateway logic directly inside the agent's own process, eliminating the network hop. Bifrost's architecture is the clear example here: MCP servers run as long-lived subprocesses talking over stdio, and the published throughput claim is 11 microseconds of overhead at 5,000 requests per second. The sidecar pattern is the close cousin: the gateway runs as a co-located container or process sitting right next to the agent, getting latency close to in-process while keeping isolation and independent deployability, so the gateway can be updated or restarted without touching the agent itself.

Dedicated security zone and Kubernetes-native patterns for regulated industries

Finance and healthcare don't have the luxury of picking gateway architecture on latency numbers alone. Compliance requirements shape the decision as much as performance does, and two patterns recur repeatedly in those environments.

The dedicated security zone pattern isolates every MCP component, servers, databases, supporting services, inside a tightly restricted network segment, with its own firewall rules, its own monitoring, and its own IAM separate from the rest of the org's infrastructure. The upside includes clean isolation, clear boundaries for auditors, and an easier story to tell when a compliance review comes around. The cost is real too. It adds operational overhead, and done carelessly it creates an infrastructure silo that other teams have to work around instead of with. This pattern fits organizations with mature network segmentation already in place and hard data-residency rules they can't negotiate around, which in practice means finance and healthcare more than most other sectors.

The Kubernetes-native pattern takes a different angle: gateways run as Kubernetes workloads, scale horizontally the way any Kubernetes service does, and treat MCP servers as pods or remote services rather than as standalone infrastructure. It's a strong fit for multi-tenant setups where a platform team needs to govern tool access across a lot of different agent deployments at once, rather than one team owning one gateway for one use case.

Microsoft's MCP Gateway is the clearest example of this pattern done fully: a C# reverse proxy using StatefulSet-based session affinity, Azure Entra ID for authentication, built-in RBAC, and a Tool Gateway Router that handles dynamic routing to registered servers, Kubernetes-native by design, built to run as cluster workloads. Which is exactly the trade-off regulated industries are making across every pattern here: less flexibility, more governance, because in finance and healthcare, that trade almost always wins.