Stateful vs Stateless MCP Server Design

MCP (Model Context Protocol) connects AI agents to outside tools, data, and prompts through one shared protocol. The spec was published in November 2024 to fix a basic math problem: without a shared standard, every agent had to be wired to every tool by hand, an N-times-M mess. With MCP, each side builds to the protocol once, so N-plus-M replaces N-times-M. Underneath that convenience sits a fork most teams don't think about until it bites them: does your server remember anything between calls, or not? Get that wrong and you hit a wall made of plumbing you didn't know you'd installed, separate from your agent's logic entirely.

A stateful server holds onto something across calls. Maybe it stores data, changes internal system state, or keeps context sitting in memory while your agent works through a task. A stateless server has none of that; every call stands alone, self-contained, usually read-only. Brave Search takes one query, returns results, and forgets you existed a second later, while the Knowledge Graph Memory server does the opposite: it keeps a graph of entities and relationships in memory and builds on what came before. I've seen teams treat this choice like an implementation detail, something you sort out in code review. In practice it shapes how your system handles concurrent users, whether it can support a multi-step workflow, and how it scales, and it decides all that before you've written a line of agent logic.

How MCP was originally built around persistent, stateful connections

MCP started life as a persistent, two-way channel. Client and server shake hands once, negotiate capabilities, and keep that channel open for the life of the session. That's a reasonable starting point for a protocol built by people who were, at first, mostly wiring up local tools to local models.

Two transports carried this at launch. Stdio runs over standard input and output streams, meant for a local process talking to another local process, usually one client per server, while HTTP with SSE was the remote option: the client sends messages over HTTP POST, and the server streams events back over Server-Sent Events, built for multi-client, remote setups.

This gave teams things they actually needed. Capability negotiation happened once, at connection time, instead of on every request, and sessions could hold context that matters: an open database transaction, a file lock, an authenticated login. Servers could push notifications when something changed, and long jobs could report progress as they went.

None of that was free. Every one of those benefits demands connection management, reconnection logic, and session recovery, work a stateless API never has to touch. For a local, single-client setup, that trade barely registers. But the trouble started the moment teams tried to run this same model at cloud scale, and a lot of them found out the hard way.

Where stateful MCP breaks down under production load

A client's session lives on one specific server instance. Standard load balancers have no built-in way to route around that unless someone sets it up by hand, and most people don't find out until it's already broken in production, usually at 2am, usually right before a demo.

Run this in a Kubernetes cluster and you hit three failure patterns, roughly in order. First, random pod routing: your first request lands on pod A, your second lands on pod B, and pod B has no idea who you are. That comes back as a 400 Session Not Found error. Fix it with sticky session affinity, and now traffic can't spread evenly across pods, so autoscaling gets worse than it should be. Then a pod restarts or crashes, taking every session it was holding down with it, and every agent mid-task on that pod gets an error thrown back at it, no warning, no graceful handoff.

There's a quieter cost underneath all this. Load balancers and API gateways have to read the full JSON-RPC payload just to figure out where a request should go, instead of routing off a plain HTTP header the way most infrastructure is built to work.

Session data itself is usually small and easy enough to carry around; the real weight comes from the long-lived connection, which breaks serverless setups and fights autoscaling at every turn. Any team running a stateful MCP server behind a standard cloud load balancer runs into one of these three failures, and not eventually, but before they reach any real scale.

How the ecosystem split: stateless servers already dominated before the spec caught up

A crawl of the Smithery registry found 1,520 servers, of which 416 had at least one passing tool. Of those, 131 were stateful and 285 were stateless, close to a 2-to-1 split favoring stateless design.

That ratio isn't an accident. Most of what agents actually do (search, lookup, format conversion, pulling a document) is naturally self-contained, and stateless design fits that work without forcing anything. The stateful minority shows up where statefulness is genuinely load-bearing: multi-step workflows, resource management, streaming jobs that need to remember something between steps.

Developers were already deciding this with their own code, long before the protocol caught up. They built stateless servers and absorbed the scaling headaches that came with the old connection model, because the alternative was worse. The spec was playing catch-up to what people had already voted for with their architecture.

What the July 2026 specification actually changed at the protocol level

The 2026-07-28 spec is the biggest revision MCP has had since launch, and the core move is simple: MCP itself becomes stateless at the protocol layer.

Two changes carry the weight here. SEP-2575 kills the old initialize/initialized handshake. Protocol version, client info, and client capabilities now ride inside _meta on every request, and a new server/discover method lets a client pull capabilities whenever it needs them. SEP-2567 goes further, removing the Mcp-Session-Id header along with the protocol-level session concept entirely.

Put it together and any request can land on any server instance. Sticky routing, shared session stores, none of it is required at the protocol layer anymore. Servers can scale down to zero when nobody's using them, which makes serverless deployment actually workable, and pod restarts, rollouts, and autoscaling events become invisible to the client: a container crashes, gets replaced, and the next request just routes to a healthy peer with no session to lose.

On release day, TypeScript, Python, Go, and C# SDKs all support 2026-07-28; Rust sits at Tier 2, still in beta. The adoption numbers back this up: across the Tier 1 SDKs, MCP pulls close to half a billion downloads a month, and the TypeScript and Python SDKs have together crossed a billion downloads total. Agentic applications can stay stateful even as the protocol underneath them no longer has to be. That's really the whole point.

Three new protocol features that make stateless operation practical rather than just possible

Three additions turn "technically stateless" into something teams can actually run without white-knuckling it.

Header-based routing, SEP-2243, requires every Streamable HTTP request to carry Mcp-Method and Mcp-Name headers. Gateways, rate limiters, and web application firewalls can route and meter traffic off those headers directly, no need to read the JSON-RPC body. There's a security payoff hiding in there too: the spec requires the header and the body to match, which closes off an attack where a load balancer routes on the header while the server executes on the body, and the two drift out of sync.

Cacheable list results, SEP-2549, gives list and resource-read responses a ttlMs and a cacheScope, modeled directly on HTTP's Cache-Control. A client knows exactly how long a tools/list response stays fresh and whether it's safe to reuse across different users, and a stable tool catalog also keeps upstream prompt caches stable across reconnects, a real line item in token cost once you're running at any scale. A long-lived SSE stream is no longer the only way to find out a list changed.

Distributed tracing, SEP-414, propagates W3C Trace Context headers (traceparent, tracestate, baggage) inside _meta with fixed key names. A trace that starts in the host app can follow a tool call through the client SDK, into the MCP server, out to whatever it calls downstream, and show up as one span tree in any OpenTelemetry backend. For governance, that's the foundation for knowing, in real time, what an agent touched and in what order, instead of piecing it together after the fact from logs that were never built to talk to each other.

When stateful application-layer design is still the right call

The spec going stateless is a transport-layer decision. It says nothing about your workflow logic, and application-level statefulness is still completely valid, sometimes the only thing that makes sense.

The mechanism that makes this work is the explicit handle. A tool call mints something, a basket_id, a browser_id, an order number, and the model carries that handle forward through the calls that follow. Hiding state inside a session object only the server can see works very differently: an explicit handle stays visible, and it can be logged, audited, checked against what actually happened.

Some jobs genuinely need this. A multi-step workflow where each step depends on the result of the one before it can't always be broken into independent requests. Some resources are expensive to spin up per call, a database connection pool, a large in-memory index, an authenticated session with an upstream system, and re-initializing them on every request wastes time and money. Streaming, pagination, and long jobs spanning several round trips need somewhere to keep their place too.

Cloud providers are building for exactly this. Cloud providers have been building managed solutions for exactly these stateful use cases, a direct answer to workflows that never went away, and never will.

The hybrid pattern most production teams end up using

Most production teams land somewhere in the middle. Tool calls stay stateless at the request level, while the server keeps a thin session layer for infrastructure only: auth tokens, connection pools, never business logic.

One common version pushes state out entirely. Session context sits in Redis or DynamoDB, keyed by a token the agent passes with each request, and every server instance reads and writes to that same store. The app tier stays stateless and scales horizontally without much trouble; the state lives somewhere else, in a tier built to scale on its own terms.

What you get is most of the operational upside of stateless (no sticky sessions, clean autoscaling, real compatibility with serverless) plus just enough memory to avoid re-initializing expensive resources on every call. The cost is honest too: that external store is now a dependency with its own availability and latency budget to manage. Not a hard problem, but not free either.

For teams whose workflows aren't purely read-only lookups but aren't tightly coupled multi-step pipelines either, this is usually where things land.

Making the design decision on purpose: a framework for matching architecture to agent behavior

One question drives the whole decision: what does the agent actually need to carry from one tool call to the next?

Go stateless if the answer is nothing, each call fully self-contained. Weather lookups, document search, unit conversion, single-turn retrieval, all of it fits cleanly here. The hybrid pattern (stateless transport with an external state tier) is the right default if the answer is a lightweight token or handle. Stateful application design is warranted if the answer is real phase-to-phase context that can't be reduced to a handle. Forcing a mismatch rarely pays off, and I've watched teams burn months finding that out the hard way.

Governance sits on top of all this. A server that buries state inside hidden session objects is hard to audit by design; explicit handles and external state tiers make what an agent touched visible and recoverable in real time, not something you piece together after an incident from scattered logs.

Get the direction wrong and both sides pay for it. Default to stateful when the job didn't need it, and you inherit sticky-session overhead, pod-crash fragility, and autoscaling friction that never had to exist. Force stateless onto a workflow that's genuinely multi-step, and that complexity doesn't disappear; it just moves into the agent's context window, carried as fragile free text instead of a managed handle.

Whichever way a team goes, the new spec's distributed tracing and header-based routing mean a well-built MCP gateway can enforce access control and give real-time visibility no matter where the state lives: in memory, in an external store, or nowhere at all. That's roughly the layer MCP Manager sits at, a control point between agents and the servers they call, watching what happens regardless of which architecture sits underneath.

Starting stateless and adding an external state tier as workflows get more complex is a far lower-risk path than starting stateful and untangling it later. Teams that make this call on purpose, based on what their agents actually need to do, skip both the scaling wall and the governance gap that come from defaulting to whatever pattern felt familiar at the time.