Runtime Risk Scoring Platforms for Third-Party MCP Servers in Production Agent Fleets

Third-party MCP server risk scoring and rug pull protection require continuous inspection. Production agent pipelines need controls that track tool descriptions across sessions, check outputs before the model reads them, and apply tool-level policy before actions run. Runtime scoring lets a fleet use those changing signals alongside the checks that pre-deployment audits provide.

Why third-party MCP servers create a risk category that pre-deployment audits cannot close

A conventional API caller decides what to do with a response. An MCP-connected agent instead reads tool descriptions and outputs as part of its planning context, where they can be treated as trusted instructions. A malicious server therefore does not need to exploit the caller’s code to influence what the caller does next.

That difference extends to control flow. An agent can chain tool calls, construct arguments dynamically, and trigger server-side actions that reach sensitive resources through other tools. The path takes shape as the agent runs, so only part of it can be set before deployment. A static audit can examine a server’s code and declared capabilities, but it cannot establish every path an agent will take after reading that server’s responses.

Connected servers also share the agent’s context. A single session can expose file systems, code repositories, communication tools, databases, and payment processors to the same reasoning process. Cross-server tool shadowing exploits that arrangement by injecting instructions that change how the agent understands an adjacent, previously trusted server. The compromised server need not own the targeted tool. This structural gap makes a pre-deployment scan insufficient as a production control.

What the 2026 incident record shows about how third-party MCP servers fail in production

AgentsID’s March 2026 audit of 100 MCP servers found that every vendor-maintained server that exposed tools got a failing grade. Nearly three quarters of the graded servers scored F, and none scored A. The two B-grade servers each exposed only one tool, while three servers scored C and seven scored D. Smaller tool surfaces limited exposure in that sample. The audit also identified hallucination-based vulnerabilities, or HBVs, in which vague definitions cause the model to over-privilege, misroute data, or behave unpredictably.

Dolt MCP versions 0.3.1–0.3.6 handled requests with invalid tokens and accepted attacker-generated session IDs, so database tools could run with the server’s credentials. Version 0.3.7 fixed the issue. Grafana MCP likewise accepted well-formed caller-generated session IDs, allowing callers to list and run tools using its Grafana service account. Pillar Security’s disclosure also described CVE-2026-19516, an SSRF flaw that exposed cloud metadata endpoints and received a CVSS score of 9.1. Both Grafana flaws were addressed in mcp-grafana 1.1.0.

OX Security’s “Mother of All AI Supply Chains” advisory, published April 15, 2026, documented a systemic flaw in Anthropic’s official MCP SDKs for Python, TypeScript, Java, and Rust. Command and argument values supplied through StdioServerParameters passed to subprocess execution without sanitization. The weakness sat below individual server integrations. Unlike behavioral deception, authentication and SDK flaws can have statically detectable causes, but their production consequences depend on live requests, credentials, and execution paths. Runtime scoring must therefore check for both known weaknesses and unsafe behavior that occurs only when the agent uses the tools.

What a runtime risk score for an MCP server actually measures

A runtime risk score is a composite of at least five dimensions. The first is tool definition integrity, including whether descriptions change and whether their meaning gives the model clear limits. Wording is part of the security boundary. Unbounded string parameters, missing schema validation, and vague scope boundaries leave room for HBVs, so safe code alone cannot show that the definitions are sound.

The second dimension is tool result integrity. Every return string needs a check before it enters the next model turn, because a valid read response can also contain instructions to leak data. The useful content does not cancel the attack. Per-call inspection evaluates what the server actually returned, rather than assuming a previously approved tool will keep returning safe content.

The third dimension is permission and scope posture. Shell execution, credential access, and financial tools expose different consequences from a single read-only tool. AgentsID’s scanner reflects that distinction through destructive, execution, financial, and credential risk tiers. The score therefore needs to distinguish what a server can do from whether a particular response looks suspicious.

The fourth dimension is supply chain and provenance, including package version, publisher identity, typosquat proximity, and prior CVE history. The Sandworm_Mode campaign in February 2026 used npm typosquatting to install rogue MCP servers through packages mimicking popular utilities. The fifth dimension is behavioral consistency, which checks whether what the server does live matches what it declared at registration. History is essential to that comparison. Rug pull detection requires comparing descriptions across sessions, and all five dimensions require continuous reassessment as their inputs change.

How runtime risk scoring platforms are architected to enforce decisions before damage occurs

Continuous reassessment needs an enforcement point in the execution path. For actions, that point must sit before the tool call runs. For returned content, inspection must happen before the next model turn consumes it. A log reader cannot undo an action that has already run. Scoring infrastructure therefore needs both a view of changing signals and authority to act at the relevant boundary.

An open registry supplies a reference layer. AgentSeal’s awesome-mcp-security repository applies nine security analyzers to more than 800 MCP servers, updates daily, and scans for prompt injection susceptibility, toxic flows, and attack surface breadth. Commercial platforms can build on that information, but a catalog is not itself an inline control. Enumeration is also a prerequisite. The AgentsID audit found nearly three in five sampled servers opaque to independent review, without a standardized health-check endpoint or unauthenticated tool listing.

Runtime hooks and gateways provide different enforcement positions. Pillar Security’s Runtime Guardian uses a pre-tool hook, while Palo Alto Networks’ Prisma AIRS AI Gateway sits inline across LLM calls, MCP tool calls, and agent-to-agent traffic. Fleet topology determines which position fits the deployment. Neither a server registry nor a traffic interception point, on its own, establishes coverage of every scoring dimension.

Authorization must also operate below the connection level. An agent allowed onto a database server can otherwise have access to both SELECT and DROP operations. Per-tool controls distinguish those actions. Architecture choice consequently determines which signals a platform can observe and which decisions it can enforce, from provenance at registration to suspicious content and tool use during execution.

Which signals matter most in real time and why static scanning produces false confidence

Changes in signals should come before the raw number of findings. “Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability,” a 2026 arXiv paper, documented substantial disagreement between scanners examining the same server population. It also showed why detection counts can include capabilities built to do risky work as well as hidden vulnerabilities. Counts need interpretation. A catalog pass likewise says nothing about a server changing behavior after the scan.

For production fleets, the first priority is tool description changes between tools/list refreshes. A check within a single run misses a rug pull added between sessions, so detection needs fingerprints kept for days or weeks. The next priority is tool output content before the model reads it. These two signals address different moments when a server can influence the agent’s reasoning, including through a response from a server with a clean description.

Session-cumulative outbound volume comes next, measured per server and per agent. Palo Alto Networks’ firewall telemetry recorded individual MCP sessions moving hundreds of megabytes of enterprise data outbound. Volume needs context, but it requires continuous measurement. The remaining priorities are tool arguments that deviate from declared schemas and cross-server instruction patterns that reference adjacent servers’ tools. Routing errors also become security events when an overloaded agent selects the wrong tool or endpoint for sensitive data.

Session history changes how these signals should be interpreted. Runtime Guardian tightens controls after untrusted web or MCP content enters the session. Static scanning still checks packages, capabilities, and definitions. Runtime inspection complements those checks by covering the changing descriptions, outputs, data movement, and reasoning context that a registration-time assessment cannot capture.

How to build the governance layer that makes runtime scoring operationally actionable across a fleet

To turn those signals into fleet policy, governance needs a registry, a policy engine, and observability working together. Every connected server needs an identity and a registry entry before it can be scored. Enumeration comes before policy configuration. Where tool listing requires credentials, the governance layer needs authorized visibility into that surface rather than treating an uninspected server as covered.

Policy decisions then belong at the tool level. The AgentsID Permission Specification provides constraints covering data classification, budget, chain depth, cooldown, approval gates, and risk score thresholds. Those controls let a team distinguish a permitted read from an action that needs approval. The session need not stop entirely. Approval gates can escalate a high-risk call to a person while preserving the agent’s ability to perform permitted work.

A shared taxonomy makes those decisions easier to explain and audit. The OWASP MCP Top 10 maps MCP01, token mismanagement, to authentication posture, MCP03, tool poisoning, to description integrity, and MCP04, supply chain, to provenance. Searchable, real-time, per-interaction logs then connect policy to session activity. Regulated deployments need that audit trail. Such logs support system-access audit requirements under SOC 2, HIPAA, and GDPR, while establishing compliance requires more than the logs alone.

Between-session fingerprinting must also be an ongoing fleet operation. To compare descriptions across servers and over weeks, the fleet needs to keep history as well as inspect each session. Centralized, enforced access policies make production deployments easier to approve with confidence, while “secure it later” leaves room for shadow adoption without visibility. Governance enables a safe yes by defining which actions can proceed, which require review, and which must be blocked.

How to evaluate tools for third-party MCP server risk scoring and rug pull protection

A useful evaluation starts with evidence across all five scoring dimensions. The platform needs to assess definition integrity, result integrity, permissions and scope, provenance, and behavioral consistency. A registration score covers only part of that test. Coverage also means being able to see every server’s tool surface, even when its tools/list endpoint needs credentials. An unknown surface is a coverage gap.

The evaluation should establish whether decisions happen in a pre-tool hook or gateway before execution, and whether outputs are inspected before model consumption. Description checks must run on every tools/list refresh, and their history must be kept across sessions. A cached approval cannot detect a later substitution. Session-state awareness adds another test, since a session that has ingested untrusted content should not be assessed as though nothing has changed.

Tool-level data classification, chain-depth limits, risk thresholds, and approval gates provide more useful control than server-level allow/block alone. Every interaction also needs a searchable, session-level record available in real time. Batch exports serve a different operational purpose. They do not provide the live visibility needed to govern an active session.

The named options address different parts of this framework. Pillar Security combines per-version scoring with Runtime Guardian’s pre-tool hook and session-state controls across Claude Code, Codex CLI, and Cursor. Prisma AIRS AI Gateway, generally available in July 2026, covers inline LLM, MCP, and agent-to-agent traffic, including detection of tool definition, input, and output manipulation. The deployment path distinguishes these approaches. Their fit depends on where the fleet runs and where enforcement must occur.

For vetting and reference baselines, AgentSeal’s registry and AgentsID’s open scanner and Permission Specification RFC provide resources against which commercial controls can be evaluated. They also support environments without a commercial gateway. Their role differs from live enforcement. Buyers need to know whether the tools they choose together can detect a server that changed between sessions and stop the resulting unsafe action or content before the agent goes on.