HIPAA Considerations for AI Agents Accessing Health Data via MCP

HIPAA compliance for AI agents is not something to bolt on after deployment. It has to get built into the wiring of the Model Context Protocol server itself, because that server is the one place every request for patient data actually passes through.

MCP, the protocol Anthropic introduced in November 2024 and later handed to the Linux Foundation's Agentic AI Foundation in December 2025, gives AI agents a standard way to talk to outside tools and data sources. It runs on JSON-RPC 2.0, and people have taken to calling it the "USB-C for AI": one MCP server can expose a hospital's scheduling system, billing platform, or FHIR API to any compatible AI client, so nobody has to hand-build a custom integration for every agent and every system. In healthcare, that means one server sits between an AI agent and the EHR, the billing engine, the scheduling tool, sometimes all three in the same conversation.

That convenience is exactly why the old HIPAA playbook doesn't fit anymore. Agents built on MCP don't just fetch a record and display it. They reason over the data, decide on their own what to pull next, generate text that might repeat PHI nobody explicitly gave them, and chain calls across multiple systems inside a single session. HIPAA's checklist assumed deterministic, traceable data flows, a database query here, a REST call there. It did not assume a system that perceives, reasons, and acts on its own judgment in real time. The June 2025 MCP spec update formally classified MCP servers as OAuth Resource Servers, which lines up with how enterprises already manage identity, but that architecture still has to be built by someone. It doesn't come standard. And the early record isn't reassuring: the original MCP spec left authentication largely to implementers, and plenty of servers running today still lack real authorization checks or any runtime policy enforcement.

The three layers where PHI is exposed in an MCP-driven agent pipeline

Old-school data governance solved one problem: lock down who can query which table. MCP-connected agents open up PHI exposure at three separate layers that table-level locks were never built to handle.

Start with training data. When an agent gets fine-tuned on clinical notes, PHI can end up baked into the model's weights, and under the right adversarial prompting, large language models can surface training data in their outputs. A model trained on an EHR dataset without careful de-identification is a liability sitting there quietly, no matter how tight the access controls are on the original tables.

Next comes the retrieval window. In a RAG-based clinical agent, the retrieval step is what actually pulls PHI into the model's context, and that's where HIPAA's minimum necessary standard starts to matter in a very literal way. That rule says PHI access should be limited to what's needed for the task at hand. An agent booking a follow-up visit needs a name, a provider, and an open time slot. It doesn't need the full chart, but without deliberate scoping, that's often exactly what gets pulled.

Then there's the output itself. Even when retrieval gets locked down, a model can still write PHI back into its output, simply because those facts sat in its context window. A staff member cleared only for de-identified, population-level dashboards could end up reading a summary that names an actual patient.

Break the agent's loop into its three moves, perceive, reason, act, and each one carries its own exposure. Perceiving means every retrieval is a potential compliance event if access rules aren't enforced right there at the point of retrieval. Reasoning on stale data can produce a bad recommendation, a care plan the agent never saw get updated. And acting, triggering a downstream step like flagging a patient or suggesting a billing code, is itself a PHI access event that needs its own audit record, not an afterthought.

There's also a quieter problem hiding in RAG architectures: vector databases. Once patient data gets embedded for semantic search, those embeddings become copies of PHI that don't answer to the same deletion or access rules as the source record. Delete the record in the EHR, and the embedding may just sit there in the vector store, unaffected.

All of which points to one conclusion: governance has to happen at the point where context gets delivered, the MCP server, not somewhere downstream after the model has already generated a response.

How HIPAA's core rules map onto MCP deployments specifically

MCP is a protocol. The spec defines how messages get structured and sent. It says nothing about encryption, nothing about authentication, nothing about who's allowed to see what. Compliance has to get built at the server layer, by the organization running it, because the protocol itself won't do it for you.

Take the minimum necessary standard. Most AI agents are built to grab as much context as possible, because more context tends to mean better answers. That instinct is precisely what gets organizations in trouble: broad FHIR scopes and conversation memory that persists across sessions are the two most common ways agents blow past this rule without anyone deciding they should. The fix has to live in the MCP server itself. Before any data goes to the agent, the server needs to check authorization, check the data's classification, and check whether the request is actually scoped to the task, all in one pass. The stakes here aren't abstract: in a 2024 OCR enforcement action, a covered entity paid $1.19 million for failing to shut off former employees' access to ePHI and for lacking procedures to manage who could see what. That was a far simpler system than an MCP pipeline touching six data sources at once.

Audit controls under §164.312(b) require a way to record and review activity on any system holding PHI, down to the operation level. MCP tool calls often don't land in an enterprise system of record automatically, which is a real structural gap, not a minor one. A usable audit log needs, at minimum: a timestamp, a unique identifier for the action, who or what triggered it, exactly which data elements got touched, why (the task or purpose behind the call), and the outcome. Agent workflows need more on top of that: the decision points where the agent picked one action over another, which tools or APIs got called along with request and response metadata (with the PHI itself kept out of the log), and any point where a human had to approve a step. A log someone reads after something has already gone wrong is a postmortem. It's not a safety net.

None of this happens in a vacuum, either. The HIPAA Privacy Rule governs every use and disclosure of PHI. The 21st Century Cures Act requires interoperability while banning information blocking. States like California, Texas, and New York layer their own rules on top. An MCP deployment has to satisfy all of it simultaneously, not pick whichever rule is most convenient.

And there's a bigger shift coming. The proposed 2025 HIPAA Security Rule update, published January 6, 2025, closed its comment period on March 7 with over 4,000 responses, and is currently slated for finalization in July 2027. It represents a sweeping overhaul of the Security Rule, and it names AI directly as a risk vector. It would erase the old split between "required" and "addressable" safeguards, making nearly everything mandatory with only narrow exceptions. It would require organizations to maintain documented oversight of every system that touches ePHI. It would mandate encryption in transit and at rest, regular vulnerability scans, periodic penetration testing, and network segmentation. Covered entities and business associates would have to respond to security incidents within defined timeframes. Two weeks after it hit the Federal Register, a presidential executive order calling for a "Regulatory Freeze Pending Review" threw the finalization timeline into question. Even so, the direction is clear enough that organizations should be building toward it now, freeze or no freeze.

Why business associate agreements break down across MCP's tool chain

If an AI vendor's infrastructure touches, processes, or moves PHI, even briefly, during inference, that vendor is acting as a business associate under HIPAA. That means a BAA is required, full stop.

Most standard BAAs were written for a world where a business associate meant an EHR vendor, a billing company, or a cloud storage provider doing one predictable thing with data. Today's AI systems ingest, analyze, and sometimes retain PHI for training or fine-tuning, uses that fall outside what a typical BAA permits unless the contract explicitly forbids them.

Then there's the subcontractor chain. Under 45 CFR §164.308(b)(4), if a business associate brings in a subcontractor that also touches PHI, that subcontractor needs its own BAA too. In an MCP setup with several tool servers chained together, this can turn into a long chain of vendors, several links deep, that nobody has fully mapped or audited.

The current vendor landscape reflects that mess. Several major AI vendors have rolled out HIPAA-compliant BAAs built for AI workloads, but coverage isn't automatic or universal. Anthropic's BAA, for instance, does not extend universally across all its products and tiers, and organizations must confirm which services are actually covered before routing PHI through them. Google Workspace with Gemini became HIPAA-eligible under Google's BAA for Business and Enterprise tiers in late 2024, but a consumer using Gemini through a personal Google account gets no such coverage.

That exposure is compounded by the fact that The Netskope Threat Labs Healthcare 2025 report found that 88% of healthcare organizations have already brought cloud-based generative AI tools into their operations, and 96% are using tools that train on user data. That means the overwhelming majority of healthcare organizations have PHI flowing into AI systems on terms that don't restrict training, and most of them probably don't know it's happening.

The proposed 2025 Security Rule update would push this from a one-time signature into an ongoing obligation, requiring covered entities to continuously verify BAA terms and check each vendor's compliance posture. In practice, that means auditing every piece of the AI stack, the model host, the API gateway, the vector database provider, to confirm BAA coverage exists for every path PHI takes, including the MCP server and every tool it calls downstream.

The MCP-specific attack surface that makes PHI governance harder than it looks

Authentication tokens are the soft underbelly here. MCP servers typically rely on tokens to authenticate models and grant tool access, and when those tokens are long-lived, shared across services, or scoped too broadly, one leak can expose several systems at once. Worse, token misuse through MCP tends to look exactly like normal API traffic, so it slides past the login-anomaly detection that would catch a traditional account takeover. MCP servers make especially juicy targets precisely because they often hold authentication tokens for multiple services in one place.

Prompt injection and tool poisoning add a second front. Three types of adversaries exploit MCP's openness: attackers who bury malicious instructions inside data the agent is going to process anyway, attackers who distribute compromised MCP servers through the supply chain, and, less obviously, agents that turn into accidental adversaries by overstepping their role somewhere in a multi-step workflow.

These aren't theoretical. In mid-2025, Supabase's Cursor agent, running with privileged service-role access, processed support tickets that treated user-submitted text as commands. Researchers showed an attacker could slip SQL instructions into a ticket and get sensitive integration tokens leaked into a public support thread. Supabase confirmed no actual customer data got out through that path, but the mechanics combined privileged access, untrusted input, and an external channel, all in one place.

Then in April 2026, researcher Aonan Guan of Wyze Labs, working alongside Johns Hopkins researchers Zhengyu Liu and Gavin Zhong, managed to hijack Claude Code, Gemini CLI, and GitHub Copilot by hiding malicious instructions inside GitHub pull request titles, issue bodies, and issue comments. The agents read that PR data as part of their normal task context, followed the hidden instructions, and exfiltrated GitHub Actions secrets, posting the results right back as PR comments. No outside infrastructure needed. Anthropic, Google, and GitHub all paid bug bounties for it, but none published a public advisory or assigned it a formal vulnerability identifier.

What security researchers call the confused deputy vulnerability, where a system acts beyond what the user actually authorized, causes both incidents. An MCP server acts on a user's behalf, but the line between "acting for the user" and "acting beyond what the user actually authorized" blurs fast, especially once an agent starts chaining tool calls across an EHR, a billing system, and a scheduler in one session. Add in the fact that language models can summarize and reformat sensitive data on their way out the door, sometimes returning PHI in a form that doesn't look like PHI at first glance, and it's clear that output filtering has to be context-aware, going well beyond a keyword check.

The scale of this isn't limited to healthcare, either. An anomaly detection and response system deployed at Uber, described in a paper on arxiv.org, ran for ten months across more than 7,200 unique hosts, handling over 10,000 agent sessions a day, and turned up hundreds of credential exposures spanning 26 categories. That's outside healthcare, a scale far beyond it. Inside it, the same dynamics apply to data where the cost of a leak is measured in more than dollars.

What a HIPAA-compliant MCP server layer requires in practice

The MCP server has to be treated as the enforcement point, full stop. Minimum necessary, audit controls, access management, all of it needs to live there, not scattered across whichever individual AI application happens to be calling in.

Authentication and authorization come first. Every request needs to be checked twice: who's asking, and what are they actually allowed to do, before any PHI goes out the door. Role-based access control for AI agents isn't about limiting what the AI can do for its own sake. It's about making "what the AI can do" a decision someone in the organization actually made on purpose, rather than a default nobody examined. Tokens should be short-lived and narrowly scoped to the specific task at hand, not long-lived credentials passed around across every environment the agent touches. The OAuth Resource Server classification from the June 2025 MCP spec gives organizations a foundation to build on here, a starting point for identity architecture, and a natural anchor for BAA and access-control reviews.

Minimum necessary enforcement belongs inside the server too. Before context reaches the agent, the server checks authorization, checks data classification, and checks whether the request is scoped to the actual task, in a single pass. Field-level encryption, AES-GCM being one common choice, means that even when a record gets pulled, only the fields the task actually requires can be decrypted by the requesting agent. A dedicated privacy and compliance layer should sit in front of the model, trimming data down to size before the agent ever sees a response.

And logging has to be real-time and tamper-evident, not something that gets reconstructed after the fact. At minimum, each log should capture a timestamp, a unique action identifier, who or what triggered it, which data elements were touched, the purpose behind the request, and the outcome. For agents specifically, add the decision points where the agent chose between competing actions, the tools and APIs it called along with request and response metadata (again, no PHI inside the log entry itself), and any human approval steps along the way. None of this is optional dressing on top of a working system. It's the difference between an MCP deployment that can survive an OCR audit and one that's a lawsuit waiting for a trigger.