HIPAA and AI: What Healthcare Organizations Can and Cannot Do
Author: Rami Akeela, Ph.D., Founder of Nera Systems. Ph.D. in Electrical Engineering (Santa Clara University) and Computer Engineering (Lehigh University). Previously founded DZK (FPGA-accelerated zero-knowledge proof systems) and co-founded Fabric Cryptography. 20 years building privacy and cryptographic infrastructure for regulated industries.
Reading time: 9 minutes
TL;DR
HIPAA does not prohibit AI in healthcare. It governs how AI handles protected health information (PHI). Most commercial AI tools, including enterprise tiers of ChatGPT, Copilot, and Gemini, process your data on external infrastructure. If PHI touches that infrastructure, a Business Associate Agreement (BAA) is required, and the vendor must be able to sign one. Many cannot. The safer architecture: AI that analyzes your data without the data ever leaving your environment. The query goes out. The data stays in.
The Problem Most Healthcare AI Deployments Have
Here is the situation at most healthcare organizations right now.
A clinical team wants to use AI to analyze patient outcomes by cohort, identify readmission risk patterns, or query a decade of treatment records in plain English. The data exists. The AI model that could answer the questions exists. And somewhere between the two, a compliance officer says no.
Not because the use case is wrong. Because nobody can answer the question that actually matters: where does the patient data go when the AI processes it?
Nearly 90% of healthcare leaders identify AI as critical for improving patient access, streamlining operational efficiency, and reducing clinician burnout. Yet adoption often stalls at the pilot stage. The barrier is not technology; it is trust.
That trust gap is a HIPAA problem. And it is solvable, but only if you understand exactly what HIPAA requires of AI systems handling PHI.
What HIPAA Actually Says About AI
HIPAA does not mention AI by name. It does not need to. The regulation governs access to protected health information, and it applies regardless of what technology is performing that access.
HIPAA applies to AI tools regardless of how they are built or what model they use. The regulation governs access to PHI, not the technology performing the access. Whether a tool uses a commercial LLM or a proprietary clinical model is immaterial to an auditor.
Three HIPAA rules apply directly to AI deployments:
The Privacy Rule governs who can access PHI, for what purpose, and under what conditions. AI systems that access patient records, even to generate aggregate analytics, must comply. AI tools must be designed to access and use only the PHI strictly necessary for their purpose, even though AI models often seek comprehensive datasets to optimize performance. This is the minimum necessary standard, and it applies to every AI query touching patient data.
The Security Rule governs how electronic PHI (ePHI) must be protected. It is about to get significantly stricter. In January 2025, HHS issued its first major proposed update to the Security Rule in over a decade. The current rule says encryption of electronic PHI is an "addressable" implementation specification, meaning it was widely misunderstood to be optional. The proposed rule would finally modernize that, making encryption required with only limited exceptions. The proposed rule would also require mandatory MFA for ePHI access, network segmentation, and regular vulnerability scanning and penetration testing.
The Business Associate Rule governs third-party vendors that handle PHI on behalf of a covered entity. This is where most AI deployments run into trouble.
The BAA Requirement: Where Most AI Tools Fail
If an AI vendor's infrastructure accesses, processes, or transmits PHI, even transiently during model inference, that vendor is a business associate under HIPAA. A Business Associate Agreement is required.
If an AI vendor's infrastructure accesses, processes, or transmits PHI, even transiently as part of model inference, that constitutes a business associate function under HIPAA. A BAA is required. The 2025 amendments expand direct business associate accountability, making vendor compliance independently enforceable.
The problem: most AI tools cannot or will not sign a BAA that covers their core inference infrastructure.
No major AI coding tool signs a Business Associate Agreement. The LLM APIs behind them, OpenAI, Anthropic, and Google, do offer BAA pathways, but the tools built on top of them often do not. If your teams are pasting PHI into an AI tool during their work, and that tool does not have a BAA, that is a HIPAA violation. The tool is not a business associate. There is no contract. There is no protection.
Even where BAAs exist, they come with important limitations:
A service that is "HIPAA-eligible" has the necessary security features and the vendor is willing to sign a BAA. However, using an eligible service does not automatically make your implementation compliant. You remain responsible for proper configuration, access controls, and usage policies.
In other words: a BAA is necessary but not sufficient. The vendor signs the agreement. Your organization remains accountable for how the tool is configured and used.
What Is and Is Not Allowed Under HIPAA for AI
What is allowed
- Using AI tools that have signed a valid BAA covering their inference infrastructure
- Running AI models entirely within your own controlled environment, where PHI never leaves your custody
- Using de-identified data with AI tools, though de-identification must meet HIPAA's specific standards, not just remove obvious identifiers
- Aggregate analytics on PHI using AI, provided access controls, audit logging, and minimum necessary standards are met
- Cross-institution research collaboration using privacy-preserving architectures that do not expose raw patient records
What is not allowed
- Using consumer AI tools (standard ChatGPT, personal Google Gemini accounts) with PHI under any circumstances. Consumer ChatGPT cannot be made HIPAA compliant under any circumstances.
- Using enterprise AI tools without a valid BAA covering how PHI is processed at inference time
- Assuming that "anonymized" data is automatically HIPAA-safe. Incomplete de-identification is a common source of violations.
- Allowing employees to use AI tools with PHI without formal governance, training, and an approved BAA in place
- Using AI coding tools with real patient data during development. The tool itself is not a business associate, regardless of what model it uses.
The gray zone
The largest gray zone is enterprise-tier AI tools that offer BAAs but process data on the vendor's own infrastructure. PHI processed on the vendor's infrastructure remains subject to the vendor's security environment, even when a BAA is in place. The BAA transfers accountability contractually. It does not change where the data goes or who can technically access it during processing.
For organizations with strict data residency requirements, sovereignty concerns, or board-level risk tolerance for third-party PHI exposure, a BAA on a shared cloud infrastructure may not be sufficient, regardless of what the contract says.
The 2025 HIPAA Security Rule Proposed Changes: What Healthcare AI Leaders Need to Know
The January 2025 proposed HIPAA Security Rule update is the first round of significant updates in more than ten years since the HIPAA Omnibus Rule of 2013. It was proposed under the Biden administration and its final status under the current administration remains uncertain, but the direction of travel is clear, and organizations that build toward the proposed standards now will be better positioned regardless of the final rule.
The proposed changes most relevant to AI deployments:
Encryption becomes mandatory. Encryption of ePHI in transit and at rest would be explicitly required for all systems involving ePHI. If a device or database holds ePHI, it must be encrypted, with no exceptions. This directly impacts AI systems that cache, log, or temporarily store PHI during inference.
All security specifications become required. The proposed rules eliminate the previous distinction between "required" and "addressable" security controls, calling for the uniform implementation of all security controls to ensure consistent defense against escalating cyber threats.
Annual audits and penetration testing. Organizations will need to conduct formal risk assessments annually and penetration testing at least once per year. AI systems accessing PHI are within scope.
Technology asset inventory. Every system that touches ePHI, including AI tools, must be inventoried and mapped. Shadow AI use by clinical or operational teams creates immediate compliance gaps under this requirement.
The practical implication: healthcare organizations that have been tolerating informal AI use with patient data are facing a narrowing window to get compliant. The question is not whether to govern AI access to PHI; it is how.
The Architecture That Resolves the HIPAA-AI Tension
The core HIPAA-AI conflict is architectural: AI models need data to generate insights, but HIPAA restricts where that data can go.
Most organizations resolve this conflict in one of three ways, all of which are suboptimal:
De-identification before inference. Strip or mask PHI before the data reaches the AI model. HIPAA-safe, but it degrades the data; the clinical signal you need to answer real questions is often in the details you removed.
Full BAA with a cloud AI vendor. Legally defensible in many cases, but the data still leaves your environment. The vendor's infrastructure is processing your patient records. The BAA governs accountability; it does not change the data flow.
No AI at all. The safest option and the most costly. The use cases with the highest clinical and operational value stay permanently out of reach.
There is a fourth option that most healthcare technology leaders have not yet deployed: AI that analyzes PHI without the PHI ever leaving the organization's controlled environment.
The mechanism: the AI model receives the query, the question being asked, and returns computation logic. That logic executes against the PHI inside the organization's environment. The model never sees the underlying records. The data never leaves.
When AI runs entirely on your infrastructure and PHI never leaves your environment, no BAA is required with an external vendor. You remain the sole custodian of patient data.
This is the architecture Nera ChatApp is built on. A clinician or analyst asks a question in plain English: "which patient cohorts show the highest 30-day readmission rates by procedure type and payer?" The LLM generates the computation plan. The computation runs inside the organization's environment against the actual patient data. Charts, pivot tables, and summaries are returned. The PHI never touched external infrastructure.
The LLM gets the query. It never gets the data.
Use Cases: What Healthcare Organizations Can Now Do
With a privacy-preserving AI architecture, the use cases that have historically been blocked by HIPAA compliance concerns become accessible:
Cohort analytics. Analyze patient populations across diagnoses, treatments, and outcomes using full clinical records, without de-identification that strips clinical signal.
Readmission risk modeling. Query historical patient data to identify risk factors and outcome patterns across large populations, in seconds rather than weeks.
Cross-facility research. Multiple hospital systems running joint analytics on combined patient populations without any institution seeing another's raw records.
Operational analytics. Staffing, capacity, throughput, and cost analytics on operational data that touches PHI, run with full fidelity inside the organization's environment.
Regulatory reporting. AI-assisted analysis for CMS reporting, quality measures, and value-based care metrics, with a verifiable audit trail of what data was accessed and how.
Frequently Asked Questions
Is using ChatGPT with patient data a HIPAA violation? Yes, if the tool does not have a valid Business Associate Agreement covering its inference infrastructure and the PHI is processed on external servers. Standard consumer ChatGPT cannot be made HIPAA compliant under any circumstances. Enterprise ChatGPT for Healthcare (launched January 2026) offers a BAA pathway for enterprise customers, but PHI is still processed on OpenAI's infrastructure.
Does a BAA make an AI tool HIPAA compliant? A BAA is necessary but not sufficient. It transfers contractual accountability to the vendor and documents their security obligations. It does not change where PHI is processed or prevent the vendor's infrastructure from technically accessing the data during inference. Your organization retains compliance responsibility for configuration and use.
Can we use de-identified data with any AI tool? Yes, if the de-identification meets HIPAA's Expert Determination or Safe Harbor standards. However, true de-identification often removes the clinical detail that makes AI analysis valuable. Re-identification risk from combining de-identified datasets is also a growing concern.
What AI tools are HIPAA compliant in 2026? ChatGPT for Healthcare (enterprise, with BAA), Azure OpenAI Service (with BAA), Google Cloud Healthcare AI (with BAA), and purpose-built solutions like Nera ChatApp that keep PHI entirely within the organization's controlled environment. The key question to ask any vendor: does your inference infrastructure process our PHI, and if so, can you sign a BAA that covers that specific data path?
What happens if we violate HIPAA with an AI tool? 93% of healthcare organizations were hit by a cyber attack during the previous 12 months, and 96% faced at least two incidents involving the loss or exfiltration of sensitive healthcare data over the past two years. HIPAA penalties range from $100 to $50,000 per violation, with annual caps of $1.9 million per violation category. Reputational damage, breach notification requirements, and OCR investigations add significant additional cost. IBM's 2025 Cost of a Data Breach Report found healthcare breaches averaged $7.42 million per incident.
Do the 2025 HIPAA Security Rule proposed changes affect AI tools? Yes. If finalized, the proposed changes would make encryption mandatory (not optional) for all ePHI systems, require annual audits of all technology assets touching PHI, including AI tools, and mandate penetration testing at least annually. AI systems used informally without governance documentation would create immediate compliance gaps.
Is there an architecture where we don't need a BAA with the AI vendor? Yes. When AI runs entirely within the organization's controlled environment and PHI never leaves, the AI vendor is not a business associate and no BAA is required. The organization retains sole custodianship of the patient data. This is the model Nera ChatApp is built on.
How to Evaluate an AI Vendor for HIPAA Compliance
Before deploying any AI tool that will touch PHI, ask these questions:
Does your inference infrastructure process our PHI? If yes, a BAA is required. If they cannot sign one that covers inference, that is a stop sign.
Where is our data processed? On your infrastructure, on their servers, or on a shared cloud environment? Data residency matters for HIPAA and for state-level regulations that are stricter than federal requirements.
What is logged, retained, and for how long? HIPAA requires audit trails. It also requires that PHI not be retained beyond what is necessary. AI systems that log prompts or cache outputs may be retaining PHI without your awareness.
Is the tool included in your technology asset inventory? Under the proposed 2025 Security Rule updates, every system touching ePHI must be inventoried. AI tools deployed without IT's knowledge create immediate gaps.
What happens to our data if there is a breach at your end? BAAs require breach notification. Understand the vendor's breach response process before signing.
How to Get Started
If your organization wants to use AI on clinical and operational data compliantly, here is the practical path:
Step 1: Map your current AI exposure. Audit which AI tools clinical, operational, and administrative teams are currently using and whether any PHI is flowing into them. Most organizations underestimate this significantly.
Step 2: Classify your use cases. Which AI use cases involve PHI? Which involve de-identified data? Which could be restructured so PHI never leaves the organization's environment?
Step 3: Evaluate your BAA coverage. For every AI tool touching PHI, confirm a valid BAA is in place and that it covers inference-time data processing, not just storage or transmission.
Step 4: Identify the cases where the BAA model is insufficient. For high-sensitivity use cases such as patient outcome analytics, cross-facility research, and clinical decision support, evaluate whether a privacy-preserving architecture that keeps PHI on-premise is the right path.
Step 5: Take the free AI Readiness Assessment. Nera offers a free AI Readiness Assessment at assessment.nera.systems. Voice chat with an AI, get a personalized report on where your sensitive data is exposed and where AI could unlock clinical and operational value, then book a call with the Nera team to go over it together.
Conclusion
HIPAA does not block AI in healthcare. It sets the conditions under which AI can safely handle patient data, and those conditions are getting stricter, not looser.
The organizations that will move fastest with clinical and operational AI are not the ones with the most risk tolerance. They are the ones that figured out an architecture where HIPAA compliance and AI capability are not in conflict.
When the AI model never sees the patient data, when the query goes out and the data stays in, the compliance question has a clean answer. And when the compliance question has a clean answer, the use cases that have been sitting in the backlog for years can finally be built.
About the Author
Rami Akeela, Ph.D. is the founder of Nera Systems, a confidential AI company based in Palo Alto, CA. He previously founded DZK, the first company to build FPGA-accelerated, full-stack zero-knowledge proof systems, and co-founded Fabric Cryptography. He holds PhDs in Electrical Engineering from Santa Clara University and Computer Engineering from Lehigh University, and has spent two decades building privacy and cryptographic infrastructure that regulated industries can actually deploy.
Take the free AI Readiness Assessment at assessment.nera.systems
For a broader overview, see What Is Confidential AI