What Is Confidential AI? The Security Gap Most Enterprises Don't Know They Have
Author: Rami Akeela, Ph.D. — Founder, Nera Systems. Ph.D. in Electrical Engineering (Santa Clara University) and Computer Engineering (Lehigh University). Previously founded DZK (FPGA-accelerated zero-knowledge proof systems) and co-founded Fabric Cryptography.
Last updated: April 2026
Reading time: 8 minutes
Meta description: Confidential AI protects enterprise data during computation — not just at rest or in transit. Learn what it is, how it works, and why regulated industries like healthcare and finance need it now.
TL;DR
Your data is encrypted at rest. Encrypted in transit. But the moment an AI model processes it, it sits fully exposed in memory. Confidential AI closes that gap by ensuring sensitive data is never exposed to the underlying infrastructure — not to the cloud provider, not to the AI platform, not to anyone. For enterprises in healthcare, finance, or any regulated industry, this is the difference between unlocking AI's full value and leaving your most sensitive data permanently off-limits.
The Hidden Exposure Point
Every enterprise security team knows the basics: encrypt data at rest, encrypt data in transit. These are table stakes.
But here's what most security briefings skip: data must be decrypted to be processed. The moment your AI model runs inference on a patient record, a financial projection, or a trade secret, that data is exposed in plaintext in memory. Any vulnerability at the infrastructure layer — a rogue cloud administrator, a compromised hypervisor, a misconfigured API — can access it.
The numbers reflect this. IBM's 2025 Cost of a Data Breach Report found that 13% of organizations experienced breaches of AI models or applications — and of those, 97% lacked proper AI-specific controls. Healthcare breaches averaged $7.42 million per incident. Meanwhile, Cyberhaven's 2026 AI Adoption and Risk Report found that 39.7% of enterprise AI use already involves sensitive data, a figure rising as AI embeds deeper into core business functions.
The gap is not in your encryption policy. It is in the moment your AI model thinks.
What Is Confidential AI?
Confidential AI is the practice of running AI on sensitive data without that data ever leaving the organization's control — not during storage, not during transmission, and not during computation.
Traditional AI security protects the road and the destination. Confidential AI protects what happens inside the vehicle.
The key distinction: most enterprise AI tools — including "enterprise-grade" versions of ChatGPT, Copilot, and Gemini — process your data on external infrastructure. They offer contractual protections. Confidential AI provides architectural ones. The data never leaves your environment in a form anyone else can read or access.
For industries sitting on valuable but restricted data — clinical records, financial models, legal documents, proprietary research — this distinction determines whether AI is a strategic asset or a compliance liability.
How Confidential AI Works
Confidential AI is not a single technology. It is an architectural approach that combines multiple privacy-preserving techniques to match the sensitivity of the data and the requirements of the use case.
The core principle: the AI model gets the query. It never gets the data.
Here is what that looks like in practice using Nera ChatApp as an example:
- A user asks a question in plain English — "Which patient cohorts have the highest readmission rates by procedure type?"
- The query is sent to the AI model (ChatGPT, Claude, or Gemini).
- The model generates a computation plan — the logic for how to answer the question.
- That computation runs against the sensitive data entirely within the organization's controlled environment.
- The result — charts, pivot tables, summaries — is returned to the user.
The LLM never sees the underlying records. The sensitive data never touches external infrastructure. The organization gets the full intelligence of frontier AI models without any of the exposure.
The Technologies Behind Confidential AI
Confidential AI is not built on a single tool. Depending on the use case, it draws from a suite of proven cryptographic and computational techniques:
Trusted Execution Environments (TEEs): Hardware-based secure enclaves built into modern CPUs and GPUs (Intel SGX, AMD SEV, NVIDIA Confidential Computing). Data processed inside a TEE cannot be accessed by the host OS, cloud provider, or infrastructure layer. The hardware itself enforces the guarantee.
Homomorphic Encryption (HE): Allows computation directly on encrypted data without decrypting it first. Performance has improved substantially for targeted AI workloads, particularly analytics and query processing.
Secure Multi-Party Computation (SMPC): Allows multiple parties to jointly compute a result without any party seeing the other's input data — critical for cross-institution AI collaboration in healthcare and finance.
Federated Learning: Trains AI models across distributed datasets without centralizing raw data. Often combined with TEEs for stronger guarantees.
Query Isolation Architecture: An alternative approach — and Nera's primary method for LLM workloads — where the AI model receives only the query logic, never the underlying data. The computation executes inside the organization's controlled environment and only the result is returned.
These techniques are not mutually exclusive. A robust confidential AI deployment typically layers multiple approaches based on the sensitivity of the data and the nature of the workload.
Confidential AI vs. Traditional AI Security
| Security Layer | Traditional AI | Confidential AI |
|---|---|---|
| Data at rest | Encrypted | Encrypted |
| Data in transit | Encrypted | Encrypted |
| Data during computation | Exposed in memory | Protected — never leaves your environment |
| Cloud provider access | Possible | Architecturally blocked |
| Regulatory auditability | Policy-based | Verifiable |
| Works with sensitive data | Requires data movement | Data stays in place |
| Usable by regulated industries | Limited by compliance | Designed for compliance |
The critical row is computation. Every major AI vendor — OpenAI, Google, Microsoft, Anthropic — processes queries on their own infrastructure. Enterprise tiers add access controls and contractual data handling commitments. They do not prevent decryption at inference time. Confidential AI addresses the computation itself.
Why Enterprises Need It Now
Over 70% of enterprise AI workloads will involve sensitive data. Most organizations are handling this in one of three ways:
Anonymizing or masking data before it reaches the AI model. This degrades data quality and reduces model accuracy — the insights you get are a fraction of what the full data would yield.
Keeping sensitive data siloed entirely. The safest option, and the most costly. The AI use cases with the highest value — clinical decision support, fraud detection, risk modeling — stay permanently inaccessible.
Accepting the exposure risk and sending data to external AI tools anyway. This is the option most organizations quietly choose, and the one creating the most compliance and security liability.
Confidential AI offers a fourth path: use AI on sensitive data, fully and accurately, without moving it outside the organization's control boundary.
For regulated industries, this is not optional for much longer. HIPAA, GDPR, SOC 2, and emerging AI governance frameworks in the EU and US are tightening requirements around where and how sensitive data gets processed. The organizations building confidential AI infrastructure now will have a significant compliance and competitive advantage over those still choosing between options one and two.
Use Cases: Healthcare and Finance
Healthcare
Clinical AI is one of the highest-value opportunities in enterprise technology — and one of the most constrained by data privacy. Patient records, genomic data, imaging studies, and care histories are subject to HIPAA and increasingly complex state-level regulations.
Confidential AI enables:
- Cross-hospital analytics — multiple institutions running joint research on patient population data without any institution seeing another's raw records
- Clinical decision support — AI-assisted analysis using full patient histories, without de-identification that reduces clinical signal
- AI-powered diagnostics — running imaging and diagnostic models on regulated data without moving it to external cloud environments
- Regulatory reporting — AI-assisted analysis that generates a verifiable audit trail
The American Hospital Association's 2026 response to HHS identified data privacy as the primary barrier to AI adoption in clinical care. Confidential AI is purpose-built to remove that barrier.
Finance
Financial services firms hold some of the most valuable and most regulated data in the world: trading strategies, client portfolios, credit models, transaction histories. Most cannot legally or competitively afford to send this data to external AI systems.
Confidential AI enables:
- Fraud detection on live transaction data without exposing customer PII to third parties
- Risk modeling using proprietary datasets that stay within the institution's control boundary
- Cross-institution AML collaboration — detecting money laundering patterns across institutions without sharing raw customer data
- Regulatory reporting with AI-assisted analysis and a cryptographic audit trail
Common Misconceptions
"Enterprise AI tools already protect my data." Enterprise tiers add access controls and stronger contractual commitments. They do not prevent your data from being decrypted and processed on a third party's infrastructure. The distinction between contractual protection and architectural protection is the core of confidential AI.
"We just need better anonymization." Anonymization degrades the data before it reaches the model. For high-stakes use cases — clinical decision support, fraud detection, financial risk modeling — the accuracy loss is significant. Confidential AI preserves full data fidelity because the data never needs to leave your environment.
"Confidential AI requires rebuilding our infrastructure." Purpose-built solutions like Nera ChatApp and Nera SecureAnalytics are designed to deploy on existing commercial cloud infrastructure — AWS, GCP, Azure — and integrate with the AI models your teams already use. No dedicated cryptography team required.
"Access controls are enough." Access control tells you who walked in the door. It says nothing about what happens to the data once it enters the processing layer. Confidential AI addresses computation — the gap access controls cannot reach.
"It is too slow to be practical." Performance for inference and analytics workloads has improved substantially, particularly with query isolation architectures that separate the AI computation from the data access entirely. For most production analytics workloads, the overhead is negligible.
Frequently Asked Questions
What is confidential AI in simple terms? Confidential AI lets organizations run AI on sensitive data without that data ever leaving their control. The AI model answers questions about the data without seeing the data itself.
How is confidential AI different from encrypted AI? Standard encryption protects data at rest and in transit, but data must be decrypted to be processed. Confidential AI protects data during processing — the most vulnerable moment — so the data is never exposed in plaintext outside the organization's controlled environment.
Can confidential AI work with ChatGPT, Claude, or Gemini? Yes. Solutions like Nera ChatApp sit between the LLM and the data. The model receives the query and generates computation logic. The computation runs inside the organization's environment. The data never reaches the model's servers.
Is confidential AI HIPAA compliant? Confidential AI architectures are designed to maintain data within the organization's control boundary, which directly addresses HIPAA's requirements around where PHI can be processed and by whom. Organizations should verify specific implementations with their compliance teams.
What industries benefit most from confidential AI? Healthcare, financial services, pharmaceutical, insurance, legal, and government — any industry handling regulated or commercially sensitive data that has historically been unable to use AI because of where that data would need to go.
How long does it take to deploy confidential AI? With purpose-built solutions like Nera, deployment does not require rebuilding infrastructure. Integration with existing data environments and cloud infrastructure typically takes days to weeks, not months. The free AI Readiness Assessment at assessment.nera.systems helps map your specific environment before any commitment.
What data formats does confidential AI support? This depends on the implementation. Nera's current products are optimized for structured, numerical analytics — the kind of data held in databases, data warehouses, and spreadsheets. Text-based and unstructured data workloads have different requirements.
How to Get Started
Getting started with confidential AI does not require rebuilding your infrastructure.
Step 1: Identify your highest-value locked data. These are the datasets your teams most want to use AI on but cannot, due to compliance or security constraints. In healthcare, this is often patient cohort data. In finance, transaction or portfolio data.
Step 2: Map your current AI exposure. Audit which AI tools your teams are already using and what data is flowing into them. Most organizations significantly underestimate this. Cyberhaven's 2026 report found the average enterprise has more sensitive data flowing through AI tools than security teams are aware of.
Step 3: Evaluate computation-time exposure. Your current setup likely protects data at rest and in transit. Does it protect the data while the AI model is processing it? If you cannot answer that question with confidence, it is worth investigating.
Step 4: Pilot with one specific use case. A single high-value workflow is enough to validate the architecture before scaling. Start narrow: one data source, one question type, one team.
Step 5: Work with a purpose-built solution. Nera Systems helps enterprises in healthcare and finance run AI and analytics on their most sensitive data without sending it to external tools.
- Nera ChatApp — Ask questions in plain English. Get charts, pivot tables, and summaries in seconds. The LLM gets the query. It never gets the data.
- Nera SecureAnalytics — The same capability, directly inside Excel.
Both products deploy on commercial cloud infrastructure your teams already use. No infrastructure rebuild. No dedicated cryptography team.
Not sure where to start? Nera offers a free AI Readiness Assessment at assessment.nera.systems. Voice chat with an AI, get a personalized report on where your sensitive data is exposed and what confidential AI could unlock, then book a call with the Nera team to go over it together.
Conclusion
Confidential AI addresses the single most important gap in enterprise AI security: what happens to your data while it is being processed. As AI adoption accelerates across regulated industries, the organizations that figure out how to use AI on sensitive data — securely, verifiably, and without compliance exposure — will have a decisive advantage over those still keeping their best data locked away.
The technology is mature. The use cases are proven. The only question is whether your AI strategy accounts for computation-time exposure, or assumes that encryption at rest and in transit is enough.
It is not enough.
About the Author
Rami Akeela, Ph.D. is the founder of Nera Systems, a confidential AI company based in Palo Alto, CA. He previously founded DZK — the first company to build FPGA-accelerated, full-stack zero-knowledge proof systems — and co-founded Fabric Cryptography. He holds PhDs in Electrical Engineering from Santa Clara University and Computer Engineering from Lehigh University, and has spent two decades building privacy and cryptographic infrastructure that regulated industries can actually deploy.
Take the free AI Readiness Assessment at assessment.nera.systems
For healthcare-specific guidance, see our guide on HIPAA and AI