How to Run AI on Sensitive Data Without It Leaving Your Control

TL;DR

The common advice for using AI on sensitive data is either "redact first" or "use an enterprise tier with a BAA." Both approaches share the same underlying constraint: your data still has to leave your environment to be processed. There is a fourth option, one where the AI model receives the query logic but never sees the underlying data. The computation runs inside your controlled environment. The result comes back. Nothing sensitive reached an external server. This guide explains how each approach works, where each one falls short, and how to choose the right one for your organization.

The Scale of the Problem

Before comparing approaches, it helps to understand how widespread this problem actually is.

According to Cyberhaven's 2026 AI Adoption and Risk Report, 39.7 percent of all AI interactions involve sensitive data, including prompt text, copy-paste actions, and file uploads. On average, employees input sensitive data into AI tools once every three days. More than 60 percent of those interactions happen on personal accounts that are invisible to corporate IT.

Per CIO.com's April 2026 reporting, 49 percent of workers admit to running prompts without employer approval, and 38 percent have shared sensitive work content with consumer AI assistants.

These are not reckless employees. They are doing their jobs. The problem is structural: the tools that make AI useful require data to leave the organization, and most organizations have not found a way to close that gap without blocking AI adoption entirely.

The Samsung incident from 2023 remains the most cited example. Samsung engineers accidentally uploaded proprietary semiconductor source code and internal meeting notes to ChatGPT while trying to improve their workflow. Samsung responded by banning external AI tools company-wide. The cost of that decision, measured in lost productivity and delayed AI adoption, is harder to quantify but real.

In April 2026, Check Point Research disclosed a class of ChatGPT vulnerabilities that silently exfiltrated user prompts, uploaded PDFs, and attached medical records to attacker-controlled servers without the user seeing anything in the interface. The exposure happened not because employees were careless but because the architecture placed sensitive data in a system where the attack surface existed.

The pattern is consistent. Organizations that want to use AI on sensitive data are being asked to accept risk they cannot fully control.

The Problem With Every Answer You Have Heard

If you have asked your security or compliance team how to use AI on sensitive data, you have probably heard one of three answers.

"Just anonymize it first." Strip the identifying fields, summarize the records, replace real values with tokens. Then send the cleaned version to the model.

"Use an enterprise tier with a BAA." Enterprise-grade deployments of frontier models offer stronger contractual protections, data processing agreements, and in some cases Business Associate Agreements for HIPAA compliance.

"Run open source models locally." Deploy an open-weight model on your own infrastructure. The data never leaves because the model runs on your servers.

None of these answers fully solves the problem. They each work around a different part of it, while leaving other parts unaddressed.

The Four Approaches Compared

Approach 1: Redact Before Sending

How it works: Strip or mask sensitive fields before the data reaches the model. Send the cleaned version.

Who is doing this: Most organizations using AI on regulated data today rely on some form of redaction or pseudonymization as a first line of defense. Legal and financial services firms have built preprocessing pipelines that replace names, account numbers, and dates before data reaches AI tools.

The real-world numbers: Harmonic Security telemetry found that 16.9 percent of all enterprise sensitive-data exposures (98,034 separate instances in their dataset) happen on personal free-tier accounts where no redaction controls are in place. For organizations with formal redaction pipelines, the exposure is lower, but not zero.

Best for: Lower-stakes use cases where some data quality degradation is acceptable. Consumer-facing applications where PII removal is required regardless of the AI system.

Where it falls short for regulated industries: Anonymization degrades the data. The clinical signal you removed is often the signal that makes the AI analysis meaningful. A de-identified patient record that strips diagnosis details, treatment history, and outcomes has lost much of its value. The model gives you a worse answer because you gave it worse data.

Modern AI systems can often re-identify individuals from anonymized datasets by correlating behavioral patterns, a risk that regulators in the EU and US are increasingly addressing. Open text fields, including clinical notes, contract language, and financial commentary, are extremely difficult to redact reliably at scale.

The data still leaves your environment, which means data residency requirements may still apply even with redaction in place.

Verdict: A useful tool for reducing exposure. Not a complete compliance strategy for regulated data at enterprise scale.

Approach 2: Enterprise Tier with BAA

How it works: Use an enterprise-grade deployment of a frontier model that includes a Business Associate Agreement, data processing agreement, and zero-data-retention configuration.

Who is doing this: Microsoft Copilot for Healthcare, OpenAI's enterprise tier, and Google Cloud Healthcare AI all offer BAA pathways. Large health systems, insurers, and financial institutions are actively evaluating and deploying these tiers. In January 2026, OpenAI launched ChatGPT for Healthcare with a BAA pathway specifically for enterprise customers in clinical environments.

Best for: Organizations that need frontier AI capability and can accept third-party data processing with strong contractual controls. This is the right answer for many organizations and represents a meaningful improvement over consumer-tier tools.

Where it falls short for regulated industries: A BAA transfers contractual accountability. It does not change where your data goes during computation. With these deployments, your data leaves your environment, reaches the provider's infrastructure, and is processed there. The vendor agrees to handle it responsibly under the terms of the agreement.

Consider the following real scenario. A European bank integrated AI into its contract analysis system through a third-party AI API. The middleware stored temporary logs for debugging. Those logs were retained for 90 days. An internal audit discovered sensitive client clauses exposed in plaintext logs. The issue was not the AI provider acting in bad faith. It was that the architecture created a surface where the data existed outside the bank's direct control, and that surface expanded in ways nobody had fully mapped before deployment.

Zero-data-retention means the provider does not store your data after processing, not that they never receive it. For organizations with board-level policies that prohibit PHI, financial data, or proprietary IP from reaching external infrastructure regardless of contractual protections, this distinction matters significantly.

Verdict: The right choice for many organizations. For those with strict data residency requirements or low tolerance for third-party infrastructure risk, additional architectural controls may be needed alongside the contractual ones.

Approach 3: Local Open-Weight Models

How it works: Deploy an open-weight model such as Llama, Mistral, or Qwen on your own servers or cloud environment. The data never leaves because the model runs on your infrastructure.

Who is doing this: Eli Lilly, Honeywell, and Samsung are among the enterprises Dell Technologies cited in May 2026 as choosing local AI infrastructure deployments, running workloads on hardware they own and control rather than in the cloud. Dell's Deskside Agentic AI workstations are specifically designed for organizations that need always-on AI without sending sensitive data to external environments, and Dell reports cloud token cost reductions of up to 87 percent for organizations making this shift.

Best for: Organizations with strong engineering capabilities, tolerance for capability gaps relative to frontier models, and the infrastructure budget to support large model deployments. Particularly well-suited for organizations with very high data sensitivity requirements and the engineering resources to match.

Where it falls short for regulated industries: Local deployment requires significant upfront investment in GPU infrastructure, model deployment, and ongoing maintenance. Open-weight models vary in capability compared to frontier models on complex analytical and reasoning tasks. For most production analytics workloads, the gap is material.

Fine-tuning for domain-specific tasks adds additional complexity and cost. Most regulated enterprises do not have dedicated ML engineering teams in place, and building toward that capability takes time and budget that many organizations cannot justify for a single set of use cases.

Verdict: A viable and increasingly practical path for large enterprises with significant engineering resources. Not the right fit for most mid-market regulated organizations today.

Approach 4: Query Isolation Architecture

How it works: The AI model receives the schema and structure of your data, plus the question being asked. It generates a computation plan, the logic for how to answer. That computation runs against the actual data inside your controlled environment. The result is returned. The model never receives the underlying data values.

Who is doing this: This architectural pattern is emerging across regulated industries as organizations look for ways to use frontier AI models without the data residency and compliance risks of full cloud processing. Nera ChatApp is built on this architecture, letting data teams at healthcare, financial services, and CPG enterprises query their sensitive data using ChatGPT, Claude, or Gemini without the data reaching those models' servers.

Best for: Structured analytical workloads including financial analytics, clinical cohort analysis, operational reporting, fraud detection pattern analysis, and campaign performance measurement, where the question can be expressed as a computation over data rather than a reasoning task over text content.

Where it works and where it does not: Query isolation is optimized for structured, numerical data in databases, data warehouses, and spreadsheets. If your question can be answered by computing over rows and columns, this approach works cleanly and keeps data entirely within your environment.

It is not the right architecture for tasks that require the model to read and interpret actual document content. Summarizing contracts, analyzing clinical notes, and extracting information from free text require the model to see the content directly. Query isolation addresses analytical workloads over structured data, not document intelligence.

Verdict: The cleanest architectural answer for regulated industries running analytics workloads. No data leaves. No BAA required with the AI vendor. The compliance question is answered at the architecture level rather than the policy level.

What This Looks Like in Practice

A data analyst at a hospital system opens Nera ChatApp. They type a question in plain English: "Which patient cohorts show the highest 30-day readmission rates by procedure type and payer mix?"

Here is what happens:

1. The question is sent to a frontier AI model, ChatGPT, Claude, or Gemini, the user's choice.
2. The model receives the schema of the patient database: column names, types, and relationships, but none of the actual patient records.
3. The model generates a computation plan: the query logic, aggregation method, and output format that would answer the question.
4. That computation plan executes against the live patient database inside the hospital's controlled environment.
5. The result, charts, pivot tables, and key insights, is returned to the analyst.

The patient records never left the hospital's environment. The AI model never saw a single patient record. The analyst got a frontier AI answer in seconds rather than waiting weeks for a manual analysis.

The compliance team can answer "where does our patient data go when this runs?" with a clean, accurate answer: nowhere.

The Questions to Ask Before Choosing an Approach

1. What type of data is involved? Structured numerical data in databases, spreadsheets, and data warehouses versus unstructured text in documents, emails, and clinical notes require different approaches. Start here before evaluating tools.

2. What does your data residency requirement actually say? Some requirements prohibit data from leaving a specific geography. Others prohibit it from reaching third-party infrastructure at all. Read the requirement directly, not a summary of it.

3. What is your organization's actual risk threshold? A BAA is contractual protection. If your board's risk policy requires that sensitive data not reach external infrastructure under any circumstances, a BAA alone does not satisfy that requirement.

4. What is the business cost of the current approach? Manual workarounds, degraded analysis, and delayed decisions have real costs that rarely appear in the compliance conversation. The Samsung ban cost the company access to productivity tools their competitors were using freely. Organizations routing every analytical question through manual processes pay a cost that never shows up on a compliance dashboard.

5. What does compliant actually mean for your specific use case? HIPAA, GDPR, SOC 2, and emerging AI governance frameworks each have different requirements. Map your specific requirements to the architecture before choosing an approach.

How to Get Started

Step 1: Identify your highest-value locked use cases. Which analytical questions do your teams want to answer using AI but cannot because of data sensitivity? These are the use cases with the highest return from a query isolation approach.

Step 2: Map your actual data residency and compliance requirements. Read the specific requirements directly. Understand whether your constraint is about geography, third-party access, or both.

Step 3: Evaluate whether your data is structured or unstructured. Query isolation works best for structured analytical workloads. If your primary use case involves unstructured text, a different architecture may be more appropriate.

Step 4: Take the free AI Readiness Assessment. Nera offers a free AI Readiness Assessment at assessment.nera.systems. Voice chat with an AI, get a personalized report on where your sensitive data is exposed and what query isolation could unlock for your specific environment, then book a call with the Nera team to go over it.

Conclusion

The question is not whether to use AI on your sensitive data. The question is how to do it without accepting a tradeoff you should not have to make.

Redaction degrades your data. Enterprise agreements transfer accountability without changing the underlying architecture. Local models require infrastructure investment most organizations cannot justify for every use case. Query isolation addresses the problem at the architectural level: the model gets the question, never the data, and your organization gets frontier AI capability without data leaving your control.

For regulated industries sitting on structured analytical data they have never been able to fully use, that is not a workaround. That is the answer.

References

1. Cyberhaven. (2026). 2026 AI Adoption and Risk Report. https://www.cyberhaven.com/blog/sensitive-data-flowing-into-ai-tools
2. CIO.com. (April 2026). Workers and AI prompts: enterprise usage survey. cio.com
3. Harmonic Security. (2026). Enterprise sensitive data exposure telemetry. Covered by The Hacker News. https://thehackernews.com
4. Check Point Research. (April 2026). ChatGPT prompt exfiltration vulnerability disclosure. research.checkpoint.com
5. Dell Technologies. (May 2026). Dell Technologies World 2026: Enterprise AI Announcements. https://www.dell.com/en-us/blog/dell-technologies-world-2026-enterprise-ai-announcements-this-week/
6. IBM Security. (2025). Cost of a Data Breach Report 2025. https://www.ibm.com/reports/data-breach
7. NTT DATA. (May 14, 2026). Enterprise AI Hits the Wall: 2026 Global AI Report. https://us.nttdata.com/en/news/press-release/2026/may/enterprise-ai-hits-the-wall-ntt-data-research-reveals-growing-privacy-and-sovereignty-barriers
8. OpenAI. (January 2026). Introducing ChatGPT for Healthcare. openai.com
9. OpenAI. (April 22, 2026). Introducing OpenAI Privacy Filter. https://openai.com/index/introducing-openai-privacy-filter/