Every organization is asking the same question right now: how do we unlock the benefits of AI on our most sensitive data? The answers so far are mostly contractual: terms of service, acceptable use policies, ethical guidelines and commitments from vendors about what they will not do with your data. That is not enough.

The Anthropic conflict with the Department of Defense (DoD), in which the agency demanded unrestricted use of the Claude AI model for the military, recently crystallized the challenge the industry has been facing. When a company's privacy commitments meet sufficient external pressure, whether from a government, a regulator, an investor or a potential acquirer, they become a negotiating position. 

Privacy as a human right cannot rest on the goodwill of whoever happens to be running the system. It needs to be built into the system itself.

The Gap Between the Promise and the Architecture

"Privacy by design" is now standard marketing language. Almost every AI vendor uses it, but few define it precisely enough to mean anything.

To hold credibility, a true privacy-by-design AI system is one where breaking a user's privacy should be architecturally impossible. The distinction matters because it removes risk: customers do not have to evaluate the vendor's intentions, the robustness of their legal terms, or their ability to resist external pressure. 

In practice, a traditional AI architecture has three distinct exposure surfaces: the training pipeline, where models learn from data; the retraining pipeline, where models are updated as data drifts; and inference, where the model processes real user input to generate predictions. Most current approaches protect none of these adequately. Data is accessible to providers and cloud operators during training, inference inputs are processed in plaintext in server memory, and every retraining cycle creates a new window during which sensitive data is exposed.

The standard the industry should hold itself to is not "we protect your data." It is "we built a system where we cannot see your data, even if we wanted to."

A Practical Approach to Privacy-Preserving AI

Training Without Exposing Customer Data

Every model starts somewhere. The deliberate choice AI vendors need to make is to initialize their model on non-sensitive, public data, and initialize the system on a dataset composed exclusively of non-sensitive data: publicly available content, web-crawled sources or synthetic data with no personally identifiable information.

Because this initial dataset carries no privacy exposure, standard machine learning infrastructure can be sufficient. The resulting model can be deployed into production, generating predictions against real-world data, and this gives you your baseline.

The hard problem begins there. AI models drift over time as the data they encounter changes, and retraining requires fresh, real-world user data. Each retraining cycle is a new privacy exposure event that needs to be carefully considered.

One solution is the use of confidential computing: hardware-attested secure enclaves as the training boundary. A secure enclave is a hardware-backed trusted execution environment with isolated memory and CPU, cryptographically separated from the parent server, the cloud provider, the operating system and even from the administrators managing the infrastructure. Data is encrypted end-to-end and decrypted exclusively within that isolated memory.

Extending the Guarantee Through to Inference

Training is half the problem. Inference, the point at which the model processes real user data to generate predictions, introduces its own privacy exposure surface, and one that most privacy discussions underweight. 

The architecture pattern also needs to address this. A hybrid deployment model supporting two distinct inference modes can best address this: on-device execution, where the model runs locally on the client device, and enclave-based remote inference, where the model is instantiated within a verified secure enclave on remote infrastructure.

Each mode provides strong privacy guarantees through different mechanisms. The choice can be made between them based on model size, device capability, network connectivity and latency requirements. Together, they ensure that the confidentiality guarantees established during training extend through to inference, completing the end-to-end privacy boundary.

On-device inference is the default wherever device capability and model size allow: the model runs locally on the client device, no data is transmitted and no external party has any exposure to inputs or outputs. This is not a performance optimization. It is a privacy architecture decision.

When the model is too large or the device too constrained, inference can be routed to an enclave-based remote service. The client encrypts input data before transmission. It travels through untrusted infrastructure as encrypted bytes the server operator cannot read. Inside the verified enclave, data is decrypted in isolated memory, inference executes, and predictions are re-encrypted before leaving the enclave boundary. The client decrypts on receipt. At no point in that chain does any external party see plaintext data.

Together, with training data protected in the enclave and inference data never exposed outside verified boundaries, this closes the loop across all three exposure surfaces. That is what end-to-end privacy in an AI system actually means.

What Technical Leaders Should Ask

If you are deploying AI on sensitive data, or evaluating vendors who are, the right question to ask is not "how do you protect our data?" It is "can you expose our data?"

The architecture should make the answer to the second question structurally “no.”

There are a few more questions security leaders should ask:

• Is continuous learning treated as a privacy event? Every retraining cycle is a new data collection. If your vendor has not designed that pipeline with the same rigor as initial data ingestion, the gap is there whether or not anyone is looking at it.
• What is the inference architecture? On-device execution eliminates an entire class of privacy risk. For enclave-based remote inference, ask for specifics on attestation, key management, and what the cloud operator can and cannot see.
• Where is the trade-off documented? A system that claims perfect privacy does not exist. The honest answer involves trade-offs. Ask for them in writing.
• How does your architecture support regulatory intent? The EU AI Act and similar frameworks are pushing toward demonstrable data protection guarantees. Cryptographic proof of confidentiality is precisely what regulators are trying to express in legal language. Building the architecture now is not about anticipating regulation: it is about encoding its intent into the system before the law catches up.

The Crossroads

AI is moving faster than the norms governing it. The decisions being made now, by the companies building and deploying AI today, will define what is considered acceptable for the next decade.

The divergence is already visible. OpenAI moved toward expanded government partnership with contractual safeguards. Anthropic drew architectural red lines in its recent conflict with the DoD and paid for them commercially. Neither position is settled. The pressures that forced that conflict will not go away; they will intensify as AI becomes more deeply embedded in sensitive infrastructure.

If the industry normalizes the idea that accessing sensitive data is the price of AI capability, we will have traded a foundational right for convenience. That is a bad trade. And unlike most bad trades, it may not be reversible once the infrastructure is built and the norms are set.

Architecture as Commitment

Privacy in an AI world requires what privacy has always required: transparency, meaningful consent, and the structural inability of those with power over your data to abuse it.

Architecture is how you encode those principles into software. It is how you make a human right enforceable rather than aspirational. A cryptographic guarantee does not depend on who is in the room, who wins the next election, or what pressure the next contract negotiation brings.

The future of AI does not have to be a surveillance bargain. But that future will only exist if the people building AI decide it should, and then build systems that make the decision durable.