The prototype worked. The demo landed. Then the security questionnaire shows up, and question 41 asks which legal entity can be compelled to produce the contents of your prompts.

That one question stalls more AI rollouts than latency or accuracy ever have. Inside hospitals, banks, law firms, and defense subcontractors, data sovereignty AI requirements have become the gate every model decision clears before anything reaches production. What those teams want is a model they can put somewhere their lawyers can defend. Smartest on the leaderboard doesn’t enter into it.

Benchmark tables stop carrying weight the moment an auditor asks where inference happened and who could read the request body in flight. Open-weight models keep winning these reviews, and the reason has almost nothing to do with price or leaderboard position. Weights you can download are weights you can move. The model travels to the data instead of the data traveling to the model, and that inversion is the whole argument.

Compliance teams worked this out before most engineering teams did.

Sovereignty, Residency, and Localization Are Not the Same Thing

Three terms get used interchangeably in vendor decks. The difference decides whether your architecture passes.

Residency is a checkbox. Sovereignty is a jurisdiction question, and as Dust’s breakdown puts it, sovereignty follows the law of the country where data is stored and processed, not where your company is headquartered.

None of this is confined to Europe. India’s DPDPA 2023, China’s Data Security Law and PIPL, and the Australian Privacy Act 1988 each impose their own duties on cross-border data transfer, and AI21’s overview lists active localization regimes in Nigeria, Russia, Indonesia, Vietnam, Kazakhstan, South Korea, and Rwanda. Sell into those markets and “we use a US API” becomes a sentence with legal consequences attached.

Most teams find this out the hard way. They pick an EU region, ship, and get a follow-up they can’t answer.

Why the Standard Vendor Answers Do Not Survive an Audit

Four responses come back on every RFP. Each has a hole an auditor will find.

“We don’t train on your data.” That’s a promise about training. It says nothing about retention windows, prompt logging for abuse detection, subprocessor access, or how long a request body sits in a queue before it gets dropped. Training is one of maybe six things that can happen to a payload.

“You can select an EU region.” Region selection controls residency, not jurisdiction. A US-operated platform running servers in Frankfurt can still be reached by the US CLOUD Act, which Dust describes as a direct conflict with GDPR obligations, noting that a platform’s terms of service may not shield you from the resulting liability. The bytes are in Germany. The company holding the keys answers a US warrant.

“We have a DPA and we’re SOC 2.” Both are necessary and neither is sufficient. A DPA allocates liability after a breach. SOC 2 attests to controls inside a defined scope, and the scope is the part nobody reads.

“The model is private to your tenant.” Tenancy in a shared inference fleet means logical isolation on hardware you don’t control, run by staff you can’t enumerate.

None of these are lies. They answer a narrower question than the one being asked: who can be compelled, and by which court.

What a Data Sovereignty AI Review Actually Asks For

Most teams answer the questionnaire thinking about one data flow: prompt out, completion back. A serious data sovereignty AI review counts every surface where regulated content lands, and an LLM application has more of them than a CRUD service does.

Embeddings deserve extra scrutiny. Inversion research has shown that text can be reconstructed from vectors well enough to matter, so a vector database in the wrong region is a residency problem, not a cache.

Trace logs are the quiet one. An observability tool wired up during a hackathon will ship complete request bodies to a third-party SaaS in a jurisdiction nobody evaluated, and it keeps doing that for however long your retention policy runs. Map every hop, including the ones your platform team added for debugging.

Open Weights Change Who Holds the Boundary

A closed model is a service you call. An open-weight model is an artifact you possess. Possession is what makes the compliance argument work: you pick the jurisdiction instead of inheriting your vendor’s.

The old objection was quality. Buying compliance meant paying a capability tax, and for a while that held. Kimi K3 is the clearest evidence it stopped. Moonshot published the weights for a 2.8-trillion-parameter mixture-of-experts model with a 1M-token context window and native vision, and Artificial Analysis scores it 57 on the Intelligence Index, third overall and first among open-weight models, behind only Fable 5 and GPT-5.6 Sol.

DeepInfra does not host it yet and Moonshot’s own API runs $3.00 and $15.00 per million tokens, so treat K3 as a ceiling marker rather than a line item. Reasoning at that tier now ships as a file you can run in the jurisdiction your lawyers pick. DeepInfra’s comparison of open and closed models on intelligence, price, and speed has the wider picture, and a companion piece covers where closed models still lead.

Here’s what’s available today, with context windows and prices from the model pages:

Any of these runs inside a boundary you define. Closed alternatives can’t offer that at any price, which is why procurement in regulated industries now opens with licensing instead of benchmarks. Vendors sell the result as sovereign AI. The mechanism underneath the label is a license that lets you hold the weights.

The Data Sovereignty AI Deployment Spectrum

“Self-host or don’t” is a false binary, and it stalls a lot of teams. Four tiers sit between a shared public API and an air-gapped rack, each buying more control at a higher operational price. Matching the tier to your data sovereignty AI obligations is most of the work.

Tier 1 clears more reviews than engineers expect. DeepInfra runs zero data retention on bare-metal infrastructure in US data centers under SOC 2 and ISO 27001, and the data handling documentation is specific enough for a security reviewer. For a US company under HIPAA or GLBA whose auditor cares about retention and access control rather than foreign jurisdiction, that’s the end of it.

Tier 2 is where most regulated teams land. GPU Instances give you dedicated on-demand compute behind a private endpoint, which takes shared tenancy out of the threat model. If you fine-tuned on regulated data and the weights themselves are now sensitive, deploying custom LLMs on a private endpoint keeps that artifact under your control.

Tiers 3 and 4 are the self-hosted end of the range: real requirements for ITAR work, classified environments, and hospitals with in-building processing rules. They also cost headcount. Don’t climb a tier you can’t staff. An unpatched GPU cluster with a stale inference server is a worse compliance story than a documented Tier 2 endpoint.

Read the license before you commit to those tiers. Kimi K3 ships under the Kimi K3 License, not a stock MIT grant: internal deployment is unencumbered, but reselling it as a hosted service once your revenue crosses $20 million over any twelve months takes a separate agreement with Moonshot. Open weights and open source are different claims, and counsel reads the difference.

Wiring It Up Without Rewriting Your Stack

The fear is that changing the boundary means changing the code. It doesn’t. DeepInfra exposes an OpenAI-compatible Chat Completions API, so the client you already have keeps working and the boundary becomes a configuration value.

import os
from openai import OpenAI

# Tier 1 shared API, or a Tier 2 dedicated endpoint.
# Same client, same request shape, different boundary.
client = OpenAI(
    api_key=os.environ["DEEPINFRA_API_TOKEN"],
    base_url=os.environ.get(
        "INFERENCE_BASE_URL",
        "https://api.deepinfra.com/v1/openai",
    ),
)

def summarize_record(record_text: str) -> str:
    response = client.chat.completions.create(
        model="deepseek-ai/DeepSeek-V3.2",
        messages=[
            {"role": "system", "content": "Summarize the record. Do not restate identifiers."},
            {"role": "user", "content": record_text},
        ],
        max_tokens=800,
    )
    return response.choices[0].message.content
copy

Two details matter more than the SDK call. Keep `INFERENCE_BASE_URL` in configuration rather than in code, because that’s the line an auditor asks you to point at. And check what your logging middleware captures before you ship. A framework that logs the full request object writes PHI straight into your application logs, and provider-side zero data retention will not save you from your own log aggregator.

Moving from a shared endpoint to a dedicated one is an environment variable change and a redeploy. The model ID stays the same, and prompts, evals, and retry logic survive intact.

It works in the other direction too. If a customer in a new jurisdiction imposes a stricter rule, the same workload climbs a tier without touching application code. Teams on a proprietary API have no equivalent move.

Where Cost Enters the Decision

Open weights are cheaper, and it shows at volume. Take a contract review pipeline processing 50,000 documents a month at roughly 8,000 input and 800 output tokens each. That’s 400M input and 40M output tokens. On DeepSeek-V3.2 at $0.26 and $0.38 per million, the estimate lands near $119 a month. Run your own volumes before quoting it, and DeepInfra’s breakdown of what drives inference cost at scale covers the parts this arithmetic skips.

That number is a nice side effect. The deal closed for a different reason.

The buying trigger is that the workload couldn’t run at all on the closed alternative. A hospital can’t send patient records to an endpoint whose operator answers a foreign subpoena, and a $0 API wouldn’t change that. Cost enters after sovereignty is satisfied, as a tiebreak between compliant options, which inverts how most vendor comparisons get written. DeepInfra’s look at whether the price gap between open and closed models is worth it is a useful read once you’re choosing among models you’re already allowed to use. Before that point, the cheapest noncompliant model has an effective price of infinity.

Getting Started

Draw the boundary before you pick the model. Inventory every surface from the table above, name the jurisdiction each sits in, and pick the tier your obligations require, not the one that sounds safest. Most teams need less isolation than they feared and have more logging exposure than they knew.

Then pick the model that clears your quality bar at the lowest tier you can defend. DeepSeek-V3.2 handles general reasoning cheaply enough for high-volume pipelines, gpt-oss-120b covers triage and classification for a fraction of that, and Kimi-K2.6 takes long-document and agentic work. All three run with zero data retention, and the documentation has the request-handling details your security reviewer will ask for.

If your review raises a question we haven’t covered, we’d rather answer it than have you guess. Email us at feedback@deepinfra.com, join the conversation on Discord, or find us on X at @DeepInfra. Bring the questionnaire. We’ve seen most of the questions before.