DeepInfra raises $107M Series B to scale the inference cloud — read the announcement

The Enterprise AI Security Checklist: What to Verify Before Your Data Reaches an Inference Provider - Deep Infra

Published on 2026.11.16 by DeepInfra
The Enterprise AI Security Checklist: What to Verify Before Your Data Reaches an Inference Provider - Deep Infra

Most enterprise AI security reviews do not fail on the headline items. Nobody is surprised that a vendor needs encryption or a SOC 2 report. They fail on questions nobody thought to ask until the contract was nearly signed: who owns the servers, which companies sit between you and the GPU, and where a prompt goes when it is not being answered in real time.

This checklist follows the prompt. A request passes through seven hops, shown in Figure 1, and control changes hands at nearly every one. Walking the path hop by hop means nothing falls into the gap between two teams’ responsibilities. It builds on our guide to what enterprise buyers should look for in AI inference and ends with a 25-question checklist you can send to vendors.

Figure 1. The seven hops a prompt passes through, from your application through the network, the provider's gateway, routing, the GPU, caches and storage, to logs and subprocessors, each mapped to its checklist area

The seven areas at a glance

AreaWhat to verifyWhy it matters
Infrastructure ownershipWho runs the GPUs, in which countries, and whether any requests are forwarded to other companiesEvery extra party is another contract, jurisdiction and breach surface
SubprocessorsThe public list, which entries touch content, and how changes are announcedYour data flows through every entry on the list
Data lifecycleRetention by API path, caches, logs and deletionZero retention is not uniform across paths
Identity, access, encryptionSSO, scoped keys, staff access and encryption wherever data is storedLimits the blast radius and speeds up incident response
Supply chainWeights provenance, file format and version pinningThe model is software that arrives from somewhere
Your sidePrompt injection, agent permissions and your own logsNo provider can fix these for you
Incidents and exitNotification hours, audit rights and portabilityDecides how a bad day goes

Infrastructure ownership: who owns the machine your prompt runs on?

Providers describe their infrastructure in loose terms. “Our platform” can mean hardware they bought, capacity they rent or another company’s API. Each extra party is another set of employees, another contract and another jurisdiction.

How the provider runs its infrastructureWhat it meansSecurity implication
Owns the hardwareThe provider buys the GPUs and operates them in data centers it controls or leases directly.Fewer parties between you and the GPU. The provider is accountable end to end, and an audit can cover the hardware.
Rents capacityThe provider runs on a hyperscaler or a GPU marketplace under its own account.The landlord is an additional party. Tenancy, region and physical controls depend on a company you have no contract with.
Aggregates or routesThe provider forwards your request to other providers.The policy depends on which provider answers. Unless routing is pinned, you inherit the weakest terms on the route.
  • Ownership is not a control by itself. Owning hardware means fewer parties, but also full responsibility for physical security, patching and capacity. Look for evidence that an audit covers the hardware.
  • Geography. Ask for the exact list of countries where requests are processed, and get it into the contract.
  • Aggregators. The question changes from who runs the hardware to who answers each request. As we showed in our article on Chinese open-weight models, the same model can be served with very different retention and training terms.
DeepInfra specifics • Owns its GPU infrastructure across eight US data centers instead of renting from a hyperscaler. • Its dedicated cluster offering is hosted in Tier 3 data centers. • All processing happens in the United States. That simplifies things for US or flexible-residency buyers and is a limit for buyers who need EU processing today. Additional locations are planned.

Subprocessors: every company that can touch the data

A subprocessor is a third party that a provider uses to process customer data or deliver its service. A good list is public, current and specific about purpose and location. Ask four things:

  • Is a current list public?
  • Which entries can touch request content? Ask for the answer per entry, in writing.
  • Will you give advance notice of changes, and do you have a right to object?
  • Are the provider’s data protection obligations passed on to each subprocessor in equivalent terms?
DeepInfra specifics • Publishes its list in the documentation and the trust center. • Names Amazon Web Services and Google Cloud Platform for infrastructure and Stripe for payments, all in the United States. • A provider can run inference on its own hardware and still use cloud infrastructure for other parts of its service, so read the list next to the ownership claim.

The data lifecycle: retention is a property of an API path

Zero retention claims are usually accurate for the main real-time path and less simple elsewhere. Map each path you will use, including real time, batch, fine-tuning and media generation.

API pathWhat DeepInfra’s documentation saysWhat to do
Real-time inferenceInput is held in memory during processing and not stored to disk. Output is deleted after it is returned.Confirm the same language in your DPA.
Bulk (batch) inferenceData may be held longer, potentially encrypted on disk, and deleted after a short retention period.Ask what “short” means in hours.
Google and Anthropic modelsRequests are passed to the model owner, whose retention and training terms apply.Apply the stricter policy to these models.

Beyond the main path, five places deserve a direct question:

  • Caches. Confirm they are scoped to your account, held in memory and bounded in lifetime.
  • Logs. Request identifiers, token counts and timing are normal. Content should not appear.
  • Support channels. A ticket containing a pasted prompt is a copy of your data. Agree a redaction practice.
  • Training and evaluation. DeepInfra’s privacy policy says submitted data is not stored, sold or used for training without explicit consent. Make sure your DPA says the same.
  • Termination. Ask what happens to your data and fine-tuned weights when the contract ends, how long deletion takes and whether you get written confirmation.

Identity, access and encryption

It is a Tuesday morning. A developer pastes a config file into a public ticket, and an API key goes with it. What happens next depends entirely on how your keys were set up.

One shared keyOne scoped key per service
Which system leaked it?You cannot tellThe key’s name tells you
Your responseRotate everywhere at once, and every integration breaks togetherRotate one key, with no downtime
What was exposedEverything that used the keyOne service

The same idea applies to the people on your side. Single sign-on ties vendor access to your identity provider, so offboarding someone is one action instead of a scavenger hunt. DeepInfra documents Okta SSO.

The provider’s staff are the other half of the access picture. Ask who can reach production systems, who approves it and who reviews the logs afterwards. A good answer is specific and a little boring.

Encryption is the item most likely to be ticked without a second thought, so make the question specific. Ask about data in transit and at rest, wherever it is persisted. A real-time request may persist almost nothing. Batch data, fine-tuning datasets and fine-tuned weights are where stored data actually lives, so name them in the question.

On a dedicated deployment, two more questions apply: customer-managed keys and IP allowlists. Vendors and tiers answer these differently, which is why you ask.

The model and software supply chain

Teams that would never install an unreviewed package will download a model file and load it without a second look. Four common beliefs deserve a challenge.

“A model file is just data.” A pickle-based checkpoint can run code the moment it is loaded, which makes it closer to a script than a data file. Safetensors removes that attack class.

“Safetensors means the model is trustworthy.” It guarantees the file cannot execute code. It does not prove the file holds the weights you expected. That is a separate check: ask where the weights come from and verify them against the publisher’s checksums, which is the minimum.

“The model name tells me what I am running.” If a provider changes what sits behind a stable name, your application’s behavior shifts with no deployment on your side. Ask for pinned versions and written notice before a model is retired.

“Where the model was trained decides where my data goes.” As we covered in our article on Chinese open-weight models, it does not. Origin matters for output risk, and you test that with your own evaluations.

If you fine-tune, add one more question. Your weights are your intellectual property and may encode your training data, so ask how they are stored, whether they are isolated per customer, who can access them and whether you can export them.

Incident response and exit

Two questions decide how a bad day goes. How fast will you find out, and how fast can you leave?

Firstly, the law leaves a gap. Under GDPR Article 33, the controller has 72 hours to notify the supervisory authority once it is aware of a breach. The processor’s duty is to tell the controller without undue delay, and the regulation sets no number of hours. That phrase is doing a lot of work, and your contract should replace it with a figure.

Pair the clause with a named contact route and a commitment to share forensic information afterwards. Ask for independent penetration tests at least annually, with a summary on request, and for defined rights to review audit evidence.

Exit planning deserves a test of its own. If your provider had a serious incident this morning, could you be serving traffic from somewhere else by the end of the week? Because the major inference APIs are OpenAI compatible, switching is often a configuration change, but only if you are prepared. Keep the base URL in configuration and keep a second provider tested. Confirm that you can export fine-tuned weights and receive written confirmation of deletion. Then run the test once, while nothing is on fire.

The checklist

Of the 25 questions below, 13 are Critical and 12 are Important. The last two rows concern your own side of the line, because a vendor review that ignores them gives a false sense of coverage.

AreaQuestion to askWhat a good answer looks likePriority
InfrastructureWho owns and operates the GPUs that process my requests?Hardware owned or directly leased, with facilities named and covered by the audit scopeCritical
InfrastructureAre any requests forwarded to another company’s API, and for which models?A written list of proxied models with the retention and training terms for eachCritical
InfrastructureIn which countries are requests processed?A specific list, stated in the contract and not only on the websiteCritical
InfrastructureAre data center tier, redundancy and physical access controls documented?Documented, and inside the scope of the SOC 2 or ISO 27001 auditImportant
SubprocessorsIs a current subprocessor list public?Published with the purpose and location of each entryCritical
SubprocessorsWhich subprocessors can touch request content?An answer per subprocessor, in writingCritical
SubprocessorsWill you notify us of changes, and can we object?A notice period and an objection right written into the DPAImportant
SubprocessorsAre your obligations passed on to subprocessors?The DPA requires equivalent obligationsImportant
Data lifecycleIs request content written to disk on any API path?No on the real-time path, with every exception documentedCritical
Data lifecycleHow long are batch or queued requests held, and in what form?A defined period in hours or days, encryptedCritical
Data lifecycleIs any of my data used for training or evaluation?No, stated in the contract and not only in a policyCritical
Data lifecycleHow are prompt caches scoped and where do they live?Per customer, in memory, with a defined lifetimeImportant
Data lifecycleWhat operational metadata is logged?Identifiers, token counts and timing, with no contentImportant
Data lifecycleWhat happens to my data and fine-tuned weights when the contract ends?Deletion within a defined window, confirmed in writingImportant
Access and encryptionIs data encrypted in transit and at rest wherever it is stored?TLS in transit, and encryption at rest for anything persistedCritical
Access and encryptionAre SSO, scoped API keys and key rotation supported?Yes, documentedImportant
Access and encryptionWho at the provider can reach production systems, and how is it logged?Least privilege, logged and reviewedImportant
Supply chainWhere do model weights come from and how are they verified?Known sources, checksums and safe serialization formatsImportant
Supply chainAre model versions pinned, and how much notice comes before retirement?Pinned versions and a written notice periodImportant
Assurance and incidentsDo you hold a current SOC 2 Type II and ISO 27001 whose scope covers inference?Reports under NDA, with the platform inside the boundaryCritical
Assurance and incidentsHow many hours until you tell us about a security incident?A defined number of hours stated in the contractCritical
Assurance and incidentsDo independent parties run penetration tests?At least annually, with a summary shared on requestImportant
Assurance and incidentsCan we review audit evidence or audit you on request?Defined audit rights or an evidence packageImportant
Your sideDo our own logs, traces and error reporters capture prompts?Redaction in place before anything is storedCritical
Your sideDo agents and tools run with least-privilege credentials?Scoped credentials and approval steps for sensitive actionsCritical

How DeepInfra answers

A checklist is only useful if the answers can be checked. Here is what DeepInfra publishes against the main areas, and where you should confirm details yourself. For how other providers compare on retention, compliance and residency, see our roundup of the best AI inference providers for enterprise.

AreaWhat DeepInfra publishesWhat to confirm yourself
InfrastructureOwns GPU infrastructure across eight US data centers. Its dedicated cluster offering is hosted in Tier 3 facilities.The exact country list in your contract. Processing is US only today, with additional locations planned.
CertificationsSOC 2 and ISO 27001, with a public trust center.The report under NDA, and whether its scope covers the inference platform.
Data handlingStandard inference input is held in memory only. The privacy policy says submitted data is not stored, sold or used for training without explicit consent.The bulk inference retention period and the matching DPA language.
Third-party modelsGoogle and Anthropic models are forwarded to their owners.Those companies’ retention and training terms.
SubprocessorsPublic list naming AWS, Google Cloud Platform and Stripe, all in the US.Which entries can touch request content.
AccessOkta SSO is documented. Private deployments run on dedicated A100 to B300 GPUs.Network-level isolation options and key management.
ComplianceThe trust center FAQ covers DPAs, HIPAA business associate agreements and sensitive data.The current BAA answer, in writing, before sending regulated health data.

Frequently asked questions

Is a provider that owns its hardware automatically safer?

No. Fewer parties between you and the GPU helps, but the provider then carries physical security, patching and capacity itself. Look for audit evidence that covers the hardware.

Does zero data retention cover every API path?

Not necessarily. Standard real-time inference, batch inference and proxied third-party models can each have different terms, so check each path you plan to use.

What is a subprocessor, and why does the list matter?

It is a third party that handles customer data on the provider’s behalf. Your data flows through every entry, so ask which ones can touch request content and how changes are announced.
Do I need a business associate agreement to send health data?

If the data is protected health information, a BAA is generally required with vendors that handle it. Confirm the provider’s current position in writing and check with your own counsel.

How fast must a provider report a breach?

Under GDPR the processor must tell the controller without undue delay, with no fixed hours, and the controller has 72 hours to notify the authority. A contractual number of hours removes the ambiguity.

Security documentation and private deployments on DeepInfra

DeepInfra publishes its security posture, subprocessor list and data processing terms in its trust center, and its documentation covers data privacy, authentication and private deployments. Teams can start on the shared OpenAI-compatible API and move to dedicated deployments on A100 to B300 GPUs when isolation or volume requirements call for it.

Trust center: trust.deepinfra.com
Documentation: docs.deepinfra.com
Browse models and current pricing: deepinfra.com/models

Related articles
GLM-5 API Benchmarks: Latency, Throughput & CostGLM-5 API Benchmarks: Latency, Throughput & Cost<p>GLM-5 is the latest open-weights reasoning model released by Z AI (Zhipu AI) in February 2026, characterized by high &#8220;thinking token&#8221; usage. It is a Mixture of Experts (MoE) model with 744B total parameters and 40B active parameters, scaling up from GLM-4.5&#8217;s 355B parameters. The model was pre-trained on 28.5T tokens and features a 200K+ [&hellip;]</p>
GLM-5.3 API Providers: Speed, Latency & CostGLM-5.3 API Providers: Speed, Latency & Cost<p>GLM-5.3 API Review Summary GLM-5.3 is Z AI’s flagship coding and agentic reasoning model, released on August 14, 2026, and available on DeepInfra since launch. The model is built on the same ~753-billion parameter Mixture of Experts (MoE) base architecture as GLM-5.2, with all performance gains derived entirely from scaled post-training rather than architectural changes. [&hellip;]</p>
Introducing GLM-5.2 on DeepInfraIntroducing GLM-5.2 on DeepInfra<p>GLM-5.2 is Z-AI&#8217;s latest flagship model, built around one core capability: a stable, 1,048,576-token context window designed for long-horizon tasks. Most million-token context claims come with practical asterisks — degraded retrieval, inconsistent behavior at range. Z-AI describes this as the first time that scale has been delivered with reliability for sustained, long-horizon work. The coding [&hellip;]</p>