DeepInfra raises $107M Series B to scale the inference cloud — read the announcement
How DeepInfra Built on NVIDIA's Inference Stack and Why It Paid Off
When we built DeepInfra, we made a deliberate bet on the NVIDIA inference software stack. Not as a hedge — as a conviction. Today, that bet is paying off in ways that are easy to measure.
The stack
DeepInfra runs on Blackwell-generation GPUs including B300s and our inference stack is built on TensorRT-LLM and NVIDIA Dynamo for distributed serving. We use ModelOpt to quantize models to NVFP4 weights, which reduces memory and compute requirements without meaningful accuracy loss.
These aren't just checkboxes. The components of the NVIDIA inference software stack work together — NVFP4 reduces memory pressure. Dynamo handles KV-aware routing and disaggregated prefill/decode. TensorRT-LLM makes sure the kernels are actually optimized for the hardware underneath. When they work together, the result is production economics that go beyond benchmark performance. Read more about it here.
What it looks like in practice: DeepSeek V4
The clearest proof point is what happened when DeepSeek V4 dropped.
We served it in production on day 0. We launched on Hopper first, then measured performance on B300. The result: 4x better performance. A workload that previously required 4×H200 now runs on a single B300 at higher tokens per second.
"That's not a theoretical improvement. That's a direct reduction in infrastructure cost for the same production traffic."
It happened because the software stack — TensorRT-LLM, Dynamo, NVFP4 — was already in place and ready to take advantage of the hardware the moment it was available. A direct line to the NVIDIA team also helped. Getting a new frontier model deployed and serving production traffic at scale from day one requires more than good software, it requires coordination.
What this means for developers building on DeepInfra
As NVIDIA continues to optimize the inference software stack, those improvements flow through to every model on DeepInfra. Developers don't have to do anything. The same API call gets faster and cheaper as the stack compounds underneath.
That's the reason we invested early in this ecosystem and why we keep investing. The flywheel is real.
Get started
Explore the models available on DeepInfra, including DeepSeek V4 Pro on NVIDIA Blackwell, at deepinfra.com.
DeepInfra Now Serves NVIDIA Nemotron 3 Embed: Frontier Retrieval for RAG and AgentsDeepInfra now serves NVIDIA Nemotron 3 Embed, the industry's leading open embedding model for enterprise search and agentic retrieval, available today in both 8B and 1B sizes.
DeepSeek V4 Flash vs Qwen3.6 vs GLM-4.6 Benchmarks<p>A breakdown of three open-weight models across intelligence, speed, and inference cost. Three open-weight models cover most of what a developer needs from open inference right now: DeepSeek V4 Flash, Qwen3.6 35B A3B, and GLM-4.6. All three run on DeepInfra, and all three use a Mixture-of-Experts design that keeps active parameters low while total capacity […]</p>
Best API Providers for NVIDIA Nemotron 3 Super 120B<p>Nemotron 3 Super 120B is available across a growing number of hosted APIs and deployment platforms. At 120B total parameters with 12B active per inference pass, the right provider matters: latency, throughput, and cost vary significantly depending on where you run it. This guide covers the top options by use case — from fully managed […]</p>
© 2026 DeepInfra. All rights reserved.