DeepInfra runs AI agents in production, so when NVIDIA built a CPU for agents, we measured it ourselves with our own harness, our own agent, and a methodology we locked before the hardware arrived.

NVIDIA is positioning its new Vera CPU, with 88 NVIDIA-designed Olympus cores, as the CPU for agents. Agentic AI turns inference into a loop, and the CPU executes all the work around each model call: orchestration logic, parsing information, spawning sandboxed code execution, and moving data. When the CPU is slow, agents wait.

We got early access to Vera hardware and tested this claim independently, rather than relying on NVIDIA's numbers. The short version: the claims held up, and the headline claim was beaten by a wide margin.

How we measured it

We used a real agentic workload from our production environment and replayed previously captured traffic from the live agent at fixed pacing. No live LLM tokens are generated during a run, so every measured difference comes from the CPU, not model randomness.

Each agent turn carries three kinds of injected, fixed-size CPU work modeled on real agent behavior: branch-heavy orchestration logic, sandboxed code-execution spawning, and memory-bandwidth-heavy data work. Every chip ran an identical fair configuration 20 physical cores, one thread per core, single NUMA node with CPU confinement verified and automatically enforced. If a run escapes its core budget, it's discarded.

We report two runtimes everywhere: runc (near bare-metal, isolates pure silicon speed) and Kata microVMs (hardware-isolated sandboxes, how we actually deploy untrusted multi-tenant agents in production).

The field: four architectures under identical conditions. NVIDIA Vera, AMD Zen5 (Turin), Intel Granite Rapids, Intel Sapphire Rapids. A fifth machine, a DeepInfra production Xeon tuned for inference, ran as an unranked reference.

The headline: 2.2× the x86 baseline

NVIDIA's public claim is "80% faster agentic CPU performance" a 1.8× bar. To keep that from being retrofitted, we locked the comparison basis before Vera was measured: orchestration-step latency, bare-metal, per-core.

Measured: Vera 29 ms vs Intel Sapphire Rapids 64 ms, 2.2×

Vera's margin is 1.76× over the best Intel (Granite Rapids), and 1.2× over the best x86 overall (AMD Zen5). The honest framing is "fastest of all four."

Orchestration work-step latency (per core)

The mechanism is real per-core execution, not a config artifact: Vera sustained 3.62 instructions per clock on the branch-heavy workload versus 2.44 for the best measured x86 (Intel; AMD Zen5 had no IPC capture), with sub-1% cache misses exactly the "complex, branch-heavy software" regime Olympus was designed for.

Vera swept every workload category

Among the three incumbents, (AMD won orchestration, Sapphire Rapids won bandwidth), while their base task throughput clustered close together (89–97 tasks/s). Vera sets the new bar: fastest orchestration, fastest intrinsic code-execution spawn, fastest memory-bandwidth work, and highest base throughput in both bare-metal and microVM modes.

The capacity story: more agents per core at the same QoS

Raw speed matters less to an operator than how many agents a chip can host before latency degrades. For DeepInfra, this is the metric that matters: more agents per server at the same latency target. That means better utilization, lower cost per agent, and more predictable service for customers. With a strict service-level bar (99th-percentile response ≤ 6 s, zero errors), on identical 20-core partitions running secure microVMs:

• Vera sustained 256 concurrent agents; incumbents managed 160 (Granite Rapids reached 192 on its best day), 1.33–1.6× more agents per core at identical QoS.

• Vera's latency tail at 192 agents matched what incumbents showed at ~128. It degrades gracefully where others collapse.

• On a full socket, Vera held 574+ concurrent agents within the SLO and never actually broke it, the eventual ceiling was the microVM boot path, not the CPU.

• Spatial Multithreading (two threads per core) delivered +20.5% deployed throughput at high load. The second thread absorbs virtualization housekeeping.

The bonus finding: free inference on the leftover cores

Agent infrastructure rarely runs only orchestration. A useful production node also needs to keep models served, sandboxes isolated, data moving, and GPUs fed without one layer starving another.

With the 256-agent fleet running at full tilt on its partition, we put the remaining 56 cores to work serving a real LLM (gpt-oss-20b) on CPU: 118.5 tokens/s of decode under full agent contention, more than we measured on an entire 72-core NVIDIA Grace (GH200) socket doing nothing else (84.5 tok/s). The agent fleet stayed within its SLO throughout, losing under 3%.

A capstone demo ran everything at once on one Vera node. 256 secure agent microVMs, CPU inference on the leftover cores, and GPU model-serving on an installed RTX PRO 6000 with every layer within ~2–4% of its performance running alone. That's a measured version of NVIDIA's "one node runs the whole AI loop" pitch.

A finding that cuts both ways

Across every chip including Vera, the biggest tax on agent hosting isn't the CPU. It's the secure-isolation layer itself. MicroVM sandboxing costs 1.5–2× on typical work and roughly 5–13.5× on process spawning across architectures, with Arm paying about half the x86 spawn tax. Vera is the best-behaved chip under that stack, but the isolation is what makes it safe to run untrusted multi-tenant agents in production so the real opportunity for anyone hosting agents is cutting that overhead.

The self-own that proves the thesis

Our own production Xeon, the same Granite Rapids silicon as one of the ranked chips, but tuned over years to maximize inference throughput was slower at agent work than the stock cloud version of the same chip, despite higher IPC (2.63 vs 2.44). The reason: our tuning caps all-core clock at ~2.55 GHz for predictable inference serving. That's a deliberate tradeoff, not a defect that box's job is feeding GPUs, and it does that job well. Nobody asked it to run agents.

That's NVIDIA's central argument, infrastructure built for the last design target is the wrong shape for agents reproduced independently, on our own hardware, holding the microarchitecture constant. It's why we included it as an unranked reference: the workload shape matters more than the silicon.

Why we published this

We benchmark the infrastructure we run agents on because our customers run agents on it. Vera is built for exactly the workload we operate at scale, and the numbers back up the design. As Vera Rubin platforms become available, this is the CPU foundation we're planning around.

• Explore models and infrastructure: deepinfra.com