DeepInfra raises $107M Series B to scale the inference cloud — read the announcement

Starting from langchain v0.0.322 you can make efficient async generation and streaming tokens with deepinfra.
The deepinfra wrapper now supports native async calls, so you can expect more performance (no more threads per invocation) from your async pipelines.
from langchain.llms.deepinfra import DeepInfra
async def async_predict():
llm = DeepInfra(model_id="meta-llama/Llama-2-7b-chat-hf")
output = await llm.apredict("What is 2 + 2?")
print(output)
Streaming lets you receive each token of the response as it gets generated. This is indispensable in user-facing applications.
def streaming():
llm = DeepInfra(model_id="meta-llama/Llama-2-7b-chat-hf")
for chunk in llm.stream("[INST] Hello [/INST] "):
print(chunk, end='', flush=True)
print()
You can also use the asynchronous streaming API, natively implemented underneath.
async def async_streaming():
llm = DeepInfra(model_id="meta-llama/Llama-2-7b-chat-hf")
async for chunk in llm.astream("[INST] Hello [/INST] "):
print(chunk, end='', flush=True)
print()
NVIDIA Nemotron 3.5 Lightning Is Live on DeepInfra: Day-Zero Access to the Fastest Open Model for AgentsNVIDIA Nemotron 3.5 Lightning is available on DeepInfra's serverless API from day zero. It's a 30B hybrid MoE model with 3B active parameters, a 1M-token context window, and up to 4x higher throughput for high-volume agentic workloads.
DeepSeek V4.1 Flash 1017 Pricing: API Costs & Providers 2026<p>Updated November 2026 for DeepSeek-V4.1-Flash-1017. If you’re evaluating long-context reasoning models in late 2026, DeepSeek-V4.1-Flash-1017 is hard to ignore. The weights are open, the provider market is crowded, and DeepInfra’s promotional price undercuts DeepSeek’s own off-peak rate. DeepInfra now serves the 1017 release at $0.098 per million input tokens, $0.294 per million output tokens and […]</p>
Gemini 3.7 Flash: What's New, Benchmarks & API Guide<p>Gemini 3.7 Flash is Google’s Flash-tier reasoning model. It takes text, images, video, audio and PDF files as input, returns text, and has a 1,000,000-token context window on DeepInfra. Google DeepMind released it on August 13, 2026, about three weeks after Gemini 3.6 Flash, and positioned it as its workhorse model for coding and agents. […]</p>
© 2026 DeepInfra. All rights reserved.