DeepInfra raises $107M Series B to scale the inference cloud — read the announcement
Understanding LoRA inference
Concepts
- Base model: The original model that is used as a starting point.
- LoRA adapter model: A small model that is used to adapt the base model for a specific task.
- LoRA Rank: The rank of the matrix that is used to adapt the model.
What you need to inference with LoRA adapter model
- Supported base model
- LoRA adapter model hosted on HuggingFace
- HuggingFace token if the LoRA adapter model is private
- DeepInfra account
How to inference with LoRA adapter in DeepInfra
- Go to the dashboard
- Click on the 'New Deployment' button
- Click on the 'LoRA Model' tab
- Fill the form:
- LoRA model name: model name used to reference the deployment
- Hugging Face Model Name: Hugging Face model name
- Hugging Face Token: (optional) Hugging Face token if the LoRA adapter model is private
- Click on the 'Upload' button
Note: The list of supported base models is listed on the same page. If you need a base model that is not listed, please contact us at feedback@deepinfra.com
Rate limits on LoRA adapter model
Rate limit will apply on combined traffic of all LoRA adapter models with the same base model. For example, if you have 2 LoRA adapter models with the same base model, and have rate limit of 200. Those 2 LoRA adapter models combined will have rate limit of 200.
Pricing on LoRA adapter model
Pricing is 50% higher than base model.
How is LoRA adapter model speed compared to base model speed?
LoRA adapter model speed is lower than base model, because there is additional compute and memory overhead to apply the LoRA adapter. From our benchmarks, the LoRA adapter model speed is about 50-60% slower than base model.
How to make LoRA adapter model faster?
You could merge the LoRA adapter with the base model to reduce the overhead. And use custom deployment, the speed will be close to the base model.
Qwen3.5 2B via DeepInfra: Latency, Throughput & Cost<p>About Qwen3.5 2B (Reasoning) Qwen3.5 2B is a compact 2-billion parameter open-weights model released in March 2026 as part of Alibaba Cloud’s Qwen3.5 Small Model Series. It employs an Efficient Hybrid Architecture combining Gated Delta Networks (a form of linear attention) with sparse Mixture-of-Experts, delivering high-throughput inference with minimal latency overhead — a significant architectural […]</p>
Nemotron 3 Nano Explained: NVIDIA’s Efficient Small LLM and Why It Matters<p>The open-source LLM space has exploded with models competing across size, efficiency, and reasoning capability. But while frontier models dominate headlines with enormous parameter counts, a different category has quietly become essential for real-world deployment: small yet high-performance models optimized for edge devices, private on-prem systems, and cost-sensitive applications. NVIDIA’s Nemotron family brings together open […]</p>
DeepInfra Launches Access to NVIDIA Cosmos 3 World Foundation Models for Physical AIDeepInfra is serving NVIDIA Cosmos 3, the first open world foundation model for physical AI that reasons before it generates, from day zero of its release. Available as two variants—Cosmos 3 Nano and Cosmos 3 Super—these models give developers a cost-efficient foundation for building robots, autonomous vehicles, simulation workflows, and synthetic data generation at scale.
© 2026 DeepInfra. All rights reserved.