Understanding LoRA inference

Concepts

Base model: The original model that is used as a starting point.
LoRA adapter model: A small model that is used to adapt the base model for a specific task.
LoRA Rank: The rank of the matrix that is used to adapt the model.

What you need to inference with LoRA adapter model

Supported base model
LoRA adapter model hosted on HuggingFace
HuggingFace token if the LoRA adapter model is private
DeepInfra account

How to inference with LoRA adapter in DeepInfra

Go to the dashboard
Click on the 'New Deployment' button
Click on the 'LoRA Model' tab
Fill the form:
- LoRA model name: model name used to reference the deployment
- Hugging Face Model Name: Hugging Face model name
- Hugging Face Token: (optional) Hugging Face token if the LoRA adapter model is private
Click on the 'Upload' button

Note: The list of supported base models is listed on the same page. If you need a base model that is not listed, please contact us at feedback@deepinfra.com

Rate limits on LoRA adapter model

Rate limit will apply on combined traffic of all LoRA adapter models with the same base model. For example, if you have 2 LoRA adapter models with the same base model, and have rate limit of 200. Those 2 LoRA adapter models combined will have rate limit of 200.

Pricing on LoRA adapter model

Pricing is 50% higher than base model.

How is LoRA adapter model speed compared to base model speed?

LoRA adapter model speed is lower than base model, because there is additional compute and memory overhead to apply the LoRA adapter. From our benchmarks, the LoRA adapter model speed is about 50-60% slower than base model.

How to make LoRA adapter model faster?

You could merge the LoRA adapter with the base model to reduce the overhead. And use custom deployment, the speed will be close to the base model.

Related articles

Kimi K3: 2.8T Open-Weight Multimodal ModelKimi K3, developed by Moonshot AI, represents a landmark achievement in open-source artificial intelligence. As a 2.8-trillion-parameter native multimodal Mixture-of-Experts (MoE) model, Kimi K3 is engineered to handle demanding computational tasks, from complex software engineering and long-horizon agentic workflows to deep scientific research. By combining a one-million-token context window with its architectural innovations, Kimi K3 […]

Step 3.5 Flash API Benchmarks: Latency, Throughput & CostAbout Step 3.5 Flash Step 3.5 Flash is an open-weights reasoning model released in February 2026 by StepFun. It leverages a sparse Mixture of Experts (MoE) architecture with 196 billion total parameters and only 11 billion active parameters per token during inference — delivering state-of-the-art performance at a fraction of the cost of dense models. […]

vLLM vs SGLang: Performance, Features & Deployment ComparedSomebody on your team read a benchmark post, and now there’s a ticket to migrate the inference stack. That’s how most vLLM vs SGLang decisions start. A published test reports a 29 percent throughput gap, the number lands in Slack, and two weeks later you’re debugging kernel version conflicts at midnight while p99 latency sits […]

View all