Picking an inference vendor used to be a short conversation. You wanted Llama behind an HTTP endpoint, three companies served it, and their prices sat close enough that the decision came down to whoever had capacity. The market has since split into a dozen serious operators running truly different businesses. The best open source LLM API providers now differ from each other more than the models they host.

Some own GPUs and run their own serving stacks. Some rent capacity and route your request elsewhere. Some hand you a container and a Kubernetes-shaped bill. They all describe themselves as an inference provider.

An open-weights provider hosts models whose weights are published: DeepSeek, Qwen, Llama, GLM, Kimi, MiniMax. You can read the architecture, run the same checkpoint on your own hardware, and move hosts without asking permission. A closed API sells access to weights nobody outside the vendor has seen. The difference drives everything below. When the weights are public, your provider becomes a commodity supplier competing on price, speed, and reliability. Leaving one for another costs you a single string change in your client config.

What You Are Actually Buying

Three products get sold under one “inference API” label, and confusing them is the most expensive mistake. All three can serve the same open weight models.

A token API bills per million tokens on shared, always-warm endpoints. You send a request, you get tokens, and you pay for what you consumed. Idle time costs nothing.

A dedicated endpoint reserves specific hardware for you, billed per GPU-hour or per GPU-minute whether or not traffic arrives. Rates vary more than most buyers expect. DeepInfra lists dedicated H100 capacity at $2.20 per GPU-hour and B200 at $3.69, billed in minute granularity. Baseten prices H100 at $0.10833 per minute, roughly $6.50 per wall-clock hour. Fireworks lists on-demand H100 and H200 at $7.00 per hour. Dedicated capacity wins on sustained volume, strict tail latency, and any model the shared catalog lacks.

GPU hosting goes one level lower and rents you the machine. You bring the serving stack, the scaling policy, and the pager rotation.

Serverless token pricing beats dedicated capacity until your endpoint runs hot most of the day. Measure your actual duty cycle before reserving anything.

The Best Open Source LLM API Providers at a Glance

Six companies below run their own inference on open weights, and a seventh routes traffic to the rest of them. They stop looking interchangeable the moment you line up what each one actually sells. One optimizes for model catalog depth and floor price, one for training, one for compliance, one for raw speed, one for discount rates, and one for shipping your own weights onto managed hardware.

Positioning is the easy half. The numbers are where the market stops being polite. Here is one model, gpt-oss-120b, on seven endpoints, with prices from each vendor’s own live pricing page or public API on August 12, 2026 and speed from Artificial Analysis, which measures every endpoint independently.

Run 50M input and 10M output tokens a month through each and the spread stops being academic. Roughly $3.55 on DeepInfra’s base endpoint, $5.00 on Novita, $10.00 on Baseten, and $13.50 on Groq, Together, or Fireworks. Same weights, same request, a 3.8x range. Note what that table does not show: a single winner. Three endpoints share the identical $0.15 and $0.60 list price and deliver measured throughput of 159, 477.6, and nothing published at all.

Five Factors That Decide Which Provider You Ship On

Most rankings here are published by vendors who put themselves first and never explain why. We are a vendor too, so the reasoning comes before the entries. Five things changed our ordering when we ran the numbers. Each one has a failure mode that costs you weeks rather than dollars.

Catalog Depth and How Fast New Weights Land 

A provider that ships new Qwen, DeepSeek, GLM, and Kimi releases in days rather than months keeps you from onboarding a second vendor for one model. That sounds like a convenience argument until you have done it. Adding a vendor means adding:

• A second contract and a second security review.
• A second set of credentials in your secret manager.
• A second rate-limit ceiling your capacity planning has to model.
• A second set of error codes and timeout behaviors your retry logic has to special-case.
• A second invoice to reconcile at month end.
• A second dashboard to open at 3am, before you know which half of your traffic is affected.
• A per-model vendor map in your routing layer that every new engineer has to learn.

The token price is rarely what makes that expensive. The weeks are.

Prices You Can Read Without a Sales Call

Opaque pricing is a real and widespread problem in this market, and it takes four common shapes:

• The partial rate card. Small models are priced on the page, and the ones you would actually ship say “contact sales.”
• The credit surcharge. A clean per-token rate billed through prepaid credits that carry a payment processing fee, so your effective rate runs several percent above the published number.
• The invisible tier. The good rate is gated behind a spend threshold you cannot see until you cross it.
• The undocumented discount. Cached-input and batch rates are frequently real and frequently unpublished, so you pay list for traffic that qualified for half.

One line item catches first-time buyers more than any of those. Reasoning models bill their thinking tokens as output, which is where a $0.60 output rate quietly becomes most of your monthly bill. The check is simple: before you commit volume, confirm the vendor publishes input, output, cached input, and batch rates for the exact model you plan to run, visible without a login.

Speed That Somebody Else Measured

Vendor speed claims are marketing. They run on the vendor’s hardware, at a concurrency the vendor chose, on a prompt the vendor wrote. Third-party measurement on identical weights is evidence, and it separates two numbers that get collapsed into one. Time to first token decides whether a voice agent feels alive or feels broken. Output throughput decides how long a 4,000-token summary takes to finish. A provider can win one and lose the other badly, which is exactly what Baseten and Groq do in the table above.

What It Costs You to Leave

OpenAI-compatible is a spectrum, and vendors land at very different points on it. Plain Chat Completions with text in and text out ports everywhere in an afternoon. Tool calling, parallel tool calls, JSON-schema structured output, streaming delta shapes, logprobs, seeds, and stop-sequence handling are where compatibility frays. The failure mode is nasty because it is silent: the request succeeds, the response comes back, and the shape is subtly different from what your parser expects. Run your own suite against the full surface you depend on before the switching argument becomes theoretical.

Vendor Transparency Around the Way They Serve the Model

Two providers can list the same checkpoint at the same context length and serve it at different numerical precisions with different sampling defaults, and your evaluation scores will move even though the model card did not. Almost nobody publishes the choice. DeepInfra exposes quantization per model in its public models API, which is why the section below can name precisions instead of guessing at them. The same disclosure question covers retention policy and serving region, and regulated buyers filter on those before they ever look at price.

What This All Means

Take those five to a vendor as questions. The answers sort the field fast.

The Best Open Source LLM API Providers, Ranked

1. DeepInfra

Every model page lists its per-token price and context window.

Best for: Teams running several models across different jobs who want one key, one bill, and a price floor.

Pricing: Pay-as-you-go per token, no minimum and no prepaid credits. gpt-oss-120b starts at $0.037 in and $0.17 out per million tokens on the base endpoint, and the same weights are available at $0.15 / $0.60 on Turbo and $0.20 / $0.95 on Ultra when you want the speed instead. Flagships sit at the low end too. DeepSeek-V4-Pro runs $1.30 / $2.60 and GLM-5.2 runs $0.75 / $2.40. Dedicated GPUs start at $0.89 per A100-hour and $2.20 per H100-hour, billed by the minute.

DeepInfra was founded in 2022 by the team behind imo, a messenger app that reached more than 200 million users. The company owns and operates its own GPU infrastructure across nine data centers rather than renting hyperscaler capacity, and processes close to five trillion tokens a week. In 2026, that scale helped close a $107 million Series B, with Nvidia among the investors

Open weights are the entire product here. No proprietary flagship sits above them. The public models API lists 360 entries spanning text, vision, image, video, speech, transcription, embeddings, and reranking, with more than 220 actively served, priced at the low end on nearly every model we checked. DeepSeek-V4-Pro runs $1.30 in and $2.60 out per million tokens against $1.74 and $3.48 at Together AI, Fireworks AI, and Baseten. GLM-5.2 runs $0.75 and $2.40 against $1.40 and $4.40 at Fireworks and Novita. Kimi K3 undercuts the $3.00 and $15.00 that everyone else charges.

What separates them is tiering. gpt-oss-120b ships as three endpoints at three price points: the base endpoint at $0.037 and $0.17, Turbo at $0.15 and $0.60, and Ultra at $0.20 and $0.95. Artificial Analysis measured the base endpoint at 46 output tokens per second and Turbo at 159, so you buy a speed tier deliberately instead of accepting one blended number. Batch summarization runs on the cheap tier while an interactive agent runs on Turbo, under one key and one bill.

Switching is a base_url change, the open-weights argument made concrete:

from openai import OpenAI
import os

# Same request, same model name, three different vendors.
PROVIDERS = {
    "deepinfra": ("https://api.deepinfra.com/v1/openai", os.environ["DEEPINFRA_API_TOKEN"]),
    "together":  ("https://api.together.xyz/v1",         os.environ["TOGETHER_API_KEY"]),
    "groq":      ("https://api.groq.com/openai/v1",      os.environ["GROQ_API_KEY"]),
}

for name, (base_url, api_key) in PROVIDERS.items():
    client = OpenAI(base_url=base_url, api_key=api_key)
    response = client.chat.completions.create(
        model="openai/gpt-oss-120b",
        messages=[{"role": "user", "content": "Summarize FP8 quantization in two sentences."}],
    )
    print(name, response.choices[0].message.content)
copy

What Makes This Special

• Three price points on one checkpoint. Base, Turbo, and Ultra are the same weights at different speeds, so you pick the tier per job instead of paying interactive rates for a nightly batch run.
• Quantization published per model. The models API returns the serving precision for every endpoint, and that number is what explains why your evaluation scores moved when you switched hosts.
• One key across text, vision, image, video, speech, and embeddings. Agent steps that swap modalities do not swap vendors.
• Serverless and dedicated behind the same client. Moving a hot workload onto reserved H100s does not mean rewriting the calling code.
• Zero-retention policy with SOC 2 and ISO 27001 certification, the filter regulated buyers apply before they ever compare prices.

Recent Changes

Verified against DeepInfra’s public models API:

• August 17, 2026: Qwen3.8-27B added at $0.40 and $3.00 per million tokens with a 262K context.
• August 14, 2026: DeepSeek-V4-Pro-0813 landed as the official successor to the preview checkpoint, at the same $1.30 and $2.60.
• August 12, 2026: Qwen3.8-2.4T-A95B added at $2.00 and $6.00, served at FP4.

Pros and Cons

Pros

• Deepest open-weight catalog of the six, refreshed within days of a major release.
• Lowest per-token pricing on most models we checked, with no credit prepay or spend gate to reach it.
• Explicit speed tiers on the same weights, so latency becomes a per-job choice.
• Serving precision published per model, which almost no competitor does.

Cons

• The fastest first token at high concurrency costs extra, since that means paying for Turbo or Ultra.
• Base-tier throughput of 46 output tokens per second is deliberately modest, so the cheapest tier is the wrong default for anything interactive.
• Fine-tuning is available but the training tooling is less built out than Together’s, which sells training as a product in its own right.
• No region-pinned model variants of the kind Fireworks publishes, so region-specific routing requires a dedicated deployment.

2. Together AI

Together AI pairs a broad serverless catalog with first-class fine-tuning.

Best for: Teams whose plan runs through a fine-tune and want training and serving under one vendor.

Pricing: Mid-to-high on serverless list. gpt-oss-120b sits at $0.15 and $0.60, roughly four times DeepInfra’s base rate on input. The discounts are the interesting part: cached input is published per model, so Kimi K3 drops from $3.00 to $0.30 per million on a cache hit and Qwen3.5-397B-A17B drops to $0.35 from $0.60, and batch runs at up to 50 percent off. Fine-tuning is priced openly at $0.48 to $8.00 per million training tokens depending on model size.

Together runs a wide catalog with a real training business next to it. If your roadmap ends in a custom checkpoint, the combination is what matters. Weights you train there serve on the same infrastructure behind the same API, so the handoff from experiment to production is a model ID change rather than a migration.

The surface is the broadest in this roundup by a distance. Chat, embeddings, rerank, images, video, text-to-speech, transcription, vision, batch, evaluations, a code interpreter for running model-generated code, and GPU clusters for training all sit under one account.

What Makes This Special

• Training and serving on one bill. Supervised fine-tuning and preference optimization run on the same platform that will host the result, so there is no export-and-redeploy step between them.
• Cached-input rates published per model, which turns prompt caching from an optimization you hope pays off into a number you can put in a spreadsheet.
• A code interpreter endpoint for executing model-generated code, which most inference-only vendors leave you to build.
• Reserved GPU clusters for teams that outgrow endpoints entirely and want to run their own training jobs.
• Project-scoped resources. Clusters, fine-tuned models, endpoints, evaluations, files, and API keys all scope to a project, which matters once more than one team shares an account.

Recent Changes

From the Together AI changelog:

• August 24, 2026: Fine-tuning quality improvements for Qwen3.5, Qwen3.6, and NVIDIA Nemotron variants, applied with no configuration change required.
• August 15, 2026: DeepSeek-V4-Pro-0813 on serverless with a 1M context, served at FP4.
• August 13, 2026: GLM-5.2 opened for fine-tuning.

Pros and Cons

Pros

• Transparent fine-tuning prices, published as a rate card you can read before you talk to anyone.
• Published cached-input rates per model, so caching savings are predictable before you build for them.
• The widest endpoint surface here: text, vision, image, video, audio, rerank, evals, and code execution.
• Long track record shipping open-weight releases quickly, with an active deprecation policy that names the replacement model.

Cons

• Mid-to-high serverless pricing, four times DeepInfra’s base rate on gpt-oss-120b input.
• Artificial Analysis publishes no independent throughput or TTFT number for Together’s gpt-oss-120b endpoint, so its speed numbers are self-reported.
• Aggressive deprecations mean pinned model IDs need periodic review, and older Qwen and Llama variants have been retired on a rolling basis through 2026.
• The breadth of the platform is overhead if all you want is one cheap chat endpoint.

3. Fireworks AI

Fireworks leans on enterprise deployment controls, including region-pinned variants.

Best for: Regulated teams serving LoRA adapters that need control over where inference runs.

Pricing: $0.15 and $0.60 for gpt-oss-120b, the same tier as Together and Groq, with a 10 percent premium on region-pinned variants: Kimi K3 costs $3.00 and $15.00 on the standard endpoint and $3.30 and $16.50 on the US-routed one. Batch inference bills at 50 percent of serverless inference rates. On-demand hardware is listed by the hour at $7.00 for H100 and H200, $10.00 for B200, and $12.00 to $18.00 for B300 and GB300.

Fireworks is the pick when compliance shapes the architecture. A 10 percent premium for a known serving region answers a question your legal team will ask, and having it published as a price rather than negotiated as a contract term is the part that saves you a quarter.

The platform supports LoRA SFT and LoRA DPO fine-tuning, and it serves multiple LoRA adapters against one base deployment, so a fleet of per-customer adapters does not become a fleet of per-customer GPUs.

What Makes This Special

• Region-pinned model variants published as list-price SKUs, so “where does inference run” has an answer with a number next to it.
• Multi-LoRA serving on a single deployment. Dozens of adapters share one base model on one set of GPUs instead of each getting their own.
• A 50 percent batch discount on any workload that tolerates delay, applied across the serverless catalog.
• Fine-tuning with real training controls, including LoRA SFT, LoRA DPO, ORPO, and configurable learning-rate schedules.
• On-demand GPU pricing published by the hour across five card types, including B300 and GB300.
• Structured outputs with JSON schema and custom grammar enforcement, which matters when the model’s output feeds a parser instead of a person.

Recent Changes

From the Fireworks changelog:

• August 14, 2026: DeepSeek V4 Flash retired from serverless, with DeepSeek V4 Flash (0731) named as the migration target.
• June 15 and 12, 2026: GLM 5.2, Kimi K2.7 Code, MiniMax M3, and Qwen 3.7 Plus added.
• June 10, 2026: Audio inference and image generation deprecated platform-wide, narrowing Fireworks to text, vision, and embeddings.

Pros and Cons

Pros

• LoRA adapter serving at scale, with many adapters per base deployment.
• Region-pinned endpoints priced openly, with the premium on the price list instead of buried in a contract.
• 50 percent batch discount across the serverless catalog.
• Strong structured-output support, including custom grammars.

Cons

• Serverless list prices match Together and Groq at $0.15 and $0.60 for gpt-oss-120b, with no cheap tier underneath.
• GLM-5.2 costs nearly double what it does elsewhere at $1.40 and $4.40.
• Audio and image generation were deprecated in June 2026, so multimodal pipelines need a second vendor.
• No independently published throughput or TTFT number for its gpt-oss-120b endpoint.

4. Groq

Groq runs custom LPU silicon rather than GPUs.

Best for: Latency-bound interactive products that only need one or two well-supported models.

Pricing: $0.15 and $0.60 for gpt-oss-120b, identical list price to Together and Fireworks, with the batch API and cached input each cutting rates in half. You pay an ordinary price for extraordinary speed, which is the whole argument. There is no discount tier below that, and no cheap endpoint for background work.

Groq built its own inference silicon, and the measured result is not subtle. Artificial Analysis clocks Groq’s gpt-oss-120b endpoint at 477.6 output tokens per second and a 0.68-second time to first token. That is roughly ten times a commodity shared endpoint on the same weights. For a voice agent, that difference is the product.

The tradeoff is catalog. Groq’s production list is short and skews toward Llama and gpt-oss variants, leaving DeepSeek, GLM, and Kimi largely absent. Workloads that want DeepSeek-V4-Pro or GLM-5.2 run elsewhere. The endpoint surface is narrow to match: chat, a beta responses API, audio transcription and speech, batch, and files, with no embeddings and no vision endpoint at all.

What Makes This Special

• Custom LPU silicon instead of GPUs, which is why the throughput number lands an order of magnitude above the field instead of a few percent above it.
• 477.6 output tokens per second measured by a third party, the highest in this roundup by a wide margin.
• Latency that holds under concurrency, the property that separates a voice-agent demo from a voice agent in production.
• Batch and cached input each halve the rate, so the one lever Groq gives you on price is a real one.
• A deliberately small catalog. Fewer models, tuned harder, is the operating model here, and it is a choice.

Recent Changes

From Groq’s deprecation notices:

• August 16, 2026: llama-3.1-8b-instant and llama-3.3-70b-versatile deprecated, with openai/gpt-oss-20b, openai/gpt-oss-120b, and qwen/qwen3.6-27b named as replacements.
• July 17, 2026: qwen/qwen3-32b and meta-llama/llama-4-scout-17b retired, also consolidated onto gpt-oss endpoints.
• Fine-tuning now appears in the API reference as a closed beta, an early move toward customization that Groq did not previously offer.

Pros and Cons

Pros

• Best measured throughput among general-purpose providers, by roughly 2.4x over the next fastest.
• Consistent latency under load, including at concurrency levels where shared endpoints start to queue.
• Batch and cache discounts that each halve the effective rate.
• OpenAI-compatible chat surface, so trying it costs an afternoon.

Cons

• Narrow catalog that lags new open-weight releases, with DeepSeek, GLM, and Kimi largely absent.
• Aggressive consolidation onto gpt-oss endpoints retired four widely used Llama and Qwen models in 2026 alone, so pinned model IDs break.
• No embeddings and no vision endpoint, which rules Groq out as a sole vendor for RAG or multimodal work.
• Fine-tuning is closed beta, so there is no self-serve custom-weights path today.

5. Novita AI

Novita competes primarily on price across a mid-sized open-weight catalog.

Best for: Cost-sensitive batch and background workloads where a second of extra latency changes nothing.

Pricing: The discount position, held consistently across the catalog. gpt-oss-120b runs $0.05 and $0.25, second cheapest in this roundup, and DeepSeek-V4-Flash-0731 lands at $0.14 and $0.28. DeepSeek-V4-Pro at $1.60 and $3.20 undercuts the $1.74 and $3.48 that Together, Fireworks, and Baseten all charge.

Measured throughput sits at 90 output tokens per second with a 0.97 second time to first token. Mid-pack. Faster than the cheapest shared endpoints, well behind the silicon players. The catalog covers the major families without matching the depth of DeepInfra or Together.

The OpenAI-compatible API surface covers embeddings and rerank as well as chat, so a RAG stack runs end to end here. Novita also sells GPU instances and serverless GPU endpoints alongside the token API, so a workload that outgrows shared capacity has somewhere to go without changing vendors.

What Makes This Special

• The discount holds on flagships. Plenty of vendors are cheap on small models. Novita’s DeepSeek-V4-Pro at $1.60 and $3.20 undercuts Together, Fireworks, and Baseten on a flagship, which is the harder half.
• Serving optimizations shipped in public. Novita published throughput gains for Kimi-K2.6 and Kimi-K2.7-Code using DSpark speculative decoding in vLLM, which is more engineering detail than most price-led vendors share.
• An EU-hosted gateway through a partnership with Opper AI, giving European teams a routing option that keeps traffic in region.
• GPU instances and serverless GPU endpoints under the same account as the token API.
• A published TTFT and throughput number from Artificial Analysis, which two of its more expensive competitors cannot say.

Recent Changes

From the Novita AI blog:

• July 15, 2026: TiDB integration, pairing managed deployment with database provisioning and migrations for AI-generated applications.
• July 10, 2026: Published inference optimization results for Kimi-K2.6 and Kimi-K2.7-Code using DSpark speculative decoding in vLLM.
• July 9, 2026: EU market expansion through Opper AI’s EU-hosted gateway, exposing 80+ open-weight models through one API.

Pros and Cons

Pros

• Aggressive pricing across the catalog, second cheapest endpoint we priced.
• Solid coverage of the DeepSeek, Qwen, GLM, and Kimi families.
• Independently measured speed numbers, unlike Together and Fireworks.
• EU-hosted routing option for teams with data-residency constraints.

Cons

• Middling measured speed at 90 output tokens per second and 0.97s TTFT, which rules it out for anything interactive.
• Thinner enterprise and compliance tooling than Fireworks or Baseten.
• Catalog depth trails DeepInfra and Together, so an unusual model may not be there.
• Recent public roadmap leans toward infrastructure and integrations, with less visible movement on catalog freshness.

6. Baseten

Baseten sells a deployment platform first and a token API second.

Best for: Teams deploying proprietary or fine-tuned weights that still want a managed control plane.

Pricing: Two meters, and the dedicated one is the main event. Hardware bills by the minute at $0.10833 for an H100, $0.16633 for a B200, and $0.06667 for an A100, which works out to roughly $6.50, $9.98, and $4.00 per wall-clock hour. The token API is competitive without leading: $0.10 and $0.50 for gpt-oss-120b, $0.13 and $0.26 for DeepSeek-V4-Flash-0731, and Nemotron 3 Ultra at $0.60 and $2.40.

Baseten approaches the problem from the deployment side. Its Truss packaging format defines the model container, dependencies, and runtime configuration, and ships custom, fine-tuned, or private weights onto dedicated hardware. Anything the shared catalog lacks, you deploy yourself, including custom Docker servers running vLLM, SGLang, or Triton.

The measured numbers are the surprise. Artificial Analysis records a 0.24 second time to first token on gpt-oss-120b, the fastest of the seventeen providers it benchmarks, alongside 198 output tokens per second.

What Makes This Special

• Fastest measured time to first token in the field, 0.24 seconds, ahead of Groq’s custom silicon on that specific metric.
• Truss as a real packaging format. Config-only deployments, custom Python model classes, and custom Docker servers all work, so an unusual serving stack is not a blocker.
• Chains for multi-step workflows, which lets each step in a pipeline run on the hardware it actually needs instead of sizing everything for the largest model.
• Per-minute GPU billing, so a deployment that scales to zero overnight stops costing money at minute granularity.
• Programmatic observability. Logs, metrics, and audit logs are exposed through APIs, so they land in your own monitoring stack.

Recent Change

From the Baseten changelog:

• August 19, 2026: Runtime OIDC, so deployments authenticate to OIDC-compatible cloud providers without storing long-lived credentials.
• August 14, 2026: DeepSeek-V4-Pro-0813 available through Model APIs on the OpenAI-compatible endpoint.
• July 22, 2026: Observability APIs for logs, metrics, and audit logs across deployments and environments.

Pros and Cons

Pros

• Fastest measured TTFT of any provider Artificial Analysis benchmarks.
• Real custom-weights deployment through Truss, including custom containers.
• Per-minute GPU billing with no hourly rounding.
• Chains for orchestrating multi-model pipelines across different hardware.

Cons

• Token pricing sits above the discount tier, roughly 2.7x DeepInfra’s base gpt-oss-120b input rate.
• Dedicated H100 capacity at roughly $6.50 per hour is close to triple DeepInfra’s $2.20.
• The platform assumes infrastructure responsibility that a pure token API removes, which is overhead if you only want an endpoint.
• Model APIs are the secondary product, so catalog breadth trails the token-first vendors.

The Aggregator Question: OpenRouter

OpenRouter routes requests to upstream providers rather than operating its own inference.

Best for: Discovery, failover, and reaching a model at a vendor you have not contracted with yet.

Pricing: Per-token rates are passed through from the upstream provider without markup, so the rate you pay is whichever vendor served the request. The fee sits on the credit purchase instead, at 5.5 percent for card payments and 5 percent for crypto. People often forget this number when they compare a routed rate against a direct one.

OpenRouter belongs in every list like this one and in a different category from the six above. It operates no inference. It routes your request to a provider that does, offering Balanced, Nitro, and Exacto modes that favor price, speed, or tool-calling accuracy.

The convenience is genuine, and so are the consequences. Your measured latency belongs to whichever upstream vendor received the request, and it can change between calls. Your data handling follows that vendor’s retention policy rather than a single published one. Rate limits come from the upstream vendor too, so your ceiling can move without notice. When throughput degrades at 3am, the operator holding the answer is one layer removed from your account manager.

What Makes This Special

• One endpoint, every catalog. A single key reaches models across dozens of upstream vendors, which is the fastest way to try something before contracting for it.
• Routing modes as a product. Balanced, Nitro, and Exacto trade price against speed against tool-calling accuracy on the same request.
• Automatic failover. When an upstream provider degrades, traffic moves without a code change on your side.
• Published benchmarks across models, including image and web-search leaderboards that are genuinely useful for shortlisting.
• No markup on inference. The business model sits on credit purchases, not on the token rate.

Recent Changes

From the OpenRouter announcements:

• August 21, 2026: Image Benchmarks comparing 39 image models across 15 prompts, with performance and pricing side by side.
• August 19, 2026: OpenRouter announced it is joining Stripe, which is worth knowing before you sign a long-term agreement.
• August 10, 2026: An Auto router that selects models from aggregated usage patterns rather than a task classifier.

Pros and Cons

Pros

• Broadest reach of any single key in this roundup.
• No per-token markup over upstream list prices.
• Routing modes and automatic failover come for free with the abstraction.
• Strong public benchmarking work across text, image, and web search.

Cons

• Your latency is whichever upstream vendor got the request, and it can change between calls.
• Data handling follows whichever upstream provider took the request, so there is no single retention policy to hand your compliance team. For regulated buyers that is a hard blocker.
• The 5.5 percent credit purchase fee is a real cost that a direct contract does not carry.
• Incident escalation runs through an extra layer at exactly the moment you need it not to.

Route through an aggregator to discover which provider suits a workload. Contract directly with that provider once you know.

What the Same Model Costs Across Providers

The gpt-oss-120b spread earlier is one model. Flagships tell a similar story, and the pattern repeats: three or four vendors cluster on an identical list price while one or two sit meaningfully below it.

In our six-provider DeepSeek-V4-Pro benchmark, five vendors tied on price and the decision fell entirely to latency and throughput. That happens more often than the price tables suggest, which is why the next section matters more than either of them.

Why the Same Model Scores Differently Across Providers

Two providers can serve identical weights and return measurably different output. Quantization is the usual culprit. Choosing between bfloat16, FP8, and FP4 trades memory footprint and throughput against numerical precision, and the loss shows up first on long-context reasoning and exact-format tasks like structured JSON.

The DeepSeek-V4-Pro-0813 release is a clean example. DeepInfra’s models API reports it serving at FP8. Together’s changelog lists its August 15 serverless launch of the same checkpoint at FP4. Same weights, same model ID, different numbers behind the endpoint, and neither vendor is doing anything wrong.

Most vendors never publish the choice at all. DeepInfra exposes it per model in its public models API: gpt-oss-120b serves at bfloat16, the Ultra variant at FP8, Qwen3.5-397B-A17B at FP8, DeepSeek-V4-Pro at FP8, and GLM-5.2 at FP4. Serving stack and default sampling parameters shift results too.

Run your own evaluation set against the specific endpoint you plan to buy. A benchmark published against the original checkpoint tells you what the weights can do. It says nothing about what a given endpoint does with them. Our Kimi K3 provider comparison and GLM-5.2 cost analysis walk through this per-endpoint variation on two current models.

Licensing: What Your Provider Lets You Ship

Provider listings rarely mention licenses. The license travels with the weights regardless of who serves them. Three families cover most of the catalog.

Apache 2.0 and MIT models impose almost nothing. gpt-oss-120b ships under Apache 2.0, so you can build commercial products, modify the weights, and redistribute derivatives without a copyleft obligation. Meta’s models, Llama 4 Maverick included, use the Llama 4 Community License, which adds naming requirements and cuts off at “greater than 700 million monthly active users in the preceding calendar month,” above which you need a separate agreement. Labs like DeepSeek and Moonshot publish custom terms that restrict specific downstream uses.

The provider matters in one respect. Check whether it lets you export a fine-tuned derivative and run it elsewhere. Read that clause before the weights become load-bearing.

How to Choose, by Workload Shape

Start from the shape of your traffic rather than the leaderboard. Model selection is a separate decision, and a good one on the wrong provider still costs you three times the money or half a second of latency you cannot afford. Each shape below ends at a provider trait to buy.

Batch and background jobs. Nightly summarization runs, enrichment pipelines that backfill a table, offline classification of a support queue. Nobody is sitting there waiting, so a job that finishes in 40 minutes instead of 20 changes nothing downstream. Volume is what moves the bill, because these jobs push tens or hundreds of millions of tokens a month. At that scale a ten cent difference per million tokens compounds into real money while half a second of latency compounds into nothing at all. Buy the cheapest per-token endpoint that clears your quality bar and take the batch discount where one exists, since Fireworks and Groq both bill batch at half rate. DeepSeek-V4-Flash-0731 at $0.08 and $0.18 with a 1M-token context handles most of this category.

Interactive chat and voice. Time to first token dominates perceived quality here, because the user is listening to silence until the first token arrives and reads the rest more slowly than the model emits it. Half a second of TTFT is the difference between a voice agent that feels like a conversation and one that feels broken. Buy measured TTFT and treat advertised throughput as a separate number that will not help you here, which means Baseten at 0.24 seconds or Groq at 0.68. Pay the premium tier without agonizing over it. MiniMax-M3 at $0.28 and $1.10 gives you a 524K context without flagship pricing.

Agentic workflows. Long multi-step runs that swap models per step want one key across a broad catalog. The routing logic gets ugly fast when each step needs a different vendor: every provider brings its own rate limit, its own retry semantics, its own failure modes, and its own place in your observability stack, and your orchestrator has to know all of it. One vendor with catalog depth turns that into a model ID per step. Our writeup on choosing models for agentic workloads covers the per-step tradeoffs.

Fine-tunes and private weights. Anything ending in a custom checkpoint needs a deployment platform, because a shared catalog will never serve weights you trained yourself. Together if you want to train and serve in one place, Baseten if you already have the weights and want to bring your own container, Fireworks if you are serving many LoRA adapters off one base model. Ask about export rights before you start training.

Compliance-bound. Filter on retention policy, serving region, and certifications first, then compare price inside whatever survives. Filtering this way rules out aggregators immediately, since their retention story belongs to whichever upstream vendor happened to take the request

Frequently Asked Questions

What Is the Cheapest Open Source LLM API? 

On the models we priced August 12, 2026, DeepInfra’s base endpoints were lowest on most models, including gpt-oss-120b at $0.037 and $0.17 per million tokens. The cheapest option changes monthly, so re-check the price per million tokens before you commit volume.

Can I Switch Providers Without Rewriting My Code? 

If both speak the OpenAI Chat Completions format, yes. Change base_url and the API key, as in the snippet above. Verify tool-calling and structured-output behavior, which is where compatibility usually frays.

Is Serverless Cheaper Than Dedicated Hosting? 

Until your endpoint runs hot for most of the day. Serverless bills only for tokens consumed, dedicated bills for reserved wall-clock time whether traffic arrives or not.

What Is the Difference Between Open Weight and Open Source? 

Open weights means the trained parameters are published and runnable. Open source in the strict sense would also include training data and code, which almost no frontier lab releases. Nearly every model in this article is open weights.

Why Do Two Providers Give Different Answers For the Same Model? 

Serving precision, serving stack, and default sampling parameters all vary between hosts. A checkpoint served at FP4 will behave differently from the same checkpoint at FP8, most visibly on long-context reasoning and strict output formats. Ask the vendor what quantization the endpoint runs, and run your own evaluation set against the exact endpoint you plan to buy.

Getting Started

The open-weight market has enough real operators that no single provider wins every workload. Groq owns raw throughput, Baseten owns first-token latency, Together owns the fine-tuning path, and Fireworks owns the compliance corner. DeepInfra’s argument is breadth and floor price: 360 models under one key, explicit speed tiers, published serving precision, and per-token rates at the low end of everything we checked, with dedicated instances waiting when a workload outgrows shared capacity.

Browse the catalog and current pricing at deepinfra.com/pricing, or read the API documentation to port an existing OpenAI client. Questions and benchmark disagreements are welcome: email feedback@deepinfra.com, join the DeepInfra Discord, or find us on X at @DeepInfra. If you want the model-side view of this comparison, our breakdown of open versus closed source models on intelligence, price, and speed covers the other half of the decision.