DeepInfra raises $107M Series B to scale the inference cloud — read the announcement

Compare Llama2 vs OpenAI models for FREE.

Published on 2023.09.28 by Nikola Borisov
Compare Llama2 vs OpenAI models for FREE.

At DeepInfra we host the best open source LLM models. We are always working hard to make our APIs simple and easy to use.

Today we are excited to announce a very easy way to quickly try our models like Llama2 70b and Mistral 7b and compare them to OpenAI's models. You only need to change the API endpoint URL and the model name to quickly see if these models are a good fit for your application.

Here is a quick example of how to use the OpenAI Python client with our models:

import openai

# Point OpenAI client to our endpoint
openai.api_base = "https://api.deepinfra.com/v1/openai"
# Just leave the API key empty. You don't need it to try our models. 
openai.api_key = ""

# Your chosen model here
MODEL_DI = "meta-llama/Llama-2-70b-chat-hf"
chat_completion = openai.ChatCompletion.create(
    model="meta-llama/Llama-2-70b-chat-hf",
    messages=[{"role": "user", "content": "Hello world"}],
    stream=True,
)

# print the chat completion
for event in chat_completion:
    print(event.choices)
copy

Rate limits on no API key

To make it as simple as possible you don't even have to create an account with DeepInfra to try our models. Just pass empty string as api_key and you are good to go. We rate limit the unauthenticated requests by IP address.

Pricing and Production ready

When you are ready to use our models in production, you can create an account at DeepInfra and get an API key. We offer the best pricing for the llama 2 70b model at just $1 per 1M tokens. If you need any help, just reach out to us on our Discord server.

Related articles
Best GLM-5.3-Flash API Providers in 2026Best GLM-5.3-Flash API Providers in 2026<p>As AI architectures shift toward highly optimized Mixture-of-Experts (MoE) models, GLM-5.3-Flash has emerged as a strong option for developers who need fast inference, advanced reasoning, and robust tool-calling. Deploying a model like this in production still means balancing token costs, time-to-first-token (TTFT) latency, raw throughput, and API reliability. The inference-cloud and API-gateway ecosystem for this [&hellip;]</p>
Token Verbosity Is the New Pricing WarToken Verbosity Is the New Pricing War<p>With each new model release, we see new benchmark claims and new arguments justifying the value of its tokens. However, when teams compare models for real-world workloads, the sticker price alone hides large per-task differences in the amount of work a model requires to solve a task. Cheaper tokens do not necessarily translate to lower [&hellip;]</p>
DeepSeek V4.1 Flash 1017 Pricing: API Costs & Providers 2026DeepSeek V4.1 Flash 1017 Pricing: API Costs & Providers 2026<p>Updated November 2026 for DeepSeek-V4.1-Flash-1017. If you&#8217;re evaluating long-context reasoning models in late 2026, DeepSeek-V4.1-Flash-1017 is hard to ignore. The weights are open, the provider market is crowded, and DeepInfra&#8217;s promotional price undercuts DeepSeek&#8217;s own off-peak rate. DeepInfra now serves the 1017 release at $0.098 per million input tokens, $0.294 per million output tokens and [&hellip;]</p>