Compare Llama2 vs OpenAI models for FREE.

Published on 2023.09.28 by Nikola Borisov

At DeepInfra we host the best open source LLM models. We are always working hard to make our APIs simple and easy to use.

Today we are excited to announce a very easy way to quickly try our models like Llama2 70b and Mistral 7b and compare them to OpenAI's models. You only need to change the API endpoint URL and the model name to quickly see if these models are a good fit for your application.

Here is a quick example of how to use the OpenAI Python client with our models:

import openai

# Point OpenAI client to our endpoint
openai.api_base = "https://api.deepinfra.com/v1/openai"
# Just leave the API key empty. You don't need it to try our models. 
openai.api_key = ""

# Your chosen model here
MODEL_DI = "meta-llama/Llama-2-70b-chat-hf"
chat_completion = openai.ChatCompletion.create(
    model="meta-llama/Llama-2-70b-chat-hf",
    messages=[{"role": "user", "content": "Hello world"}],
    stream=True,
)

# print the chat completion
for event in chat_completion:
    print(event.choices)
copy

Rate limits on no API key

To make it as simple as possible you don't even have to create an account with DeepInfra to try our models. Just pass empty string as api_key and you are good to go. We rate limit the unauthenticated requests by IP address.

Pricing and Production ready

When you are ready to use our models in production, you can create an account at DeepInfra and get an API key. We offer the best pricing for the llama 2 70b model at just $1 per 1M tokens. If you need any help, just reach out to us on our Discord server.

Related articles

Best API Providers for Wan 3.0 IntegrationWan3.0 is Alibaba’s all-in-one video generation model. It accepts text, images, video, audio, documents, and web pages, and it produces up to 30 seconds of 1080P video with synchronized audio. Production use raises practical questions. Video jobs run asynchronously, reference inputs can be large, and each provider exposes a different slice of the model’s features […]

Qwen3.5 35B A3B API Benchmarks: Latency, Throughput & CostAbout Qwen3.5 35B A3B Qwen3.5 35B A3B is a native vision-language model released by Alibaba Cloud in February 2026. It uses a hybrid architecture that integrates Gated Delta Networks with a sparse Mixture-of-Experts model, achieving higher inference efficiency. With 35 billion total parameters and only 3 billion activated per token through 256 experts (8 routed […]

vLLM vs SGLang: Performance, Features & Deployment ComparedSomebody on your team read a benchmark post, and now there’s a ticket to migrate the inference stack. That’s how most vLLM vs SGLang decisions start. A published test reports a 29 percent throughput gap, the number lands in Slack, and two weeks later you’re debugging kernel version conflicts at midnight while p99 latency sits […]

View all