gpt-oss-120b is an open-weight, 117B-parameter Mixture-of-Experts (MoE) language model from OpenAI designed for high-reasoning, agentic, and general-purpose production use cases. The model supports configurable reasoning depth, full chain-of-thought access, and native tool use, including function calling, browsing, and structured output generation.

gpt-oss-120b

gpt-oss-20b is an open-weight 21B parameter model released by OpenAI under the Apache 2.0 license. It uses a Mixture-of-Experts (MoE) architecture with 3.6B active parameters per forward pass, optimized for lower-latency inference. The model is trained in OpenAI’s Harmony response format and supports reasoning level configuration, fine-tuning, and agentic capabilities including function calling, tool use, and structured outputs.

gpt-oss-20b

Qwen3-Coder-480B-A35B-Instruct is the Qwen3's most agentic code model, featuring Significant Performance on Agentic Coding, Agentic Browser-Use and other foundational coding tasks, achieving results comparable to Claude Sonnet.

Qwen3-Coder-480B-A35B-Instruct-Turbo

olmOCR is a specialized AI tool that converts PDF documents into clean, structured text while preserving important formatting and layout information. What makes olmOCR particularly valuable for developers is its ability to handle challenging PDFs that traditional OCR tools struggle with—including complex layouts, poor-quality scans, handwritten text, and documents with mixed content types. Built on a fine-tuned 7B vision-language model, olmOCR provides enterprise-grade PDF processing at a fraction of the cost of proprietary solutions.

olmOCR-7B-0725-FP8

The GLM-4.5 series models are foundation models designed for intelligent agents. GLM-4.5 has 355 billion total parameters with 32 billion active parameters, while GLM-4.5-Air adopts a more compact design with 106 billion total parameters and 12 billion active parameters. GLM-4.5 models unify reasoning, coding, and intelligent agent capabilities to meet the complex demands of intelligent agent applications.

GLM-4.5

Kimi K2 is a large-scale Mixture-of-Experts (MoE) language model developed by Moonshot AI, featuring 1 trillion total parameters with 32 billion active per forward pass. It is optimized for agentic capabilities, including advanced tool use, reasoning, and code synthesis. Kimi K2 excels across a broad range of benchmarks, particularly in coding (LiveCodeBench, SWE-bench), reasoning (ZebraLogic, GPQA), and tool-use (Tau2, AceBench) tasks.

Kimi-K2-Instruct

Qwen3-235B-A22B-Thinking-2507 is the Qwen3's new model with scaling the thinking capability of Qwen3-235B-A22B, improving both the quality and depth of reasoning. 

Qwen3-235B-A22B-Thinking-2507

Qwen3-Coder-480B-A35B-Instruct

GLM-4.5-Air

Voxtral Small is an enhancement of Mistral Small 3, incorporating state-of-the-art audio input capabilities while retaining best-in-class text performance. It excels at speech transcription, translation and audio understanding.

Voxtral-Small-24B-2507

Voxtral Mini is an enhancement of Ministral 3B, incorporating state-of-the-art audio input capabilities while retaining best-in-class text performance. It excels at speech transcription, translation and audio understanding.

Voxtral-Mini-3B-2507

The DeepSeek R1 0528 turbo model is a state of the art reasoning model that can generate very quick responses

DeepSeek-R1-0528-Turbo

Qwen3-235B-A22B-Instruct-2507 is the updated version of the Qwen3-235B-A22B non-thinking mode, featuring Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage.  

Qwen3-235B-A22B-Instruct-2507

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support

Qwen3-30B-A3B

Qwen3-32B

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support. 

Qwen3-14B

DeepSeek-V3-0324-Turbo

The Llama 4 collection of models are natively multimodal AI models that enable text and multimodal experiences. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding. Llama 4 Maverick, a 17 billion parameter model with 128 experts

Llama-4-Maverick-17B-128E-Instruct-Turbo

Llama-4-Maverick-17B-128E-Instruct-FP8

The Llama 4 collection of models are natively multimodal AI models that enable text and multimodal experiences. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding. Llama 4 Scout, a 17 billion parameter model with 16 experts

Llama-4-Scout-17B-16E-Instruct

The DeepSeek R1 model has undergone a minor version upgrade, with the current version being DeepSeek-R1-0528.

DeepSeek-R1-0528

DeepSeek-V3-0324, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token, an improved iteration over DeepSeek-V3.

DeepSeek-V3-0324

Devstral is an agentic LLM for software engineering tasks, making it a great choice for software engineering agents.

Devstral-Small-2507

Mistral-Small-3.2-24B-Instruct is a drop-in upgrade over the 3.1 release, with markedly better instruction following, roughly half the infinite-generation errors, and a more robust function-calling interface—while otherwise matching or slightly improving on all previous text and vision benchmarks.

Mistral-Small-3.2-24B-Instruct-2506

Llama Guard 4 is a natively multimodal safety classifier with 12 billion parameters trained jointly on text and multiple images. Llama Guard 4 is a dense architecture pruned from the Llama 4 Scout pre-trained model and fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It itself acts as an LLM: it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.

Llama-Guard-4-12B

QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.

QwQ-32B

Anthropic’s most powerful model yet and the state-of-the-art coding model. It delivers sustained performance on long-running tasks that require focused effort and thousands of steps, significantly expanding what AI agents can solve. Claude Opus 4 is ideal for powering frontier agent products and features.

claude-4-opus

Anthropic's mid-size model with superior intelligence for high-volume uses in coding, in-depth research, agents, & more.

claude-4-sonnet

Gemini 2.5 Flash is Google's latest thinking model, designed to tackle increasingly complex problems. It's capable of reasoning through their thoughts before responding, resulting in enhanced performance and improved accuracy.  Gemini 2.5 Flash: best for balancing reasoning and speed.

gemini-2.5-flash

Gemini 2.5 Pro is Google's the most advanced thinking model, designed to tackle increasingly complex problems. Gemini 2.5 Pro leads common benchmarks by meaningful margins and showcases strong reasoning and code capabilities.  Gemini 2.5 models are thinking models, capable of reasoning through their thoughts before responding, resulting in enhanced performance and improved accuracy.  The Gemini 2.5 Pro model is now available on DeepInfra.

gemini-2.5-pro

Gemma 3 introduces multimodality, supporting vision-language input and text outputs. It handles context windows up to 128k tokens, understands over 140 languages, and offers improved math, reasoning, and chat capabilities, including structured outputs and function calling. Gemma 3 27B is Google's latest open source model, successor to Gemma 2

gemma-3-27b-it

Gemma 3 introduces multimodality, supporting vision-language input and text outputs. It handles context windows up to 128k tokens, understands over 140 languages, and offers improved math, reasoning, and chat capabilities, including structured outputs and function calling. Gemma 3-12B is Google's latest open source model, successor to Gemma 2

gemma-3-12b-it

gemma-3-4b-it

Kokoro is an open-weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, Kokoro can be deployed anywhere from production environments to personal projects.

Kokoro-82M

Orpheus TTS is a state-of-the-art, Llama-based Speech-LLM designed for high-quality, empathetic text-to-speech generation. This model has been finetuned to deliver human-level speech synthesis, achieving exceptional clarity, expressiveness, and real-time streaming performances.

orpheus-3b-0.1-ft

CSM (Conversational Speech Model) is a speech generation model from Sesame that generates RVQ audio codes from text and audio inputs. The model architecture employs a Llama backbone and a smaller audio decoder that produces Mimi audio codes.

csm-1b

DeepSeek-R1-Distill-Llama-70B is a highly efficient language model that leverages knowledge distillation to achieve state-of-the-art performance. This model distills the reasoning patterns of larger models into a smaller, more agile architecture, resulting in exceptional results on benchmarks like AIME 2024, MATH-500, and LiveCodeBench. With 70 billion parameters, DeepSeek-R1-Distill-Llama-70B offers a unique balance of accuracy and efficiency, making it an ideal choice for a wide range of natural language processing tasks. 

DeepSeek-R1-Distill-Llama-70B

DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2. 

DeepSeek-V3

Llama 3.3-70B Turbo is a highly optimized version of the Llama 3.3-70B model, utilizing FP8 quantization to deliver significantly faster inference speeds with a minor trade-off in accuracy. The model is designed to be helpful, safe, and flexible, with a focus on responsible deployment and mitigating potential risks such as bias, toxicity, and misinformation. It achieves state-of-the-art performance on various benchmarks, including conversational tasks, language translation, and text generation.

Llama-3.3-70B-Instruct-Turbo

Llama 3.3-70B is a multilingual LLM trained on a massive dataset of 15 trillion tokens, fine-tuned for instruction-following and conversational dialogue. The model is designed to be helpful, safe, and flexible, with a focus on responsible deployment and mitigating potential risks such as bias, toxicity, and misinformation. It achieves state-of-the-art performance on various benchmarks, including conversational tasks, language translation, and text generation.

Llama-3.3-70B-Instruct

Phi-4 is a model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.

phi-4

Whisper is a state-of-the-art model for automatic speech recognition (ASR) and speech translation, proposed in the paper "Robust Speech Recognition via Large-Scale Weak Supervision" by Alec Radford  et al. from OpenAI. Trained on >5M hours of labeled data, Whisper demonstrates a strong ability to generalise to many datasets and domains in a zero-shot setting. Whisper large-v3-turbo is a finetuned version of a pruned Whisper large-v3. In other words, it's the exact same model, except that the number of decoding layers have reduced from 32 to 4. As a result, the model is way faster, at the expense of a minor quality degradation.

whisper-large-v3-turbo

You can use cURL or any other http client to run inferences:

```bash
curl -X POST \
    -d '{"input": "The quick brown fox jumps over the lazy dog"}'  \
    -H "Authorization: bearer $DEEPINFRA_TOKEN"  \
    -H 'Content-Type: application/json'  \
    'https://api.deepinfra.com/v1/inference/canopylabs/orpheus-3b-0.1-ft'
```

which will give you back something similar to:

```json
{
  "audio": null,
  "input_character_length": 0,
  "output_format": "",
  "words": [
    {
      "end": 1.0,
      "start": 0.0,
      "text": "Hello"
    },
    {
      "end": 5.0,
      "start": 4.0,
      "text": "World"
    }
  ],
  "request_id": null,
  "inference_status": {
    "status": "unknown",
    "runtime_ms": 0,
    "cost": 0.0,
    "tokens_generated": 0,
    "tokens_input": 0
  }
}

```


You can use our command-line tool [deepctl](/docs/advanced/deepctl) to run
inferences:

```bash
deepctl infer \
    -m 'canopylabs/orpheus-3b-0.1-ft'  \
    -i 'input=The quick brown fox jumps over the lazy dog'
```

which will give you back something similar to:

```json
{
  "audio": null,
  "input_character_length": 0,
  "output_format": "",
  "words": [
    {
      "end": 1.0,
      "start": 0.0,
      "text": "Hello"
    },
    {
      "end": 5.0,
      "start": 4.0,
      "text": "World"
    }
  ],
  "request_id": null,
  "inference_status": {
    "status": "unknown",
    "runtime_ms": 0,
    "cost": 0.0,
    "tokens_generated": 0,
    "tokens_input": 0
  }
}

```


The DeepInfra OpenAI-compatible Speech API endpoint enables users to effortlessly convert text into speech audio. This document outlines how to integrate and utilize this endpoint to quickly create speech from text inputs, leveraging various audio output formats.

## Create speech

Use the following example of pythong code to generate an audio file from your text input:

```python
from pathlib import Path
from openai import OpenAI

client = OpenAI(base_url="https://api.deepinfra.com/v1/openai",
                api_key="$DEEPINFRA_TOKEN")

speech_file_path = Path(__file__).parent / "speech.mp3"
with client.audio.speech.with_streaming_response.create(
  model="canopylabs/orpheus-3b-0.1-ft",
  voice="tara",
  input="The quick brown fox jumped over the lazy dog.",
  response_format="mp3",
) as response:
  response.stream_to_file(speech_file_path)
```

The API returns the generated audio file in the requested format (e.g., `mp3`, `pcm`). The example above saves the audio output directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Best for lowest latency streaming*        |
| mp3    | Compressed audio suitable for storage | General use and file distribution          |
| opus   | Compressed audio ideal for streaming  | Efficient streaming and real-time playback |
| flac   | Lossless audio compression            | High-quality archival storage              |
| wav    | Uncompressed audio                    | High-quality audio processing tasks        |

### Performance Recommendations

- **For lowest latency (fastest initial audio chunk):**
  - Use the `pcm` output format.

- **For general purposes:**
  - Use the `mp3` or `opus` formats for optimal balance between quality and file size.



The DeepInfra OpenAI-compatible Speech API endpoint enables users to effortlessly convert text into speech audio. This document outlines how to integrate and utilize this endpoint to quickly create speech from text inputs, leveraging various audio output formats.

## Create speech

Use the following example of js code to generate an audio file from your text input:

```javascript
import fs from "fs";
import path from "path";
import OpenAI from "openai";

const openai = new OpenAI(base_url="https://api.deepinfra.com/v1/openai",
                          api_key="$DEEPINFRA_TOKEN");

const speechFile = path.resolve("./speech.mp3");

async function main() {
  const mp3 = await openai.audio.speech.create({
    model: "canopylabs/orpheus-3b-0.1-ft",
    voice: "tara",
    input: "The quick brown fox jumped over the lazy dog.",
    response_format: "mp3",
  });
  console.log(speechFile);
  const buffer = Buffer.from(await mp3.arrayBuffer());
  await fs.promises.writeFile(speechFile, buffer);
}
main();
```

The API returns the generated audio file in the requested format (e.g., `mp3`, `pcm`). The example above saves the audio output directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Best for lowest latency streaming*        |
| mp3    | Compressed audio suitable for storage | General use and file distribution          |
| opus   | Compressed audio ideal for streaming  | Efficient streaming and real-time playback |
| flac   | Lossless audio compression            | High-quality archival storage              |
| wav    | Uncompressed audio                    | High-quality audio processing tasks        |

### Performance Recommendations

- **For lowest latency (fastest initial audio chunk):**
  - Use the `pcm` output format.

- **For general purposes:**
  - Use the `mp3` or `opus` formats for optimal balance between quality and file size.



The DeepInfra OpenAI-compatible Speech API endpoint enables users to effortlessly convert text into speech audio. This document outlines how to integrate and utilize this endpoint to quickly create speech from text inputs, leveraging various audio output formats.

## Create speech

Use the following example `curl` request to generate an audio file from your text input:

```bash
curl https://api.deepinfra.com/v1/openai/audio/speech \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "canopylabs/orpheus-3b-0.1-ft",
    "input": "The quick brown fox jumped over the lazy dog.",
    "voice": "tara",
    "response_format": "mp3"
  }' \
  --output speech.mp3
```

The API returns the generated audio file in the requested format (e.g., `mp3`, `pcm`). The example above saves the audio output directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Best for lowest latency streaming*        |
| mp3    | Compressed audio suitable for storage | General use and file distribution          |
| opus   | Compressed audio ideal for streaming  | Efficient streaming and real-time playback |
| flac   | Lossless audio compression            | High-quality archival storage              |
| wav    | Uncompressed audio                    | High-quality audio processing tasks        |

### Performance Recommendations

- **For lowest latency (fastest initial audio chunk):**
  - Use the `pcm` output format.

- **For general purposes:**
  - Use the `mp3` or `opus` formats for optimal balance between quality and file size.



The DeepInfra ElevenLabs-compatible Speech API endpoint allows users to seamlessly convert text inputs into high-quality speech audio. This document provides guidance on integrating and using this endpoint to generate realistic audio files efficiently.

Use the following examples of `py` request to generate an audio file from your text input:

## Create Speech (non-streaming)

```bash
from elevenlabs import ElevenLabs

client = ElevenLabs(
    api_key="$DEEPINFRA_TOKEN",
    base_url="https://api.deepinfra.com/",
)
client.text_to_speech.convert(
    voice_id="tara",
    output_format="mp3",
    text="The quick brown fox jumped over the lazy dog.",
    model_id="canopylabs/orpheus-3b-0.1-ft",
)
```

## Create Speech with Streaming

```bash
from elevenlabs import ElevenLabs

client = ElevenLabs(
    api_key="$DEEPINFRA_TOKEN",
)
client.text_to_speech.convert_as_stream(
    voice_id="tara",
    output_format="pcm",
    text="The quick brown fox jumped over the lazy dog.",
    model_id="canopylabs/orpheus-3b-0.1-ft",
)
```

The API returns the generated audio in the requested format, such as `mp3` or `pcm`. The example above saves the audio directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Lowest latency streaming scenarios*       |
| mp3    | Compressed audio suitable for storage | General use and easy file sharing          |
| opus   | Compressed audio ideal for streaming  | Real-time audio streaming applications     |
| flac   | Lossless audio compression            | Archival storage and high-fidelity needs   |
| wav    | Uncompressed audio                    | Audio processing and editing tasks         |

### Performance Recommendations

- **Lowest latency:** Using the `pcm` output format for streaming applications is **HIGHLY RECOMMENDED**.
- **General use:** Use `mp3` or `opus` formats for the best trade-off between quality and file size.



The DeepInfra ElevenLabs-compatible Speech API endpoint allows users to seamlessly convert text inputs into high-quality speech audio. This document provides guidance on integrating and using this endpoint to generate realistic audio files efficiently.

Use the following examples of `js` request to generate an audio file from your text input:

## Create Speech (non-streaming)

```bash
import { ElevenLabsClient } from "elevenlabs";

const client = new ElevenLabsClient({ apiKey: "$DEEPINFRA_TOKEN", base_url: "https://api.deepinfra.com/" });
await client.textToSpeech.convert("tara", {
    output_format: "mp3",
    text: "The quick brown fox jumped over the lazy dog.",
    model_id: "canopylabs/orpheus-3b-0.1-ft"
});
```

## Create Speech with Streaming

```bash
import { ElevenLabsClient } from "elevenlabs";

const client = new ElevenLabsClient({ apiKey: "$DEEPINFRA_TOKEN" });
await client.textToSpeech.convert("tara", {
    output_format: "pcm",
    text: "The quick brown fox jumped over the lazy dog.",
    model_id: "canopylabs/orpheus-3b-0.1-ft"
});
```

The API returns the generated audio in the requested format, such as `mp3` or `pcm`. The example above saves the audio directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Lowest latency streaming scenarios*       |
| mp3    | Compressed audio suitable for storage | General use and easy file sharing          |
| opus   | Compressed audio ideal for streaming  | Real-time audio streaming applications     |
| flac   | Lossless audio compression            | Archival storage and high-fidelity needs   |
| wav    | Uncompressed audio                    | Audio processing and editing tasks         |

### Performance Recommendations

- **Lowest latency:** Using the `pcm` output format for streaming applications is **HIGHLY RECOMMENDED**.
- **General use:** Use `mp3` or `opus` formats for the best trade-off between quality and file size.



The DeepInfra ElevenLabs-compatible Speech API endpoint allows users to seamlessly convert text inputs into high-quality speech audio. This document provides guidance on integrating and using this endpoint to generate realistic audio files efficiently.

Use the following examples of `curl` request to generate an audio file from your text input:

## Create Speech (non-streaming)

```bash
curl -X POST "https://api.deepinfra.com/v1/text-to-speech/tara" \
     -H "xi-api-key: $DEEPINFRA_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
  "text": "The quick brown fox jumped over the lazy dog.",
  "model_id": "canopylabs/orpheus-3b-0.1-ft",
  "output_format": "mp3",
}' --output speech.mp3
```

## Create Speech with Streaming

```bash
curl -X POST "https://api.deepinfra.com/v1/text-to-speech/tara/stream" \
     -H "xi-api-key: $DEEPINFRA_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
  "text": "The quick brown fox jumped over the lazy dog.",
  "model_id": "canopylabs/orpheus-3b-0.1-ft",
  "output_format": "pcm",
}' --output speech.pcm
```

The API returns the generated audio in the requested format, such as `mp3` or `pcm`. The example above saves the audio directly as `speech.mp3`.

### Supported Audio Formats

| Format | Description                           | Recommended Usage                          |
|--------|---------------------------------------|--------------------------------------------|
| pcm    | Raw, uncompressed audio               | *Lowest latency streaming scenarios*       |
| mp3    | Compressed audio suitable for storage | General use and easy file sharing          |
| opus   | Compressed audio ideal for streaming  | Real-time audio streaming applications     |
| flac   | Lossless audio compression            | Archival storage and high-fidelity needs   |
| wav    | Uncompressed audio                    | Audio processing and editing tasks         |

### Performance Recommendations

- **Lowest latency:** Using the `pcm` output format for streaming applications is **HIGHLY RECOMMENDED**.
- **General use:** Use `mp3` or `opus` formats for the best trade-off between quality and file size.



input

voice

Select the desired format for the speech output. Supported formats include mp3, opus, flac, wav, and pcm.

response_format

temperature

top_p

Maximum number of tokens for the generation

max_tokens

repetition_penalty

stream

The webhook to call when inference is done, by default you will get the output in the response of your inference request

canopylabs/orpheus-3b-0.1-ft

HTTP/cURL API

Input fields

`input`string

`voice`string

`response_format`string

`temperature`number

`top_p`number

`max_tokens`integer

`repetition_penalty`number

`stream`boolean

`webhook`file

Input Schema

Output Schema

Unlock the most affordable AI hosting

canopylabs/orpheus-3b-0.1-ft

HTTP/cURL API

Input fields

inputstring

voicestring

response_formatstring

temperaturenumber

top_pnumber

max_tokensinteger

repetition_penaltynumber

streamboolean

webhookfile