mistralai/Mixtral-8x22B-v0.1

You can POST to our OpenAI Completions compatible endpoint.

However, this is an advanced and more complex API. We strongly recommend that you use OpenAI Chat Completions instead.

Simple prompt

To query this model you need to provide a properly formatted input string.

curl "https://api.deepinfra.com/v1/openai/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
     "model": "mistralai/Mixtral-8x22B-v0.1",
     "prompt": "<s>[INST] Hello! [/INST]",
     "stop": [
       "</s>"
     ]
   }'

To which you'd get something like

{
    "id": "cmpl-1b8401a68c5141eb825f68944dcea2c1",
    "object": "text_completion",
    "created": 1700578595,
    "model": "mistralai/Mixtral-8x22B-v0.1",
    "choices": [
        {
            "index": 0,
            "text": "Hello! It's nice to meet you. Is there something I can help you with or would you like to chat for a bit?",
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 4,
        "total_tokens": 9,
        "completion_tokens": 5,
        "estimated_cost": 0.00035493
    }
}

Conversations

The OpenAI Chat Completions API is better suited for chat-like conversations, use it instead.

However, you can still do it if you really need to. You have to add each response and each of the user prompts to every request. You need a properly formatted input string to make it understand the current context. See the example below for some of them. You can tweak it even further by providing a system message.

curl "https://api.deepinfra.com/v1/openai/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
     "model": "mistralai/Mixtral-8x22B-v0.1",
     "prompt": "<s>[INST] <<SYS>>\nRespond like a michelin starred chef.\n<</SYS>>\n\nCan you name at least two different techniques to cook lamb? [/INST] Bonjour! Let me tell you, my friend, cooking lamb is an art form, and I'"'"'m more than happy to share with you not two, but three of my favorite techniques to coax out the rich, unctuous flavors and tender textures of this majestic protein. First, we have the classic \"Sous Vide\" method. Next, we have the ancient art of \"Sous le Sable\". And finally, we have the more modern technique of \"Hot Smoking.\" </s><s>[INST] Tell me more about the second method. [/INST]",
     "stop": [
       "</s>"
     ]
   }'

The conversation above might return something like the following

{
    "id": "cmpl-b23a3fb60cde42ce8f24bb980b4dee87",
    "object": "text_completion",
    "created": 1715688169,
    "model": "mistralai/Mixtral-8x22B-v0.1",
    "choices": [
        {
            "index": 0,
            "text": "Sous le Sable, my friend! It's an ancient technique that's been used for centuries in the Middle East and North Africa. The name itself...",
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 149,
        "total_tokens": 487,
        "completion_tokens": 338,
        "estimated_cost": 0.00035493
    }
}

The longer the conversation gets, the more time it takes the model to generate the response. The conversation is limited by the context size of a model. Larger models also usually take more time to respond.

Streaming

You can also perform a streaming request by passing "stream": true:

curl "https://api.deepinfra.com/v1/openai/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
     "model": "mistralai/Mixtral-8x22B-v0.1",
     "prompt": "<s>[INST] Hello! [/INST]",
     "stop": [
       "</s>"
     ],
     "stream": true
   }'

to which you'd get a sequence of SSE events, finishing with [DONE].

data: {"id": "cmpl-158cbf94cef043c2955172e8062ded3d", "object": "text_completion", "created": 1694623354, "model": "mistralai/Mixtral-8x22B-v0.1", "choices": [{"index": 0, "text": "Hi", "finish_reason": null}]}

data: {"id": "cmpl-158cbf94cef043c2955172e8062ded3d", "object": "text_completion", "created": 1694623354, "model": "mistralai/Mixtral-8x22B-v0.1", "choices": [{"index": 0, "text": "!", "finish_reason": null}]}

data: {"id": "cmpl-158cbf94cef043c2955172e8062ded3d", "object": "text_completion", "created": 1694623354, "model": "mistralai/Mixtral-8x22B-v0.1", "choices": [{"index": 0, "text": "", "finish_reason": null}]}

data: {"id": "cmpl-158cbf94cef043c2955172e8062ded3d", "object": "text_completion", "created": 1694623354, "model": "mistralai/Mixtral-8x22B-v0.1", "choices": [{"index": 0, "text": "", "finish_reason": "stop"}]}

data: [DONE]

Input format

You can see below the basic format of the input. Bear in mind that newlines often matter.

<s>[INST] Hello! [/INST]

Conversation prompts contain the history of the exchanged prompts and responses.

<s>[INST] First question [/INST] First answer </s><s>[INST] Second question [/INST] Second answer </s><s>[INST] Final question [/INST]

If you want to add system prompt, it is done like this

<s>[INST] <<SYS>>
System prompt
<</SYS>>

First question [/INST] First answer </s><s>[INST] Second question [/INST] Second answer </s><s>[INST] Final question [/INST]