Getting Started

deepctl is our an open source command line tool to manage your deployments, run inference, get help on how to call endpoints, pass parameters, tail your inference logs.

Installation

Let's start by installing the deepctl command line tool.

Linux, MacOS

curl https://deepinfra.com/get.sh | sh

Sign in

You need a GitHub account to use deepctl. This command will open a login page in the browser. When you finish the login flow on the web come back to the terminal.

deepctl auth login

List supported models

List all the models already available on Deep Infra. You can also see them on our website here.

deepctl model list

Create a model deployment

Pick your favorite model from the list and let's deploy it. Here is how to deploy openai/whisper the popular voice to text model from OpenAI.

deepctl deploy create -m openai/whisper-small

It only takes few seconds, and now we are ready to use it.

Output:

deployed openai/whisper-small None -> DOtZX3MHtc7Ltqgu
status: running
deployment DOtZX3MHtc7Ltqgu --> running

Inference. Call the model.

Easiest way to try the model is to use deepctl command line tool. You pass audio file as an input and get the text as an output.

deepctl infer -m 'openai/whisper-small' -i audio=@/path/to/hello_world.mp3

Output:

{
  "text": "Hello World",
  ...
}

Inference API.

To see the full list of inference arguments and response fields, you can use models page on our website here. You also have access to this in the terminal using the deepctl model info command.

deepctl model info -m openai/whisper-small

Output:

...
CURL invocation:

 curl -X POST \
    -H "Authorization: bearer $AUTH_TOKEN"  \
    -F audio=@my_voice.mp3  \
    'https://api.deepinfra.com/v1/inference/openai/whisper-small'

deepctl invocation:

 deepctl infer \
    -m 'openai/whisper-small'  \
    -i audio=@my_voice.mp3
...

When using HTTP inference api make sure to pass your AUTH_TOKEN header. You can get the auth token using deepctl auth token command.

export AUTH_TOKEN=`deepctl auth token`
curl -X POST \
    -H "Authorization: bearer $AUTH_TOKEN"  \
    -F audio=@/path/to/hello_world.mp3  \
    'https://api.deepinfra.com/v1/inference/openai/whisper-small'