deepctl - command line tool

deepctl is the official command line tool by DeepInfra. It is an advanced tool. You can use it to do many tasks without the need of additional dependecies or the web UI. You can also integrate it with other command line tools.

Installation

Let's start by installing the deepctl command line tool.

Linux, MacOS

curl https://deepinfra.com/get.sh | sh

Download from Github

You can also download deepctl from Releases.

Sign in

You need a Github account to use deepctl. This command will open a Github Sign-on page in your browser.

deepctl auth login

Auth tokens

DeepInfra requires an API token to access any of its APIs. You can view yours with

deepctl auth token

or in the dashboard.

List your deployments

Deployments of AI models are created automatically for you the first time you run an inference request. You can see all your deployments with

deepctl deploy list

Output:

[
  {
    "created_at": "2023-01-26T19:33:23",
    "deploy_id": "DpM4BkrjEspUwmTa",
    "fail_reason": "",
    "model_name": "openai/whisper-small",
    "status": "running",
    "task": "automatic-speech-recognition",
    "updated_at": "2023-01-26T19:33:23"
  }
]

Monitor Logs

To see the logs for a particular deployment use its deploy_id

deepctl log -f DpM4BkrjEspUwmTa

Delete deployment

Deleteing a deployment is trivial using its deploy_id

deepctl deploy delete DpM4BkrjEspUwmTa

More information

You can always use

deepctl help

to view more information on any command.

Check version and update

deepctl version check
deepctl version update

Inference

You can use deepctl to do inference. Below you can find a few basic examples. For more details and more examples you should visit the API section of the models that you would like to run inference on.

Text Generation

Text generation with meta-llama/Meta-Llama-3-8B-Instruct

deepctl infer \
    -m 'meta-llama/Meta-Llama-3-8B-Instruct'  \
    -i 'input="<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nHello!<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"'  \
    -i 'stop=["<|eot_id|>"]'

Embeddings

Creating vector embeddings with BAAI/bge-large-en-v1.5

deepctl infer \
    -m 'BAAI/bge-large-en-v1.5'  \
    -i 'inputs=["I like chocolate"]'

Image Generation

Image generation with stabilityai/stable-diffusion-2-1

deepctl infer \
    -m 'stabilityai/stable-diffusion-2-1'  \
    -i 'prompt=A photo of a prince riding a horse on Mars.'

Image Classification

Image classification with google/vit-base-patch16-224

deepctl infer \
    -m 'google/vit-base-patch16-224'  \
    -i image=@image.jpg