How to deploy Databricks Dolly v2 12b, instruction tuned casual language model.

Databricks Dolly is instruction tuned 12 billion parameter casual language model based on EleutherAI's pythia-12b. It was pretrained on The Pile, GPT-J's pretraining corpus. databricks-dolly-15k open source instruction following dataset was used to tune the model.

Getting started

First install the deepctl command line tool.

curl https://deepinfra.com/get.sh | sh

Login to DeepInfra (using your GitHub account)

deepctl auth login

This will take you to the browser to login in DeepInfra using your GitHub account. When you are done, come back to the terminal.

Deployment

Deploying the databricks/dolly-v2-12b model is as easy as running the following command:

deepctl deploy create -m databricks/dolly-v2-12b

This command will setup everything for you and you can just use the model right away.

Inference

You can use it with either our rest API or our deepctl command line too. Here is how to use it with the command line tool:

deepctl infer -m databricks/dolly-v2-12b -i prompt="Who is Elvis Presley?"

We charge per inference request execution time, $0.0005 per second. Inference runs on Nvidia A100 cards. To see the full documentation of how to call this model checkout out the documentation page:

deepctl model info -m databricks/dolly-v2-12b

If you want a list of all the models you can use on DeepInfra, you can run:

deepctl model list

If you have any question, just reach out to us on our Discord server.