Falcon-40B is a 40B parameters causal decoder-only model built by TII and trained on 1,000B tokens of RefinedWeb enhanced with curated corpora. It is made available under the Apache 2.0 license.

You can use cURL or any other http client to run inferences:

```bash
curl -X POST \
    -d '{"input": {"prompt": "..."}}'  \
    -H 'Content-Type: application/json'  \
    -H "Authorization: bearer $(deepctl auth token)"  \
    'https://api.deepinfra.com/v1/inference/$MODEL'
```

The inpu/output is compatible with the cog interface.


You can use our command-line tool [deepctl](/docs/getting-started) to run
inferences:

```bash
deepctl infer \
    -m '$MODEL'  \
    -i prompt=...  \
    -i max_length=...  \
    -i top_k=...
```


prompt

maximum number of tokens to generate, -1 for unlimited

max_length

sample from top k options for each token, 0 to disable

top_k

sample from top options totalling at least p (0-1), 1.0 to disable

top_p

how much to prioritize better matching tokens, 1.0 is neutral, > 1.0 choose worse-prob tokens

tiiuae/falcon-40b-q51

Input

Output