Published on 2023.03.17 by Nikola Borisov
Flan-UL2 is probably the best open source model available right now for chatbots. In this post we will show you how to get started with it very easily. Flan-UL2 is large - 20B parameters. It is fine tuned version of the UL2 model using Flan dataset. Because this is quite a large model it is not easy to deploy it on your own machine. If you rent a GPU in AWS, it will cost you around 1080 per month. Using DeepInfra model deployments you only pay for the inference time, and we do not charge for cold starts. Our pricing is 0.0001 per token generated by Flan-UL2.
Also check out the model page https://deepinfra.com/google/flan-ul2. You can
run inferences, check the docs/API for running inferences via curl
.
First, you'll need to get an API key from the DeepInfra dashboard.
You can deploy the google/flan-ul2 model easily through the web dashboard or API. The model will be automatically deployed when you first make an inference request.
You can use it with our REST API. Here's how to use it with curl:
curl -X POST \
-d '{"prompt": "Hello, how are you?"}' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer YOUR_API_KEY" \
'https://api.deepinfra.com/v1/inference/google/flan-ul2'
To see the full documentation of how to call this model, check out the model page on the DeepInfra website or the API documentation.
If you want a list of all the models you can use on DeepInfra, you can visit the models page on our website or use the API to get a list of available models.
There is no easier way to get started with arguably one of the best open source LLM. This was quite easy right? You did not have to deal with docker, transformers, pytorch, etc. If you have any question, just reach out to us on our Discord server.