We use essential cookies to make our site work. With your consent, we may also use non-essential cookies to improve user experience and analyze website traffic…

🚀 New models by Bria.ai, generate and edit images at scale 🚀

Lzlv model for roleplaying and creative work
Published on 2023.11.02 by Nikola Borisov
Lzlv model for roleplaying and creative work

Recently an interesting new model got released. It is called Lzlv, and it is basically a merge of few existing models. This model is using the Vicuna prompt format, so keep this in mind if you are using our raw API. If you are using the OpenAI API we will take care of the prompt formatting for you.

The Lzlv model also got lots of attention on Reddit, and it was tagged as top choice for the 70b category here: https://www.reddit.com/r/LocalLLaMA/comments/17fhp9k/huge_llm_comparisontest_39_models_tested_7b70b/ It seems to have the right balance between creativity and coherence and is a good choice for roleplay use-cases. You can quickly try it out using our web chat ui. If you like the results you can easily integrate it into your application using our APIs.

How to use it

As always you can easily access the model using our LLM APIs. For example using our OpenAI compatible API:

import openai

# Point OpenAI client to our endpoint, get api key: https://deepinfra.com/dash/api_keys
openai.api_key = "<YOUR DEEPINFRA TOKEN>"
openai.api_base = "https://api.deepinfra.com/v1/openai"

# Your chosen model here
MODEL_DI = "lizpreciatior/lzlv_70b_fp16_hf"
chat_completion = openai.ChatCompletion.create(
    model=MODEL_DI,
    messages=[{"role": "user", "content": "Hello there"}],
    stream=True,
    max_tokens=100,
)

# print the chat completion
for event in chat_completion:
    print(event.choices)
copy

HTTP API

You can also use our lower level HTTP API directly. This will allow you to build your own prompt, and integrate it easily into any application.

curl -X POST \
    -d '{"input": "Tell me a joke."}'  \
    -H "Authorization: bearer <YOUR DEEPINFRA API TOKEN>"  \
    -H 'Content-Type: application/json'  \
    'https://api.deepinfra.com/v1/inference/lizpreciatior/lzlv_70b_fp16_hf'
copy

We have both streaming and non streaming APIs available and you can find more details here: https://deepinfra.com/lizpreciatior/lzlv_70b_fp16_hf/api

Cost

This is a 70b model, and it is quite expensive to run. Typically, you need at least couple A100 80GB GPUs to run it, but to get the best performance you need at least 4 high end GPUs. Getting this hardware and setting everything up is quite expensive and time-consuming. Our APIs however are the most cost-effective way to run this model. For 70b models we charge 0.7 USD per 1M input tokens, and 0.9 USD per 1M output tokens. The biggest saving is that you only pay for usage and not the time.

If you need any help, just reach out to us on our Discord server.

Related articles
Deploy Custom LLMs on DeepInfraDeploy Custom LLMs on DeepInfraDid you just finetune your favorite model and are wondering where to run it? Well, we have you covered. Simple API and predictable pricing. Put your model on huggingface Use a private repo, if you wish, we don't mind. Create a hf access token just for the repo for better security. Create c...
Search That Actually Works: A Guide to LLM RerankersSearch That Actually Works: A Guide to LLM RerankersSearch relevance isn’t a nice-to-have feature for your site or app. It can make or break the entire user experience. When a customer searches "best laptop for video editing" and gets results for gaming laptops or budget models, they leave empty-handed. Embeddings help you find similar content, bu...
Enhancing Open-Source LLMs with Function Calling FeatureEnhancing Open-Source LLMs with Function Calling FeatureWe're excited to announce that the Function Calling feature is now available on DeepInfra. We're offering Mistral-7B and Mixtral-8x7B models with this feature. Other models will be available soon. LLM models are powerful tools for various tasks. However, they're limited in their ability to per...