Fine-tuning Stable Diffusion with Dreambooth

Getting started

Before you start you will need a deepctl command line tool.

Fine-tuning

Dreambooth is a fine-tuning technique that lets Stable Diffusion mimic patterns provided in the subject's instance data. You will need an archive containing 10-20 images of an instance. We recommend resizing images to 512x512, resolution which Stable Diffusion 1.5 was trained in. For instance prompt you can use 3 letter sequence (e.g. "nkb"). For max_train_steps we recommend something between 800 and 2000. You have to pass output_model_name, to use later for inference on your fine-tuned model.

deepctl infer -m 'deepinfra/dreambooth'\
              -i 'instance_prompt=["a photo of nkb person"]'\
              -i 'class_prompt=["a photo of a person"]'\
              -i instance_data=@/path/to/instance_data.zip\
              -i max_train_steps=800\
              -i output_model_name=your_github_username/your_custom_model_name
              -i webhook=https://example.com/webhook

Make sure your model name starts with your Github username before '/'. The result will be delivered in a webhook or in a response if you haven't provided one.

Using the fine-tuned model

You can use it with either our rest API or our deepctl command line. Here is how to use it with the command line tool:

deepctl infer -m your_github_username/your_custom_model_name\
              -i prompt="a colorful painting of zwt person"\
              -o images=zwt_gta5.png

To see the full documentation of how to call this model checkout out the documentation page:

deepctl model info -m your_github_username/your_custom_model_name

If you have any question, just reach out to us on our Discord server.