If you’re a developer, chances are you’ve heard of Dreambooth. But what is it, and how can you use it to your advantage?
Dreambooth is a model that allows you to fine tune an existing generative AI model like Stable Diffusion so that it can be used to generate custom artwork around the subject in the images that you fine tune the model with. Put another way it allows you to train the model using your own photos so that you can use prompts to create artwork based on the photos you fine tuned the model with.
Let’s drive into how you can utilize Dreambooth in your own projects from within Delphi.
Table of Contents
What is Dreambooth?
Dreambooth is a unique deep learning model that stands out amongst the crowd. The purpose of Dreambooth is to take existing text-to-image models and further enhance them with fine-tuning techniques. Text to image AI generated art is an impressive new form of digital art created with the help of text-based artificial intelligence and computers. Created by a computer algorithm, text to image AI generated art takes text input from the user and then produces a visual artwork to represent the text.
By using natural language processing and machine learning, these algorithms can create stunning visuals that reflect the text–from abstract graphics to ultra-realistic images. The technology is still evolving, but text to image AI generated art offers an exciting new way for artists and designers to bring words and visuals together in creative ways.
What is Stable Diffusion?
Generative text to image art is a form of art in which a computer program is used to generate an image from text. There are a number of different solutions available to do this including Stable Diffusion, DALL-E, and Midjourney. It uses artificial intelligence and natural language processing techniques to create an image based on the text input provided by the user. Generative text to image art has become increasingly popular in 2022 and 2023, as it allows for more creative expression and exploration of different visual styles.
Stable Diffusion itself is a revolutionary deep learning text-to-image model designed to bring your wildest imaginations to life. With advanced language technology and AI capabilities, a user can create stunning visuals from any text prompt in just minutes. From inpainting and outpainting to generating image-to-image translations, Stable Diffusion is one way to help you explore more possibilities!
Read the full blog post about Stable Diffusion.
How can I use Dreambooth in Delphi?
There are a number of different API providers where you can run Dreambooth through a simple API. With Delphi’s TRESTClient it is super simple to hook up to their APIs and train a model of yourself with Dreambooth. In this blog post we’ll talk about how to do this on Replicate.com. Replicate’s Dreambooth API can be found here https://replicate.com/replicate/dreambooth/api and they also have a blog post which explains it in more detail. Their API even supports img2img with the fine tuned Dreambooth models you create. Img2img is a method where you can provide your own image as a seed or source for the model to follow in addition to the text prompt when creating a new image.
At the time of this writing, it costs around $2.50 to fine tune a custom model based on Stable Diffusion 1.5 or 2.1. It also takes around 20-25 minutes. Once the model is trained you can generate new images using your fine-tuned models at around 10 seconds per image. Each image generation costs less than a cent at the time of this writing.
Other Dreambooth API providers have similar pricing but may offer bulk pricing as well.
Should I run Dreambooth locally or via an API?
Stable Diffusion and Dreambooth have pretty steep GPU and VRAM requirements. Stable Diffusion generally needs a GPU with at least 8GB of VRAM while Dreambooth might need 24GB or VRAM. These are some heavy hardware requirements which makes using an API if you don’t have such hardware an easy choice.
How can I use the Dreambooth desktop client built in Delphi?
The Dreambooth desktop client comes with instructions that walk you through the process. The first thing you need is an API key from Replicate.com. You can sign into their site using your Github account. Once you have the API key you can place it in the client on the Config tab.
Step #1, Step #2
In step #1 and step #2, the client allows you to select a ZIP file containing JPEG images and upload it to Replicate. If you already have a ZIP with the images you want to use hosted somewhere else you can skip this step and just enter the URL to the ZIP file. Replicate.com will only accept JPEG images and it will automatically crop them to 512×512 and center the 512×512 around any face that it detects in the image.
Step #3
In step #3 you provide the name of the new model you want to create. This is in the format of username/modelname so for example fmxexpress/mymodel.
Step #4
In step #4, once this is done then the https://dreambooth-api-experimental.replicate.com/v1/trainings API end point can be used with the URL for the ZIP for that you uploaded to fine tune the Dreambooth model. Training takes around 20 minutes.
Step #5
In step #5, you can use the verify button to check and see if the model has finished generating.
Once you have trained your model you can go to the Generation tab to start generating txt2img images using your new fine tuned model. During this step you will want to use “cjw” in your text prompt as the keyword to render the subject from your fine tuning into the image being generated. For example, you could write “photo of cjw”.
Ready to get started generating fine tuned art using Dreambooth, Stable Diffusion, and Delphi?
Dreambooth can easily be incorporated into your pipeline using Delphi and an API. There are many ways to use Dreambooth, but the most important thing is to get started and to have fun. With a little bit of practice, you’ll be able to create amazing, fine-tuned art using Stable Diffusion. So what are you waiting for? Get started with Delphi today!
Download an open source Dreambooth Desktop Client built in Delphi.
Embarcadero is also running a digital fan art and AI fan art contest right now over on the Delphi reddit. You can create images and submit them over on the Delphi reddit to be eligible for prizes. Find out more about the Delphi Fan Art contest.