The world of AI creation seems like it’s jumped forward decades in only a couple of years because of tools like ChatGPT and Midjourney. However, these user-friendly apps have long-term foundations in technology and research that were carefully put together by programmers over time. That yields incredible tricks, whether it’s ChatGPT imitating a president or Midjourney creating surreal artwork.

A powerful example of these technologies, and a current competitor to Midjourney, is called Stable Diffusion. It’s an image generator that’s light enough to fit on your smartphone with a single app download but powered by intricate AI technology. Here’s everything you should know about it.

What is Stable Diffusion?

It’s a deep learning AI model that turns text into images. In other words, you type in a description like, “4K painting of cat wearing tuxedo serving hummingbirds in dainty glasses cinematic style,” and Stable Diffusion does its best to generate an image based on your requirements. The more text and details you provide or tweak, the more accurately Stable Diffusion can generate an image. A variety of image generators, such as Dreamer, use the Stable Diffusion model. It’s especially good at rendering realistic faces and images (our example above poses some problems for Stable Diffusion because of its vague nature).

Like other image-focused AI models, Stable Diffusion does a lot of its work by closely analyzing a mind-boggling number of images and interpreting that visual information as data it can use. However, a couple of things make Stable Diffusion different. First, its developers offer it as SDXL, an open source version that everyone can use, with a greater ability to tweak the model and generally peek under the hood. Second, it’s designed to be anonymous and private to use from the get-go. And to talk about the third thing, we need to do a deeper tech dive.

Where does the name Stable Diffusion come from?

Stable Diffusion is named that way because it’s a latent diffusion model. A diffusion model is a type of generative model that’s trained to produce stuff. In this case, images. It’s known as diffusion because the mathematical model adds errors or “noise” to training images to make them harder for the AI to identify correctly. The image is eventually diffused into meaningless noise, similar to the way a cube of sugar diffuses into a glass of tea. But along the way, the AI learns how to identify images with the least number of clues possible and eventually restore or create images of its own.

The “latent” part of this diffusion model means that the images Stable Diffusion analyzes are first compressed into a different form of data. That means Stable Diffusion doesn’t look at images pixel by pixel. Instead, it compresses them to numbers that represent image data and crunches those numbers. That saves an incredible amount of space, so Stable Diffusion runs more efficiently.

How did Stable Diffusion first begin?

While Stable Diffusion saw its broad release in 2022, it was a project for many years before that. The AI model began as a project between the University of Munich, Runway, and the CompVis Group, powered by funding from a variety of other organizations. The Stability AI project had its first official funding round in late 2022.

Can Stable Diffusion access any artwork online for analysis?

Yes. Stable Diffusion training is based on the LAION 5b nonprofit crawler, the tool used to collect images for analysis. This crawler grabs images from all kinds of sources. Artists cannot agree to join the program or block their work from being used to train Stable Diffusion. Not everyone is okay with that. In 2023, three artists joined together to sue the organization for copyright infringement, although that case is still ongoing.

Is Stable Diffusion free to use?

Stable Diffusion comes in various forms, and most are free to use thanks to the open source nature of the software. You can start with the Dreamer app or the more focused Anime Master that also uses the technology. The Stable Diffusion website also has a premium subscription to access more features.

Can people use Stable Diffusion to create deepfakes?

Stable Diffusion is particularly good at rendering faces and similar person-oriented details. That raises the risk of deepfakes, or fake images created to fool people into thinking they are real people doing real things. Deepfakes are the subject of deep suspicion because of their potential for blackmail, misinformation, propaganda, and other problems.

We’re still not sure how issues behind deepfakes will work out, but today’s easily accessed image generators online aren’t good enough to produce usable deepfakes. Stable Diffusion can come close in the right circumstances, which is causing some concern. Companies are more likely to use Stable Diffusion to create fake people for marketing uses, but it’s a can of worms barely opened. Stable Diffusion uses an implicit AI license that bans the use of created images for discrimination, crimes, or harm to prevent future deepfake attempts.

Image generation with Stable Diffusion

Now you know the basics behind Stable Diffusion and its image-generating powers. If you have a minute, you can visit the site and write a prompt of your own. Unlike some generators, Stable Diffusion allows you to choose from specific styles and tweak the parameters of the AI, so it’s a good intro tool for those anxious to learn. You may also want to stop by our guide on how to identify AI-generated images, an increasingly important skill in today’s online world.