Generating High-quality Images with SD.Next, HuggingFace Diffusers and W&B
A walkthrough of using SD.Next (Advanced webUI for Stable Diffusion) for generating high-quality images using HuggingFace Diffusers and managing experiments with W&B.
Created on October 2|Last edited on January 8
Comment
SD.Next, is a versatile deep learning tool originating from the Stable Diffusion WebUI project by Automatic1111, that has undergone significant evolution while maintaining a commitment to incorporating essential features from its predecessor. Its feature set includes:
- Optimized processing with the latest features included in the PyTorch 2.x releases
- Support for multiple backends (original and Diffusers)
- Compatibility with various diffusion models
- Cross-platform compatibility
- Improved prompt parsing
- Streamlined workflows with built-in queue management
- Advanced metadata handling
- Robust logging
- Modern localization
- A hint engine
- A modernized user interface with automatic updates.
SDNext's two primary backends, original and Diffusers, allow seamless switching to cater to user needs. The Original backend ensures compatibility with existing functionality and extensions, supporting all Stable Diffusion family of models, while the Diffusers backend expands capabilities by incorporating the new Diffusers implementation by HuggingFace and supports advanced models such as Stable Diffusion XL.
In this article, we will explore:
- How to set up SD.Next in your system
- How to integrate and use the Diffusers library by HuggingFace with SD.Next
- How to use Weights & Biases to manage our experiments and ensure reproducibility for our generations.
Here's what we'll be covering:
Table of Contents
Setting up SD.NextExplore Your CreationsComparing Different Sampling MethodsStable Diffusion 1.5Kandinsky 2.1Stable Diffusion XLConclusionRecommended Reading
And since this is a GenAI article, we know what you want upfront: some good-looking images to get you started.
Setting up SD.Next
Explore Your Creations
Let's explore our SD.Next creations using Weights & Biases. We will also analyze the images against different sampling methods and different models.
Comparing Different Sampling Methods
Stable Diffusion 1.5
The Stable-Diffusion-v1-5 checkpoints were released by runwayml. These checkpoints were initialized with the weights of the Stable-Diffusion-v1-2 checkpoint and subsequently fine-tuned on 595k steps at a resolution of on the Laion Aesthetics v2.5+ and 10% dropping of the text-conditioning to improve classifier-free guidance sampling.
Experiments using Stable Diffusion 1.5 checkpoints on SD.Next
3
Base: SD 1.5 with default Sampler
6
Kandinsky 2.1
Kandinsky 2.1 model inherits best practices from DALL-E 2 and Latent diffusion while introducing some new ideas. It uses the CLIP model as a text and image encoder, and diffusion image prior (mapping) between latent spaces of CLIP modalities. This approach increases the visual performance of the model and unveils new horizons in blending images and text-guided image manipulation. The model card can be found here on Huggingface.
Experiments using Kandinsky 2.1 checkpoints on SD.Next
3
Stable Diffusion XL
Stable Diffusion XL 1.0 is the latest model in the Stable Diffusion family of text-to-image models from Stability AI. Stable Diffusion XL enables us to create elaborate images with shorter descriptive prompts, as well as generate words within images. The model is a significant advancement in image generation capabilities, offering enhanced image composition and face generation that results in stunning visuals and realistic aesthetics.
SDXL consists of an ensemble of experts pipeline for latent diffusion: In the first step, the base model is used to generate (noisy) latents, which are then further processed with the refinement model specialized for the final denoising steps. Note that the base model can be used as a standalone module. The model card can be found here on huggingface.
Examples of images generated using Stable Diffusion XL 1.0 + Refiner Checkpoints using SD.Next
2
Conclusion
- In this article, we discussed and explored the setting up and usage of SDNext, an advanced UI for stable diffusion.
- We learned how we can use the Weights & Biases tool and Diffusers library by HuggingFace with SDNext.
- We briefly explored the effect of the sampling method used against a single stable diffusion model
- We then explored variants of Stable Diffusion: SD1.5, SDXL, and Kandinsky in standalone and refiner-assisted pipeline mode. We then compared these model stacks for the image generation task against the same prompt.
Recommended Reading
A Guide to Using Stable Diffusion XL with HuggingFace Diffusers and W&B
A comprehensive guide to using Stable Diffusion XL (SDXL) for generating high-quality images using HuggingFace Diffusers and managing experiments with Weights & Biases
DeepFloydAI: A New Breakthrough in Text-Guided Image Generation
In this article, we explore DeepFloydAI — an AI Research Band which is working with StabilityAI to make AI open again.
Improving Generative Images with Instructions: Prompt-to-Prompt Image Editing with Cross Attention Control
A primer on text-driven image editing for large-scale text-based image synthesis models like Stable Diffusion & Imagen
Making My Kid a Jedi Master With Stable Diffusion and Dreambooth
In this article, we'll explore how to teach and fine-tune Stable Diffusion to transform my son into his favorite Star Wars character using Dreambooth.
Running Stable Diffusion on an Apple M1 Mac With HuggingFace Diffusers
In this article, we look at running Stable Diffusion on an M1 Mac with HuggingFace diffusers, highlighting the advantages — and the things to watch out for.
How To Train a Conditional Diffusion Model From Scratch
In this article, we look at how to train a conditional diffusion model and find out what you can learn by doing so, using W&B to log and track our experiments.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.