DALL·E Mini – Generate Images From Any Text Prompt
In this article, we explore Craiyon (also known as DALL·E mini), which is an artificial intelligence model that generates images from any prompt you give.
Created on May 16|Last edited on January 28
Comment
💡
Craiyon (previously DALL·E mini) is an AI model that can generate images from any prompt you give. The model is still training and keeps improving over time.
In this article, we explain how to use it, how it works, and share some examples.
Table of Contents
How To Use It
How Does It Work?
History
Craiyon/DALL·E mini is an attempt at reproducing those results with an open-source model.
The Simple Explanation
The model is trained by looking at millions of images from the internet with their associated captions. Over time, it learns how to draw an image from a text prompt.
Some concepts are learned from memory as they may have seen similar images. However, it can also learn how to create unique images that don't exist, such as "the Eiffel tower is landing on the moon," by combining multiple concepts together.
Several models are combined together to achieve these results:
- an image encoder that turns raw images into a sequence of numbers with its associated decoder
- a model that turns a text prompt into an encoded image
- a model that judges the quality of the images generated for better filtering
The Technical Explanation
What Is DALL·E Mega?
DALL·E mega is the largest version of DALL·E mini trained and contains several improvements:
- optimizer updated to Distributed Shampoo, which proved to be more efficient following comparison of different optimizers
- a new architecture based on NormFormer and GLU variants following comparison of transformer variants, including DeepNet, Swin v2, NormFormer, Sandwich-LN, RMSNorm with GeLU/Swish/SmeLU
The model is still training.
Is the Model Available?
Yes!
While the model is still training, the latest version is regularly pushed:
It can also be downloaded programmatically with:
import wandbrun = wandb.init()artifact = run.use_artifact('dalle-mini/dalle-mini/mega-1:latest')artifact_dir = artifact.download()
Results From the Latest Model
Resources
Acknowledgements
- 🤗 Hugging Face for organizing the FLAX/JAX community week, with a stellar coordination and support from Patrick von Platen and Suraj Patil
- Authors of original DALL·E mini: Boris Dayma, Suraj Patil, Pedro Cuenca, Khalid Saifullah, Tanishq Abraham, Phúc Lê Khắc, Luke Melas, Ritobrata Ghosh
- Contributions and support from Rohan Anil, Phil Wang, Katherine Crowson, the Gradio team and the LAION & EleutherAI communities
Related Posts
Craiyon Image Generator From Text: Online Demo (formerly DALL-E Mini)
Have questions about Craiyon, the text-to-image generation tool powered by AI? We're here to answer them.
DALL·E Mega - Training Journal
The Infrastructure Behind Serving DALL·E Mini
In this article, we explore the technology underpinning Dall-E mini and set up a high-load backend infrastructure on Google TPUs.
Using OpenAI's GPT-3 to Generate 'Doctor Who' Episode Synopses
You can now track your GPT-3 fine-tunes with Weights & Biases! Here's how to do just that, but with more David Tennant than you were otherwise expecting
Subscribe to our Newsletter for more ML News like DALLE.
Add a comment
Duncan Paolo Kmet •
Very Awesome BOIIIIIISSS
Reply
Sully Dawood •
Anyone have insights on commercial use?
Reply
Liam Ames •
Each time I try to use it I receive a message saying that there is too many traffic!
1 reply
Frank W. •
Amazing! One question: Are the generated pictures free to use?
2 replies
Mohammad Bilal Shaikh •
excellent work mates!
Reply
James Webb •
Amazing work!
Reply
Rana Saleh •
Great job
Reply
Tags: Articles, Experiment, DALL-E, GAN, Intermediate, Image Generation, Large Models, GenAI, Computer Vision
Iterate on AI agents and models faster. Try Weights & Biases today.