Make-A-Scene: Meta's Generative AI Guided By Sketches & Text
Meta AI's new generative AI project lets users create AI-generated images from text prompts, directed and composed based on a sketch input.
Created on July 16|Last edited on July 16
Comment
Meta AI has revealed a new generative AI for the creation of images based on text input, but this time there's something that makes it stand out from the rest. Unlike DALL·E, Craiyon, or any of the other models that are hot right now, Make-A-Scene takes not only a text prompt but also a free-form sketch provided by the user.
This new piece to the puzzle gives the user significantly more manual control over the composition of the generated image.
With other generative AI models, the final composition of an image was completely up to the AI; Most of them were also notorious for having relative scale and positional issues. Ask the model to make an image of a giant rhinoceros beetle fighting a full-size airplane and it might make them both tiny or include a second beetle into the mix. By introducing a sketch element to the model input, users can pre-define what goes where.
This sketching aspect is reminiscent of NVIDIA's GauGAN which proved the concept a while ago. Though it was limited by pre-defined paint brushes corresponding to certain concepts, it could create stunning images.
Make-A-Scene takes away those limitations; The concepts which fill the image are totally controllable by the user with their text input, and the sketch only helps show where things should go.
Can I use Make-A-Scene?
Unfortunately Make-A-Scene is not open-source, and neither is it open-access quite yet (unlike many of Meta's other recent AI releases). Like all the big- image generation AIs, this one's being kept in-house and with limited access for now. Internal employees at Meta may have some access to it for testing, and certain artists have been granted access to the AI as well, but the general public will have to wait. For how long? Who knows.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.