DeepMind's Gato: A General Intelligence Model Able To Perform Over 600 Tasks
DeepMind has revealed a 1.2 billion parameter generalized model named Gato. This model is able to perform over 600 tasks, from playing video games to image classification.
Created on May 13|Last edited on May 14
Comment
Taking a queue from large-scale language models, Gato is a brand new Transformer model just revealed by DeepMind. Gato is capable of performing over 600 tasks on a wide variety of subjects, from playing Atari videogames and controlling robot arms to image classification and conversational language processing.
Researchers at DeepMind have released a paper describing the ins and outs of Gato, which is accessable here: https://arxiv.org/abs/2205.06175
How does DeepMind's Gato work?
Gato is a Transformer model, a popular model architecture especially for tasks like natural language processing. Transformer models start with tokenization, a process for preparing the input data before it's sent through the layers of neurons that make up the model. Because Gato is a generalized model, there's many different types of input data it has to contend with, including text, image, environmental variables, and more.
Gato was trained on over 20 datasets on a wide range of subjects, covering a variety of environmental control tasks like videogames and robot arm manipulation to language processing and image captioning. Here's the datasets used in training:
After the input data is processed, it's sent through Gato's 1.2 billion parameter internals to arrive at an appropriate output. Data processed by Gato is processed on the same collection of parameters, so there's no sneaky task type detection going on here causing the model to choose a specific part of itself to process the data; All neurons are firing when data goes through Gato.
How successful is Gato at the tasks it can perform?
It'd be nice to jump the gun and call Gato a revolutionary step in generalized AI, but we're gonna have to take a closer look before making any claims like that.
While Gato is able to perform over 600 tasks, just about all of them could easily be performed better by other models which are focused on individual tasks. This is especially noticable when looking at Gato's attempts at image classification.
While Gato is generally able to get the vibe of a picture across a few attempts, it's in no way consistent or even correct most of the time. Problems like this persist across many tasks, where Gato performs sub-optimally compared to models trained for specific tasks. Resolution to these issues such as model size increases and incorporating reinforcement learning are posited by the researchers in the paper.
Gato's real achievment is it's broad range of skills, rather than it's performance on any single one thing; A real Jack of all trades, master of none. Though, to Gato's credit, it's still able to beat average human scores for 23 Atari games.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.