BlenderBot 3: Meta AI's 175 Billion Parameter Publicly Available Chatbot
Blender Bot 3 is Meta AI's newest public access model release. This model is a chatbot built on OPT-175B, and shows improvements across the board compared to other AI chatbots.
Created on August 5|Last edited on August 5
Comment
Meta AI has come around yet again with another publicly available model, this time a 175 billion parameter model called BlenderBot 3. This model works as a chatbot that pulls its knowledge from the internet so that it can chat about virtually any topic. It was built so that it learn and take feedback from it's interactions with users, so it's constantly learning and evolving to get better based on real human interaction. You can play with BlenderBot 3 by clicking here.
What is BlenderBot 3?
BlenderBot 3 is a chatbot powered by machine learning, coming as the third installment of Meta AI's series of BlenderBot chatbot models. It's built with the newest techniques, an internet-searching capability, and an always-learning user feedback system.
BlenderBot 3's focus is how it learns from user interaction; With the BlenderBot 3 demo (currently available only in the US), users will be able to provide feedback to the model's responses, and all interaction will be collected to be made into a large real-world interaction dataset at a later date. Interactions in the wild will allow for better model growth compared to premade or AI-generated datasets because of the sheer unpredictability that can come from real-world interaction.
As for actual user interaction, BlenderBot 3, like it's earlier versions, maintains a long-term memory bank and exhibits a level of personality and empathy to ensure valuable conversation can be made with human interactors. You can chat with BlenderBot 3 with this link: https://blenderbot.ai/
How does BlenderBot 3 work?
BlenderBot 3 is built on OPT-175B, Meta AI's recent large language model. This means it's around 58 times larger than it's predecessor BlenderBot 2. This newer chatbot model also implements an improved SeeKeR architecture for internet information lookup.
Being based on OPT-175B gives BlenderBot 3 an instant edge on previous chatbots; With a massive and proven powerful large language model, BlenderBot 3 can produce perfectly coherent and readable text output. The language produced by BlenderBot 3 is natural sounding, and holds good conversation.
Then in comes the SeeKeR architecture - by grabbing information from the internet in real time, BlenderBot 3 can engage in just about any topic imaginable. The model architecture is further enhanced with Director and Fits to facilitate the real-time human feedback portion of it's always-learning approach.
To ensure that only well-intentioned human feedback is used, a seperate algorithm was trained to distinguish between helpful and problem responses. This will hopefully mitigate any users deliberately trying to cause harm to BlenderBot 3's learning process.
To further ensure productive conversation is had, and to avoid problematic or tough subjects which users might attempt to force the chatbot into, techniques were developed to teach BlenderBot 3 to steer the conversation away from difficult subjects whenever possible.
How does BlenderBot 3 compare to other AI chatbots?
BlenderBot 3 was found to evaluate better than any other chat bot in just about every metric - more factually correct compared to BlenderBot 2, more up-to-date and specific than GPT3, and clear improvements on a range of conversational benchmarks.
While it's better on all the benchmarks, it does still face a small amount of incorrectness and nonsensical output. In safety and bias tests, the raw model before safety mitigations was on par with other models, but with pre-trained models like OPT-175B is showed improvements.
Where can I get BlenderBot 3 for myself?
Like many of Meta AI's other recent AI projects, everything to do with BlenderBot 3 is readily available for download. You can find everything on Github or ParlAI, where you'll get links for downloading the various model sizes, datasets, and other things.
BlenderBot 3 comes in a few sizes: 3 billion parameters, 30 billion parameters, and 175 billion parameters. The smaller two models are freely available for download, but you'll have to request access to the 175 billion parameter version. You can check here for how to get your hands on the models.
You can find information on the datasets used to train the model here. You can check back on this page in the future for when the real-world interaction dataset is released.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.