Stella Biderman: How EleutherAI Trains and Releases LLMs
On this episode of Gradient Dissent, we’re joined by Stella Biderman, Lead Scientist at Booz Allen Hamilton and Executive Director at ElutherAI. Stella and Lukas discuss EleutherAI's origin and future, LLM similarities and differences, choosing models, reinforcement learning, pre-training/fine-tuning, GPU selection, differences from other LLM companies, interpretability, memorization importance, and public access.
Created on May 12|Last edited on May 12
Comment
About this episode
On this episode, we’re joined by Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton.
EleutherAI is a grassroots collective that enables open-source AI research and focuses on the development and interpretability of large language models (LLMs).
We discuss:
- How EleutherAI got its start and where it's headed.
- The similarities and differences between various LLMs.
- How to decide which model to use for your desired outcome.
- The benefits and challenges of reinforcement learning from human feedback.
- Details around pre-training and fine-tuning LLMs.
- Which types of GPUs are best when training LLMs.
- What separates EleutherAI from other companies training LLMs.
- Details around mechanistic interpretability.
- Why understanding what and how LLMs memorize is important.
- The importance of giving researchers and the public access to LLMs.
Thanks For Listening!
Connect With Stella
Connect With ElutherAI
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.