Training and Fine-tuning LLMs

Training and Fine-tuning LLMs
Learn to harness the power of LLMs with our comprehensive course. Discover the importance and history of LLMs, explore their architecture, training techniques, and fine-tuning methods. Gain hands-on experience with practical recipes from Jonathan Frankle (MosaicML), and other industry leaders,and learn cutting-edge techniques like LoRA and Prefix Tuning. Perfect for machine learning engineers, data scientists, researchers, and NLP enthusiasts. Stay ahead of the curve and become an expert in LLMs.
4 Hours
Free

Learnings & outcomes

  • Learn the fundamentals of large language models
  • Curate a dataset and establish an evaluation approach
  • Master training and fine-tuning techniques

Curriculum

  • Foundations
  • Evaluation
  • Data
  • Training & Fine-tuning Techniques
  • Course Assessment & Next Steps
In partnership with
In partnership with
Very interesting course! I liked the course overall, it contains a lot of information about the LLM. I'm an NLP researcher and I want to know more about the details of LLMs, this course definitely helps me. Unfortunately, I was unable to run the code locally due to an out of memory issue.
Great
Amazing Free Course. I learned a lot of invaluable insights from this course. In particular, Jonathan Fankle from MosaicML gives excellent tips about training LLMs. Two things that I agree with the most are: - Start small, and - Don't start training if you don't have an evaluation dataset. Thanks a lot for offering this amazing course!
It was really great. thanks for this comprehensive course
I am grateful for the insightful LLM tuning training you provided. Your expertise and teaching have enhanced my understanding . Thank you for your generosity and dedication to sharing knowledge.
It is great to hear from leaders in the field with so much real world experiences with these models.Loved learning from Jonathan Frankle
More of it! What a great course. For anybody telat8vrly new to deep learning and NLP, I promise you that you won't be overwhelmed. Great content.
Course instructors

Darek Kłeczek

Darek Kłeczek is a Machine Learning Engineer at Weights & Biases, where he leads the W&B education program. Previously, he applied machine learning across supply chain, manufacturing, legal, and commercial use cases. He also worked on operationalizing machine learning at P&G. Darek contributed the first Polish versions of BERT and GPT language models and is a Kaggle Competition Grandmaster.
MLE Weights & Biases

Jonathan Frankle

Jonathan Frankle is Chief Scientist at MosaicML, where he leads the company's research team toward the goal of developing more efficient algorithms for training neural networks. In his PhD at MIT, he empirically studied deep learning with Prof. Michael Carbin, specifically the properties of sparse networks that allow them to train effectively (his "Lottery Ticket Hypothesis" - ICLR 2019 Best Paper). In addition to his technical work, he is actively involved in policymaking around challenges related to machine learning. He will be joining the computer science faculty at Harvard in the fall of 2023. He earned his BSE and MSE in computer science at Princeton and has previously spent time at Google Brain, Facebook AI Research, and Microsoft as an intern and Georgetown Law as an “Adjunct Professor of Law.”
Chief Scientist MosaicML

Weiwei Yang

Weiwei Yang is a Principal Software Development Engineering Manager leading an applied machine learning team at Microsoft Research (MSR). Her research interests lie in resource-efficient learning methods inspired by biological learning. Weiwei aims to democratize AI by addressing sustainability, robustness, scalability, and efficiency in ML. She has successfully applied her research to organizational science, countering human trafficking, and stabilizing energy grids. Before joining Microsoft Research, Weiwei worked extensively in Bay Area startups and managed several engineering teams.
Principal SDE Manager Microsoft Research

Mark Saroufim

Mark Saroufim is an engineer on the PyTorch Applied AI team where he maintains and contributes to a wide variety of PyTorch projects. His interests are in broadly improving the performance and usability of real world ML systems. Mark will be setting up the evaluation pipeline, answering technical questions from the community as well reproducing winning models.
PyTorch Engineer Meta
Explore our other courses