Reproducing Machine Learning Experiments with W&B
Learn to use W&B to reproduce machine learning experiments. A sample from the free MLOps certification course from Weights & Biases.
Created on December 22|Last edited on December 28
Comment
Reproducibility is a key aspect of good machine learning practice, but it can be challenging to ensure that the results of your experiments are the same across different environments and team members. In this video from our MLOps course, we show you how to use Weights & Biases, GitHub, and conda virtual environments to reproduce machine learning experiments with ease.
By logging your experiments in Weights & Biases and version controlling your code with GitHub, you can easily track and share your work with your team. And by using conda virtual environments, you can ensure that the environment in which your experiments are run is consistent, helping to eliminate potential issues with package versions and dependencies. If you're looking to improve the reproducibility of your machine learning experiments, be sure to watch this video.
Transcript (from Whisper)
First thing first, let's reproduce Darek's results.
We can see the version of Python that was used, the git repo where the code leaves, and the state of the git repo. You can also see hardware information like the GPU that was used for training.
Scrolling down, you get the molds artifacts inputs and outputs that the run consumes and produces. If you scroll down, you see the output parameters. We also see the output parameter values that were used for training.
We are almost there for reproducing our experiment.
There is an extra goody from Weights & Biases for free. It automatically detects the Python packages that were used during training, creating this handy conda environment file that you can use to recreate the same environment that was used.
Every single Python package is pinned, so you will reproduce exactly what the machine was using at the moment of training. You can click to download the file, and now we are ready to reproduce our experiment.
Let's switch to a terminal. Once in the terminal, the first thing I will do is check that the git commit match. You can do this by using git log, and check that the hash is the same that the reported by Weights & Biases.
We are okay. I have already created a folder called lesson2 where we will put over work. I have already also copied the conda environment file.
One extra step is needed, we will need to copy the baseline. Let's take a look at the file.
This will create an environment called course, and we will install every single Python package that is in this list. Everything of the package is pinned, so we will get the exact same version.
Let's go ahead and create the environment. You can do this by calling conda m create and passing the environment file. This will take a while, so let's fast forward to the environment already created.
Let's activate the environment. Let's bring the baseline file so we can reproduce the Rx experiment.
Now we will launch Jupyter and run this notebook. Once inside Jupyter, I can open the baseline file, and I will run a single style in order. To do so, you can call run, restart kernel and run all.
We are producing this baseline run is a crucial step when you are taking over someone's work. Let's make sure that the notebook is running correctly. The way it says run is created, and training should start soon.
Let's see the progress bar increasing. We love progress bars. Okay, the small is training fine. Let's fast forward to the end of the training. You can scroll down to the end of the notebook and click on the run link.
This will bring you to the workspace. Okay, we get the metrics we are expecting and table with the mall predictions. We can go back to the workspace and compare this run against the baseline run.
As you can see, both runs overlay perfectly. Let's make sure of that. Let's switch to table view. The metrics look exactly the same.
Now we are in a good spot to start refactoring baseline and improving the results.
Add a comment
Thomas Capelle •
This is really 😎
Reply
Iterate on AI agents and models faster. Try Weights & Biases today.