Welcome to the 2021 Spring Edition of the ML Reproducibility Challenge!

Supporting the Reproducibility Challenge participants

At W&B, we strongly believe that research should be reproducible and accessible, so we’re excited to support people participating in the PaperswithCode Reproducibility Challenge. We believe this initiative is very important, and we’re happy to do what we can to support participants.
Specifically, we're looking to support efforts to make sure that every NeurIPS, ACL, CVPR, ECCV, ICLR, ICML, and EMNLP paper is reproducible and that the claims made are verifiable by tracking the model performance and predictions in W&B. If you'd like to help us achieve that, please join the #ml-reproducibility channel in our Slack community, where we coordinate these efforts.
We realise that some of these papers are computationally expensive to reproduce, and we're happy to offer participants $500 per paper reproduced, as long as it meets these guidelines.

How to participate

See Reproducibility Challenge for more details.


We understand that the current landscape of deep learning and machine learning research has matured in terms of compute requirements, and that this can prevent many researchers and students from participating in the reproducibility challenge. We will award every participant who meets the above listed guidelines with $500 towards compute costs. Exact details regarding the form and method of compensation will be communicated with eligible participants.

W&B Reproducibility Grant

To facilitate maximum experimentation and thorough validation, for the Spring edition of MLRC, we are introducing the W&B Reproducibility Grant wherein we will provide $500 upfront to each selected participant for reproducing their selected papers while adhering to all the guidelines stated above.
To submit an application for the W&B Reproducibility Grant, please fill and submit this Google Form.



See Reproducibility Challenge for more details.


Join the #ml-reproducibility channel in our Slack community, where you can collaborate on paper reproductions and ask questions about reproducing models or using W&B.
Also, please see the helpful list of resources that the folks at the Reproducibility Challenge have put together.
Here are some Reproducibility Challenge Reports from the Winter 2020 edition:
  1. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks, ML Reproducibility Challenge 2020 by Diganta Misra
  2. Weights & Biases: A Reproducibility (Research) Perspective by Diganta Misra
  3. Reformer Reproducibility - Report by Morgan McGuire and the FastAI team
We will soon release the full list of accepted reports from the Winter 2020 edition.

Integrating Weights & Biases ?

W&B makes it easier to reproduce papers by allowing you to log all your runs in one centralised dashboard and see the training progress live. W&B also provides you with features that allow you to:
and much more.
To learn more, check out the following two YouTube playlists:
You can also view our docs for extensive code-based documentation, along with detailed examples.

Get started

If you're excited to start, join the #ml-reproducibility channel in our Slack community where we coordinate our reproducibility efforts. You'll also find code and analysis guidelines for ensuring that the papers are reproducible.



If you have any questions, please email growth@wandb.com.