Skip to main content
ai2-llm
Projects
olmoe
Reports
Plot: Noise upcycle

Plot: Noise upcycle

Niklas Muennighoff
Created on June 21|Last edited on August 29

100k200k300k400kStep33.544.55
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-05noise Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2 Run set
Run set
11
Run set 2


Created with ❤️ on Weights & Biases.

https://wandb.ai/ai2-llm/olmoe/reports/Plot-Noise-upcycle---Vmlldzo4NDA3MzI2
Made with Weights & Biases. Sign up or log in to create reports like this one.