Skip to main content
ai2-llm
Projects
olmoe
Reports
Plot: Init

Plot: Init

Niklas Muennighoff
Created on June 23|Last edited on August 29

50k100k150k200k250k300kStep34567
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-init Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-scratch Run set
50k100k150k200k250k300kStep46810
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-init Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-scratch Run set
Run set
2
Run set 2


Created with ❤️ on Weights & Biases.

https://wandb.ai/ai2-llm/olmoe/reports/Plot-Init--Vmlldzo4NDIzMzM5
Made with Weights & Biases. Sign up or log in to create reports like this one.