Skip to main content
ai2-llm
Projects
olmoe
Reports
Plot: FP32 LBL

Plot: FP32 LBL

Niklas Muennighoff
Created on June 24|Last edited on August 29

20k40k60k80k100kStep2.833.23.43.6
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2 Run set
020k40k60k80k100kStep2.22.32.42.52.62.72.8
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2-lblfp32 Run set
olmoe17-8x1b-fullshard-swiglu-wrapb-k2 Run set
Run set
10
Run set 2


Created with ❤️ on Weights & Biases.

https://wandb.ai/ai2-llm/olmoe/reports/Plot-FP32-LBL--Vmlldzo4NDMxNDA4
Made with Weights & Biases. Sign up or log in to create reports like this one.