Skip to main content
ai2-llm
Projects
olmoe
Reports
OLMoE Norm Reorder, Tok, Docmask

OLMoE Norm Reorder, Tok, Docmask

Niklas Muennighoff
Created on October 21|Last edited on October 21

5k10k15k20k25k30kStep2.533.544.55
olmoe-8x1b-newhp-newds-cx5-fine1-docmask-8k Run set
olmoe-8x1b-newhp-newds-cx5-fine1-newtok Run set
olmoe-8x1b-newhp-newds-cx5-fine1-docmask Run set
olmoe-8x1b-newhp-newds-cx5-fine1-normreorder Run set
olmoe-8x1b-newhp-newds-cx5-fine1-docmask Run set
olmoe-8x1b-newhp-newds-cx5-fine1 Run set
5k10k15k20k25k30kStep23456
olmoe-8x1b-newhp-newds-cx5-fine1-docmask-8k Run set
olmoe-8x1b-newhp-newds-cx5-fine1-newtok Run set
olmoe-8x1b-newhp-newds-cx5-fine1-docmask Run set
olmoe-8x1b-newhp-newds-cx5-fine1-normreorder Run set
olmoe-8x1b-newhp-newds-cx5-fine1-docmask Run set
olmoe-8x1b-newhp-newds-cx5-fine1 Run set
Run set
6
Run set 2


Created with ❤️ on Weights & Biases.

https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-Norm-Reorder-Tok-Docmask--Vmlldzo5ODEzODYz
Made with Weights & Biases. Sign up or log in to create reports like this one.