Skip to main content
ai2-llm
Projects
olmoe
Reports
OLMoE 8x1B vs OLMo 1B
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
OLMoE 8x1B vs OLMo 1B
Niklas Muennighoff
Created on July 8
|
Last edited on August 29
Comment
eval/c4_en-validation/CrossEntropyLoss
eval/c4_en-validation/CrossEntropyLoss
200k
400k
600k
800k
1M
Step
2.6
2.7
2.8
2.9
3
olmoe-8x1b-newhp-newds-final
Run set
olmoe-8x1b-newhp-newds-final
Run set
olmoe-8x1b-newhp-newds-final
Run set
train/CrossEntropyLoss
train/CrossEntropyLoss
0
200k
400k
600k
800k
1M
Step
2.2
2.3
2.4
2.5
2.6
2.7
2.8
olmoe-8x1b-newhp-newds-final
Run set
olmoe-8x1b-newhp-newds-final
Run set
olmoe-8x1b-newhp-newds-final
Run set
Run set
3
Run set 2
Add a comment