Skip to main content
ai2-llm
Projects
olmoe
Reports
Plot: LN
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Plot: LN
Niklas Muennighoff
Created on June 24
|
Last edited on August 29
Comment
*final-norm = RMSNorm ; *final = non-parametric norm
eval/c4_en-validation/CrossEntropyLoss
eval/c4_en-validation/CrossEntropyLoss
20k
40k
60k
80k
100k
120k
140k
Step
2.6
2.8
3
3.2
olmoe17-8x1b-final-norm
Run set
olmoe17-8x1b-final-norm
Run set
olmoe17-8x1b-final
Run set
olmoe17-8x1b-final
Run set
train/CrossEntropyLoss
train/CrossEntropyLoss
20k
40k
60k
80k
100k
120k
140k
Step
4
6
8
10
olmoe17-8x1b-final-norm
Run set
olmoe17-8x1b-final-norm
Run set
olmoe17-8x1b-final-norm
Run set
olmoe17-8x1b-final
Run set
olmoe17-8x1b-final
Run set
olmoe17-8x1b-final
Run set
Run set
9
Run set 2
Add a comment