Skip to main content
eleutherai
Projects
neox
Reports
Staged Seqlen training 2
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Staged Seqlen training 2
pink = baseline blue = 512 90%, 2048 10% green = 128 50%, 2048 50%
s black
Created on May 19
|
Last edited on May 19
Comment
Section 1
validation/lm_loss_by_wallclock_time
validation/lm_loss_by_wallclock_time
2
4
6
8
10
Time (hours)
3
3.5
4
4.5
validation/lm_loss_by_tokens
validation/lm_loss_by_tokens
1G
2G
3G
4G
5G
Step
3
3.5
4
4.5
validation/lm_loss
validation/lm_loss
2k
4k
6k
8k
10k
Step
3
4
Run set
3
Run set
3
Run set
3
Run set
3
Run set
3
Add a comment