Skip to main content
eleutherai
Projects
neox
Reports
Snapshot Feb 24 2021, 2:9am
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Snapshot Feb 24 2021, 2:9am
GPT2-XL, regular adam, mp=2, pp=2, ZeRO-1
Shivanshu Purohit
Created on February 23
|
Last edited on February 23
Comment
Section 1
iteration_time
iteration_time
200
400
600
800
1k
1.2k
Step
20
30
40
50
60
lm loss
lm loss
0
200
400
600
800
1k
1.2k
Step
5
6
7
8
9
10
loss_scale
loss_scale
200
400
600
800
1k
1.2k
Step
1e+9
2e+9
3e+9
4e+9
Run set
16
Run set
16
Add a comment