Skip to main content
eleutherai
Projects
neox
Reports
Checkpoint loading parameter / gradient / optimizer state norms
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Checkpoint loading parameter / gradient / optimizer state norms
s black
Created on April 24
|
Last edited on April 24
Comment
Section 1
optimizer_state_norms/77_exp_avg
optimizer_state_norms/77_exp_avg
0
500
1k
1.5k
Step
0
0.02
0.04
0.06
0.08
optimizer_state_norms/34_exp_avg_sq
optimizer_state_norms/34_exp_avg_sq
0
500
1k
1.5k
Step
0
1e-8
2e-8
3e-8
4e-8
5e-8
optimizer_state_norms/89_exp_avg_sq
optimizer_state_norms/89_exp_avg_sq
0
500
1k
1.5k
Step
0
0.000002
0.000004
0.000006
0.000008
optimizer_state_norms/13_exp_avg_sq
optimizer_state_norms/13_exp_avg_sq
0
500
1k
1.5k
Step
0
1e-8
2e-8
3e-8
4e-8
optimizer_state_norms/72_exp_avg_sq
optimizer_state_norms/72_exp_avg_sq
0
500
1k
1.5k
Step
0
5e-7
0.000001
0.0000015
0.000002
optimizer_state_norms/53_exp_avg_sq
optimizer_state_norms/53_exp_avg_sq
0
500
1k
1.5k
Step
0
0.000002
0.000004
0.000006
0.000008
Run set
2
Run set
2
Add a comment