Skip to main content
eleutherai
Projects
neox
Reports
Regular adam, mp=1, pp=2, ZeRO-1, 12 layers, 1 node

Regular adam, mp=1, pp=2, ZeRO-1, 12 layers, 1 node

Shivanshu Purohit
Created on February 24|Last edited on February 24

Section 1


5001k1.5k2kStep51015
05001k1.5k2kStep567891011
5001k1.5k2kStep1e+92e+93e+94e+9
Run set
8



Run set
8


Created with ❤️ on Weights & Biases.

https://wandb.ai/eleutherai/neox/reports/Regular-adam-mp-1-pp-2-ZeRO-1-12-layers-1-node--Vmlldzo0ODczNDg
Made with Weights & Biases. Sign up or log in to create reports like this one.