Skip to main content
eleutherai
Projects
neox
Reports
1-bit adam, mp=1, pp=2, ZeRO-1, 12 layers, 1 node

1-bit adam, mp=1, pp=2, ZeRO-1, 12 layers, 1 node

Shivanshu Purohit
Created on February 24|Last edited on February 24

Section 1


5001k1.5k2kStep5101520
05001k1.5k2kStep2468101214
5001k1.5k2kStep1e+92e+93e+94e+9
Run set
8



Run set
8


Created with ❤️ on Weights & Biases.

https://wandb.ai/eleutherai/neox/reports/1-bit-adam-mp-1-pp-2-ZeRO-1-12-layers-1-node--Vmlldzo0ODcxMjU
Made with Weights & Biases. Sign up or log in to create reports like this one.