Skip to main content
eleutherai
Projects
neox
Reports
Adam, mp=2, pp=2, ZeRO-1, 12 layers, 1-node

Adam, mp=2, pp=2, ZeRO-1, 12 layers, 1-node

Shivanshu Purohit
Created on February 24|Last edited on February 24

Section 1


200400600800Step1015202530
0200400600800Step678910
200400600800Step1e+92e+93e+94e+9
Run set
8



Run set
8


Created with ❤️ on Weights & Biases.

https://wandb.ai/eleutherai/neox/reports/Adam-mp-2-pp-2-ZeRO-1-12-layers-1-node--Vmlldzo0ODcyMTk
Made with Weights & Biases. Sign up or log in to create reports like this one.