Skip to main content
eleutherai
Projects
mesh-transformer-jax
Reports
Normalization shootout v2

Normalization shootout v2

Ben Wang
Created on March 23|Last edited on May 4

Section 1


020k40k60k80k100k120k140kStep46810
medium_scalenorm
medium_scalenorm_bias
medium_rmsnorm_bias
medium_rmsnorm
medium_layernormnobias
medium_layernorm
020k40k60k80k100k120k140kStep46810
Run set
6



Run set
6


Created with ❤️ on Weights & Biases.

https://wandb.ai/eleutherai/mesh-transformer-jax/reports/Normalization-shootout-v2--Vmlldzo1NTE4MTQ
Made with Weights & Biases. Sign up or log in to create reports like this one.