Skip to main content
homebrewnlp
Projects
gpt
Reports
Loss/Median64 vs Steps (22/12/31 09:17:31)

Loss/Median64 vs Steps (22/12/31 09:17:31)

Lucas Nestler
Created on December 31|Last edited on December 31

3k4k5k6k7k8k9k10k20k30k40k50k60k70k80k90kStep0.70.80.91
group: param64
group: embedding-gradient-shrink
group: no-truegrad
group: fp64_loss-higher_eps
group: correct-decay
group: fp32-v12
group: remove-dead-code
group: tied-moe-modulo
Run set
60


Created with ❤️ on Weights & Biases.

https://wandb.ai/homebrewnlp/gpt/reports/Loss-Median64-vs-Steps-22-12-31-09-17-31---VmlldzozMjQxNTA5
Made with Weights & Biases. Sign up or log in to create reports like this one.