Skip to main content
homebrewnlp
Projects
gpt
Reports
TGAdam vs Baseline

TGAdam vs Baseline

Lucas Nestler
Created on December 5|Last edited on December 5

5k10k15k20kStep12345
group: adam-squarev4
group: adam-square-smallclip
group: tied-moe-modulo
5G10G15G20G25GSpeed/Tokens Seen0.80.850.90.95
group: tied-moe-modulo
Run set
7


Created with ❤️ on Weights & Biases.

https://wandb.ai/homebrewnlp/gpt/reports/TGAdam-vs-Baseline--VmlldzozMDg4NjIz
Made with Weights & Biases. Sign up or log in to create reports like this one.