Skip to main content
homebrewnlp
Projects
gpt
Reports
MoE vs Dense

MoE vs Dense

Lucas Nestler
Created on November 1|Last edited on November 1

20G40G60G80G100GSpeed/Tokens Seen0.70.750.80.850.90.95
group: tied-moe-modulo
Run set
13


Created with ❤️ on Weights & Biases.

https://wandb.ai/homebrewnlp/gpt/reports/MoE-vs-Dense--VmlldzoyODg3OTM1
Made with Weights & Biases. Sign up or log in to create reports like this one.