Skip to main content
marin-community
Projects
marin
Reports
Qwen 3 speedruns (QK Norm, Muon)

Qwen 3 speedruns (QK Norm, Muon)

Calvin Xu
Created on September 12|Last edited on September 12
Model configs match the previous Llama Muon speedruns: https://github.com/marin-community/marin/pull/1405

Section 1


05k10k15k20kStep4681012
qwen3_300m_muon_4096-ee4f99
qwen3_130m_muon_4096-04770b
qwen3_520m_muon_4096-361875
qwen3_1_2b_muon_4096-d117d2
Run set
4



Run set
4



Run set
4


Created with ❤️ on Weights & Biases.

https://wandb.ai/marin-community/marin/reports/Qwen-3-speedruns-QK-Norm-Muon---VmlldzoxNDM1NjY5MA
Made with Weights & Biases. Sign up or log in to create reports like this one.