Skip to main content
mesolitica
Projects
pretrain-mamba-1.4b
Reports
Mamba 1.4B vs Mistral 1.1B and 3B

Mamba 1.4B vs Mistral 1.1B and 3B

Comparing the loss between Mamba 1.4B vs Mistral 1.1B on 90B tokens.
Husein Zolkepli
Created on December 8|Last edited on December 9

05001k1.5k2k2.5kStep246810121416
Group mistral-1.1b
mamba-1.4b
6
mistral-1.1b
52
mistral-3b
43


Created with ❤️ on Weights & Biases.

https://wandb.ai/mesolitica/pretrain-mamba-1.4b/reports/Mamba-1-4B-vs-Mistral-1-1B-and-3B--Vmlldzo2MjA1MDc1
Made with Weights & Biases. Sign up or log in to create reports like this one.