Skip to main content
mesolitica
Projects
pretrain-mistral-3b
Reports
Pretrain Larger Malaysian Mistral

Pretrain Larger Malaysian Mistral

1.1B, 3B and 5B LLM trained on 90B tokens, 312GB JSONL file.
Husein Zolkepli
Created on November 27|Last edited on December 1

02k4k6k8kStep2468101214
Group 3B
5B
5
3B
43
1.1B
52


Created with ❤️ on Weights & Biases.

https://wandb.ai/mesolitica/pretrain-mistral-3b/reports/Pretrain-Larger-Malaysian-Mistral--Vmlldzo2MDkyOTgz
Made with Weights & Biases. Sign up or log in to create reports like this one.