Tootsie 8B cooldown v1 ("monumental-jellyfish")
See https://github.com/stanford-crfm/marin/issues/600 for narrative
Created on March 12|Last edited on May 12
Comment
Big Idea:
- Core tootsie DCLM mix to 3.7 T tokens
- Cooldown on Dolmino HQ data (without synth math or Flan) to 4.8T tokens
NB that the final run (lime green, phase 3) starts from a slightly earlier checkpoint from the red run since I messed the red one up.
Lineage Runs
This set of panels contains runs from a private project, which cannot be shown in this report
Run set
6662
Add a comment