Tootsie 32B
Created on April 30|Last edited on May 12
Comment
This is running on preemptible v5p compute. v5ps are so fast.
We use a very large batch size for this run compared to our normal. We started out at 32Mi tokens but are now doing 24Mi tokens.
We used zloss from the start and use nemotron-cc + starcoder + proofpile as the main mix.
We show the 70b for comparison.
Section 1
Run set
2
Add a comment