Skip to main content
marin-community
Projects
marin
Reports
Big Tootsies
Log in
Sign up
Share
Comment
Star
Big Tootsies
David Leo Wright Hall
Created on January 29
|
Last edited on May 12
Comment
Included are runs for:
Tootsie 8b
(including all phases/cooldowns in the main line)
Tootsie 13b
Tootsie 24b (called 22b)
Tootsie 70b
Tootsie 32b
Section 1
log(train/loss) vs log(tokens)
log(train/loss) vs log(tokens)
100M
10G
1T
throughput/total_tokens
2
3
4
5
6
7
8
9
10
Marin 32B run1
tootsie-8b-sensible-starling
llama-13b-tootsie-ema-mk3
llama-22b-tootsie-ema-mk5
llama-8b-tootsie-adept-phoenix
llama-8b-tootsie-phase3
llama-13b-tootsie-ema-mk2
llama-22b-tootsie-ema-mk2
llama-8b-tootsie-phase2
llama-real-70b-tootsie
llama-13b-tootsie-dummy-testing-214059
llama-22b-tootsie-dummy-testing-373d53
llama-8b-tootsie-0.001-19ad63
Run set
13
Add a comment