Skip to main content

griffin arch on fineweb-1M

comparing the effect of tokenizers on small griffin arch on a sample of the fineweb dataset.
Created on April 28|Last edited on April 28

tokenizers:

  • claude3 as hf gpt2 tokenizer: link
  • llama-3 tokenizer: link


experiment data


0.20.40.60.8train/epoch6
0.20.40.60.8train/epoch0.2
0.20.40.60.8train/epoch1.21.251.31.35
Run set
2



Run set
2



Run set
2



Run set
2