Tokenizer Comparison
Fineweb-EDU 1.4b
Tokenizers:
* llama2 (~32k)
* llama3 (~128k)
* neox (~50k)
Created on November 19|Last edited on May 12
Comment
Standard Panels
This set of panels contains runs from a private project, which cannot be shown in this report
This set of panels contains runs from a private project, which cannot be shown in this report
Add a comment