Skip to main content
eleutherai
Projects
mesh-transformer-jax
Reports
Position encoding shootout

Position encoding shootout

Ben Wang
Created on April 18|Last edited on August 25

Section 1


1k2k3k4k5k6k7k8k9k10k20kStep23
1k2k3k4k5k6k7k8k9k10k20kStep2345
1k2k3k4k5k6k7k8k9k10k20kStep2345
20k40k60k80kStep05000100001500020000
Run set
3



Run set
3



Run set
3


Created with ❤️ on Weights & Biases.

https://wandb.ai/eleutherai/mesh-transformer-jax/reports/Position-encoding-shootout--Vmlldzo2MTg2MzY
Made with Weights & Biases. Sign up or log in to create reports like this one.