Skip to main content
mesolitica
Projects
llama2-embedding-600m-contrastive
Reports
Contrastive embedding loss

Contrastive embedding loss

191M, 349M, 600M, 1B and 2B on contrastive loss 8k context length.
Husein Zolkepli
Created on December 1|Last edited on December 5

020k40k60k80k100kStep0.010.020.030.040.05
Group 191M
Group 2B
Group 1B
Group 349M
Group 600M
600M
19
1B
87
191M
3
349M
17
2B
41


Created with ❤️ on Weights & Biases.

https://wandb.ai/mesolitica/llama2-embedding-600m-contrastive/reports/Contrastive-embedding-loss--Vmlldzo2MTM4ODE0
Made with Weights & Biases. Sign up or log in to create reports like this one.