can be compared with laion2B run B/32 there https://wandb.ai/rom1504/eval_openclip/reports/B-32-2B--VmlldzoyNDkwNDMy

3 runs: a) bs 360k B/32 (pretty similar to bs90k) laion2B b) bs 90k B/32 laion1B (very similar to laion2B) b) bs90k B/32 with patch dropout laion2B

930M data scale, same as laion-a

Training B/32 with roberta base encoder on laion2B-en for 12B samples seen (same as original clip training schedule)