Skip to main content
W&B will undergo scheduled maintenance on Saturday, Nov 29 from 6–9 PM PT. The Models, Inference, and Serverless APIs may be briefly unavailable during this time. Check our status page for updates: https://status.wandb.com

wrap

Created on November 15|Last edited on November 29





New setting: unregularized


unreg
5


New setting: Regularized parameter scaling

# (750, 8, 3e-3, 0.8, "150m4k"),
# (750, 16, 3e-3, 1.6, "300m4k"),
# (750, 8, 1e-3, 3.2, "600m4k"),
# (750, 8, 1e-3, 3.2, "1_4b4k"),
# (750, 8, 1e-3, 3.2, "1_5b4k"),


Tuning weight decay
1362




New setting: wrap


baselines
543
x1
543
x2
543
x4
543
x8
560
x16
560


New setting: sd (repeated data)


baselines
542
x1
560
2x
560
4x
560
8x
560
16x
560


New setting: sd (fresh data, from optimal regularized teacher)


baselines
542
x1
560
2x
560
4x
560
8x
560
16x
560


New setting: sd (fresh data, from 1-ens teacher)


baselines
542
x1
560
2x
560
4x
560
8x
560
16x
560


New setting: synthetic mix (sd + wrap, 16 cpr)


baselines
542
x1
560
2x
560
4x
560
8x
560
16x
560



New setting: synthetic method compare


baselines
810
sd cpr 16
1
sd cpr 200
1
wrap cpr 16
1
symx cpr 16
2
sdn cpr 200
1





new setting: normal ensembles


Run set
2117



150m
557
300m
557
600m
557
1.5b
557