Skip to main content
jadens_team
Projects
hebby
Reports
mega clip sweep

mega clip sweep

Jaden
Created on April 30|Last edited on May 6
a big, big sweep. Lots of hyperparameters, over 500 combinations, actually. Different values of learning rate, plast clip (high plasticity), forget rate, and grad clip.

Computing group metrics from first 36 groups
020406080Step00.10.20.30.40.50.6
learning_rate: 0.00001, plast_clip: 100000
learning_rate: 0.00001, plast_clip: 1000
learning_rate: 0.00001, plast_clip: 100
learning_rate: 0.0001, plast_clip: 100000
learning_rate: 0.00001, plast_clip: 10000
learning_rate: 0.0001, plast_clip: 10000
learning_rate: 0.0001, plast_clip: 100
learning_rate: 0.001, plast_clip: 100000
learning_rate: 0.0001, plast_clip: 1000
learning_rate: 0.001, plast_clip: 10000
learning_rate: 0.001, plast_clip: 1000
learning_rate: 0.001, plast_clip: 100
Computing group metrics from first 43 groups
020406080Step0.80.912
learning_rate: 0.00001, plast_clip: 100000
learning_rate: 0.00001, plast_clip: 1000
learning_rate: 0.00001, plast_clip: 100
learning_rate: 0.0001, plast_clip: 100000
learning_rate: 0.00001, plast_clip: 10000
learning_rate: 0.0001, plast_clip: 10000
learning_rate: 0.0001, plast_clip: 100
learning_rate: 0.001, plast_clip: 100000
learning_rate: 0.0001, plast_clip: 1000
learning_rate: 0.001, plast_clip: 10000
learning_rate: 0.001, plast_clip: 1000
learning_rate: 0.001, plast_clip: 100
Run set
504
Run set 2
504
Run set 3
504
Run set 4
504



Run set
504
Run set 2
504
Run set 3
504
Run set 4
504



Run set
504
Run set 2
504
Run set 3
504
Run set 4
103



Run set
504
Run set 2
504
Run set 3
504
Run set 4
504



Created with ❤️ on Weights & Biases.

https://wandb.ai/jadens_team/hebby/reports/mega-clip-sweep--VmlldzoxMjU1NTc4MQ
Made with Weights & Biases. Sign up or log in to create reports like this one.