back to basics - static
Created on September 5|Last edited on September 11
Comment
plasticity is uniform dist, 0 to 2.
Run set
4
other lr
Run set
4
try 0 to 10.
Run set
3
try exp -2 to 4
Run set
2
set a minimum plast.
Run set
2
Static plasticity. Randomly initialized. DFA weights are normally distributed now. Sigmoid activation to avoid exploding gradients (the main problem I've been dealing with for months).
I suspect that the majority of stability is provided by the new activation. It's rather a drastic choice.
Run set
39
test the limits of how high and how wide the range plasticity can be.
Run set
48
what about a normal distribution of values?
Run set
6
chill out. no need for exp() on the plasticity values. the distribution can be less extreme.
Run set
17
Add a comment