universal decay
Created on August 21|Last edited on August 22
Comment
Allow the plasticity to reach as low as -plast_clip. in this case, that's 1e2.
Run set
5
Back to the lower cap for plasticity being 1e-15. Really just test if the decay value can be higher than plasticity.
Run set
2
have the decay match the lr.
Run set
4
compare
Run set
11
I went on to define a larger gap between the lr and the decay.
try:
- very small layer size - vsmalllayer
- much larger layer size - vlargelayer
- larger decay rate - largedecay
- incorporate negative plasticity again. - negplast
Run set
12
- tinker with a significantly smaller plasticity
Run set
9
Add a comment