Frozen Encoder
Created on August 16|Last edited on August 16
Comment
We tried to make the encoder frozen and experiment with adding dense layers instead. Dense layer description can be found here - https://www.notion.so/geraltofrivia783/Report-01-08-2022-09-08-2022-Frozen-BERT-Dense-Layers-53783b1e758744a99d387385b6d4a9a0
So let's see what happens if we make the encoder frozen and don't change anything else.
Run set
3
That is, it basically stops working. This
Very well, so then we add dense layers to compensate for "shared" parameters.
TODO: complete this report
Add a comment