Frozen Encoder
Created on August 16|Last edited on August 16
Comment
We tried to make the encoder frozen and experiment with adding dense layers instead. Dense layer description can be found here - https://www.notion.so/geraltofrivia783/Report-01-08-2022-09-08-2022-Frozen-BERT-Dense-Layers-53783b1e758744a99d387385b6d4a9a0
So let's see what happens if we make the encoder frozen and don't change anything else.
That is, it basically stops working. This
Very well, so then we add dense layers to compensate for "shared" parameters.
TODO: complete this report
Add a comment