Skip to main content
sparsify
Projects
gpt2
Reports
GPT2 layerwise vs e2e SAEs (currently only layerwise)
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
GPT2 layerwise vs e2e SAEs (currently only layerwise)
Dan Braun
Created on March 28
|
Last edited on March 28
Comment
Section 1
performance/eval/difference_ce_loss and loss.sparsity.coeff v. sparsity/eval/L_0/blocks.2.hook_resid_pre
performance/eval/difference_ce_loss and loss.sparsity.coeff v. sparsity/eval/L_0/blocks.2.hook_resid_pre
10
20
30
40
50
60
70
80
90
loss.sparsity.coeff
100
200
300
400
500
600
700
sparsity/eval/L_0/blocks.2.hook_resid_pre
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
performance/eval/difference_ce_loss
Run set
299
Fixed dictionary size, vary sparsity penalty
Run set
299
Add a comment