Jbloom's workspace
Runs
8
Name
8 visualized
State
Notes
User
Tags
Created
Runtime
Sweep
b_dec_init_method
cached_activations_path
checkpoint_path
context_size
d_in
d_sae
dataset_path
dead_feature_estimation_method
dead_feature_threshold
dead_feature_window
device
dtype
expansion_factor
feature_reinit_scale
feature_sampling_window
hook_point
hook_point_layer
is_dataset_tokenized
l1_coefficient
log_to_wandb
lr
lr_warm_up_steps
model_name
n_batches_in_buffer
n_checkpoints
resample_batches
run_name
seed
store_batch_size
tokens_per_buffer
total_training_tokens
train_batch_size
use_cached_activations
use_ghost_grads
wandb_log_frequency
wandb_project
details/current_learning_rate
details/n_training_tokens
losses/ghost_grad_loss
losses/l1_loss
losses/mse_loss
losses/overall_loss
metrics/CE_loss_score
metrics/ce_loss_with_ablation
Finished
-
jbloom
7h 5m 10s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/kknavokh
128
768
98304
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
128
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
98304-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
145.90849
0.0094026
0.021075
0.9816
11.28348
Finished
-
jbloom
4h 23m 18s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/u4xlxwrh
128
768
49152
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
64
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
49152-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
142.653
0.010028
0.021441
0.97904
11.28348
Finished
-
jbloom
3h 16m 17s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/cbvk8gtc
128
768
24576
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
32
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
24576-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
143.17953
0.011309
0.022763
0.97781
11.28348
Finished
-
jbloom
2h 29m 40s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/83dxxo6a
128
768
12288
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
16
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
12288-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
151.68957
0.012714
0.024849
0.97321
11.28348
Finished
-
jbloom
2h 16m 34s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/jy7r81cs
128
768
6144
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
8
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
6144-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
161.63934
0.01481
0.027741
0.96579
11.28348
Finished
-
jbloom
2h 12m 48s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/v5wlsd29
128
768
3072
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
4
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
3072-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
166.80302
0.017018
0.030362
0.95279
11.28348
Finished
-
jbloom
2h 3m 11s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/wv0n1cz3
128
768
1536
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
2
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
1536-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
169.89429
0.020593
0.034185
0.93199
11.28348
Finished
-
jbloom
1h 58m 35s
-
geometric_median
activations/Skylion007_openwebtext/gpt2-small/blocks.8.hook_resid_pre
checkpoints/ap977vbz
128
768
768
Skylion007/openwebtext
no_fire
1.0000e-8
5000
cuda
torch.float32
1
0.2
1000
blocks.8.hook_resid_pre
8
false
0.00008
true
0.0004
5000
gpt2-small
128
10
1028
768-L1-8e-05-LR-0.0004-Tokens-3.000e+08
42
32
67108864
300000000
4096
false
true
100
mats_sae_training_gpt2_feature_splitting_experiment
0.0004
299827200
0
158.01578
0.025005
0.037646
0.89606
11.28348
1-8
of 8