collingray

Collingray's group workspace

Group: multilayer_pythia_32x_v2

1-8

of 8

Tags

Notes

Author

collingray

State

Finished

Start time

May 9th, 2024 11:02:54 PM

Runtime

49m 51s

Tracked hours

49m 48s

Run path

collingray/multilayer_sae/n1a51tem

Linux-6.7.8-arch1-1-x86_64-with-glibc2.39

Python version

3.11.4

Command

<python with no main file> -f /home/collin/.local/share/jupyter/runtime/kernel-d27eadd9-c126-4118-b018-515b0e7a0108.json

System Hardware

CPU count	6
Logical CPU count	12
GPU count	1
GPU type	NVIDIA GeForce RTX 4090

W&B CLI Version

0.16.1

Group

multilayer_pythia_32x_v2

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 12 keys
- act_renorm_scale:
  1
- act_renorm_type:
  "none"
- beta1:
  0.9
- beta2:
  0.999
- decoder_scale:
  true
- input_scale:
  false
- l1_warmup_pct:
  null
- l1_weight:
  0.5179474949836731
- latent_p:
  1
- lr:
  0.001
- lr_decay_pct:
  0.2
- lr_warmup_pct:
  0

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 49 keys
- avg_fvu:
  4.6093058586120605
- avg_l0:
  94.94084930419922
- ▶
  feature_density.bins:[] 65 items
- ▶
  feature_density.values:[] 64 items
- layer_0.avg_fvu:
  0.39501121640205383
- layer_0.avg_l0:
  140.92913818359375
- ▶
  layer_0.feature_density.bins:[] 65 items
- ▶
  layer_0.feature_density.values:[] 64 items
- layer_0.l1:
  0.0002231597900390625
- layer_0.loss:
  0.46875
- layer_0.mse:
  0.46875
- layer_1.avg_fvu:
  0.7439377307891846
- layer_1.avg_l0:
  148.1503448486328
- ▶
  layer_1.feature_density.bins:[] 65 items
- ▶
  layer_1.feature_density.values:[] 64 items
- layer_1.l1:
  0.00000652670860290527
- layer_1.loss:
  0.8828125
- layer_1.mse:
  0.8828125
- layer_2.avg_fvu:
  14.747082710266112
- layer_2.avg_l0:
  123.94107055664062
- ▶
  layer_2.feature_density.bins:[] 65 items
- ▶
  layer_2.feature_density.values:[] 64 items
- layer_2.l1:
  0.00000275671482086182
- layer_2.loss:
  17.5
- layer_2.mse:
  17.5
- layer_3.avg_fvu:
  0.5003475546836853
- layer_3.avg_l0:
  108.95968627929688
- ▶
  layer_3.feature_density.bins:[] 65 items
- ▶
  layer_3.feature_density.values:[] 64 items
- layer_3.l1:
  0.00000181049108505249
- layer_3.loss:
  0.59375
- layer_3.mse:
  0.59375
- layer_4.avg_fvu:
  7.162869453430176
- layer_4.avg_l0:
  42.055389404296875
- ▶
  layer_4.feature_density.bins:[] 65 items
- ▶
  layer_4.feature_density.values:[] 64 items
- layer_4.l1:
  0.00103759765625
- layer_4.loss:
  8.5
- layer_4.mse:
  8.5
- layer_5.avg_fvu:
  16.959148406982422
- layer_5.avg_l0:
  5.419546127319336
- ▶
  layer_5.feature_density.bins:[] 65 items
- ▶
  layer_5.feature_density.values:[] 64 items
- layer_5.l1:
  0.00000268220901489258
- layer_5.loss:
  20.125
- layer_5.mse:
  20.125
- lr:
  0.00001285120000000181
- ▶
  total.feature_density.bins:[] 65 items
- ▶
  total.feature_density.values:[] 64 items

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-n1a51tem-history:v0

Collingray's group workspace

multilayer_pythia_32x_v2

7: ML_R5.2e-01_rtnone_rs1_LR=1.0e-03