collingray

Collingray's group workspace

Group: multilayer_pythia_32x_v3

1-8

of 8

Tags

Notes

Author

collingray

State

Finished

Start time

May 10th, 2024 12:26:17 AM

Runtime

46m 32s

Tracked hours

46m 30s

Run path

collingray/multilayer_sae/jzwtuodc

Linux-6.7.8-arch1-1-x86_64-with-glibc2.39

Python version

3.11.4

Command

<python with no main file> -f /home/collin/.local/share/jupyter/runtime/kernel-8e6c6b69-f805-4846-bfd1-2c15b3a26b01.json

System Hardware

CPU count	6
Logical CPU count	12
GPU count	1
GPU type	NVIDIA GeForce RTX 4090

W&B CLI Version

0.16.1

Group

multilayer_pythia_32x_v3

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 12 keys
- act_renorm_scale:
  1
- act_renorm_type:
  "none"
- decoder_scale:
  true
- input_scale:
  false
- latent_p:
  1
- lr:
  0.0003
- lr_decay_pct:
  0.2
- lr_warmup_pct:
  0
- β1:
  0.9
- β2:
  0.999
- λ:
  0.2154434621334076
- λ_warmup_pct:
  null

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 47 keys
- layer_0.avg_fvu:
  0.2671585977077484
- layer_0.avg_l0:
  245.2586212158203
- ▶
  layer_0.feature_density.bins:[] 65 items
- ▶
  layer_0.feature_density.values:[] 64 items
- layer_0.l1:
  0.07763671875
- layer_0.loss:
  0.337890625
- layer_0.mse:
  0.3203125
- layer_1.avg_fvu:
  0.531059205532074
- layer_1.avg_l0:
  241.11048889160156
- ▶
  layer_1.feature_density.bins:[] 65 items
- ▶
  layer_1.feature_density.values:[] 64 items
- layer_1.l1:
  0.0235595703125
- layer_1.loss:
  0.640625
- layer_1.mse:
  0.63671875
- layer_2.avg_fvu:
  11.05124568939209
- layer_2.avg_l0:
  198.60166931152344
- ▶
  layer_2.feature_density.bins:[] 65 items
- ▶
  layer_2.feature_density.values:[] 64 items
- layer_2.l1:
  0.037841796875
- layer_2.loss:
  13.25
- layer_2.mse:
  13.25
- layer_3.avg_fvu:
  0.3730446398258209
- layer_3.avg_l0:
  212.7537384033203
- ▶
  layer_3.feature_density.bins:[] 65 items
- ▶
  layer_3.feature_density.values:[] 64 items
- layer_3.l1:
  0.0252685546875
- layer_3.loss:
  0.453125
- layer_3.mse:
  0.447265625
- layer_4.avg_fvu:
  5.525622844696045
- layer_4.avg_l0:
  126.80301666259766
- ▶
  layer_4.feature_density.bins:[] 65 items
- ▶
  layer_4.feature_density.values:[] 64 items
- layer_4.l1:
  0.0439453125
- layer_4.loss:
  6.625
- layer_4.mse:
  6.625
- layer_5.avg_fvu:
  5.03040075302124
- layer_5.avg_l0:
  16.16275405883789
- ▶
  layer_5.feature_density.bins:[] 65 items
- ▶
  layer_5.feature_density.values:[] 64 items
- layer_5.l1:
  37.25
- layer_5.loss:
  14
- layer_5.mse:
  6.03125
- lr:
  0.00009144422400001588
- total.avg_fvu:
  3.127317428588867
- total.avg_l0:
  173.10069274902344
- ▶
  total.feature_density.bins:[] 65 items
- ▶
  total.feature_density.values:[] 64 items

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-jzwtuodc-history:v0

Collingray's group workspace

multilayer_pythia_32x_v3

3: λ2.2e-01_LR3.0e-04_β₁0.9_β₂0.999