collingray

Collingray's group workspace

Group: pythia_wikitext

1-20

of 25

Tags

Notes

Author

collingray

State

Finished

Start time

May 10th, 2024 3:17:23 PM

Runtime

2h 36m 8s

Tracked hours

2h 36m 4s

Run path

collingray/multilayer_sae/pbjiqsdh

Linux-6.7.8-arch1-1-x86_64-with-glibc2.39

Python version

3.11.4

Command

scratchpad.ipynb

System Hardware

CPU count	6
Logical CPU count	12
GPU count	1
GPU type	NVIDIA GeForce RTX 4090

W&B CLI Version

0.16.1

Group

pythia_wikitext

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 14 keys
- act_norms:
  6
- act_renorm_scale:
  1
- act_renorm_type:
  "none"
- device:
  "cuda"
- dtype:
  "torch.bfloat16"
- firing_bucket_size:
  262,144
- m_dim:
  16,384
- n_dim:
  512
- name:
  "autoencoder"
- num_firing_buckets:
  16
- record_data:
  true
- save_dir:
  "./weights"
- seed:
  42
- tied:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 47 keys
- layer_0.avg_fvu:
  0.03198991343379021
- layer_0.avg_l0:
  477.0677490234375
- ▶
  layer_0.feature_density.bins:[] 65 items
- ▶
  layer_0.feature_density.values:[] 64 items
- layer_0.l1:
  8.5625
- layer_0.loss:
  0.1240234375
- layer_0.mse:
  0.038330078125
- layer_1.avg_fvu:
  0.02129264920949936
- layer_1.avg_l0:
  498.5236511230469
- ▶
  layer_1.feature_density.bins:[] 65 items
- ▶
  layer_1.feature_density.values:[] 64 items
- layer_1.l1:
  13.3125
- layer_1.loss:
  0.158203125
- layer_1.mse:
  0.0255126953125
- layer_2.avg_fvu:
  0.020171983167529103
- layer_2.avg_l0:
  1,010.5702514648438
- ▶
  layer_2.feature_density.bins:[] 65 items
- ▶
  layer_2.feature_density.values:[] 64 items
- layer_2.l1:
  68
- layer_2.loss:
  0.703125
- layer_2.mse:
  0.024169921875
- layer_3.avg_fvu:
  0.03545378893613815
- layer_3.avg_l0:
  430.186279296875
- ▶
  layer_3.feature_density.bins:[] 65 items
- ▶
  layer_3.feature_density.values:[] 64 items
- layer_3.l1:
  10.8125
- layer_3.loss:
  0.150390625
- layer_3.mse:
  0.04248046875
- layer_4.avg_fvu:
  0.042992815375328064
- layer_4.avg_l0:
  582.7105102539062
- ▶
  layer_4.feature_density.bins:[] 65 items
- ▶
  layer_4.feature_density.values:[] 64 items
- layer_4.l1:
  49.5
- layer_4.loss:
  0.546875
- layer_4.mse:
  0.051513671875
- layer_5.avg_fvu:
  0.06601740419864655
- layer_5.avg_l0:
  287.7701416015625
- ▶
  layer_5.feature_density.bins:[] 65 items
- ▶
  layer_5.feature_density.values:[] 64 items
- layer_5.l1:
  92
- layer_5.loss:
  1
- layer_5.mse:
  0.0791015625
- lr:
  0.00000631543376750566
- total.avg_fvu:
  0.02891719341278076
- total.avg_l0:
  544.5386962890625
- ▶
  total.feature_density.bins:[] 65 items
- ▶
  total.feature_density.values:[] 64 items

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-pbjiqsdh-history:v0

Collingray's group workspace

pythia_wikitext

pythia_wikitext_32x_v8