levmckinney

Levmckinney's group workspace

Group: pythia-1.4b-deduped

1-1

of 1

Tags

Notes

Author

levmckinney

State

Finished

Start time

April 30th, 2023 9:58:06 PM

Runtime

13h 40m 20s

Tracked hours

13h 40m 15s

Run path

levmckinney/tuned-lens/4pimw50l

Linux-5.15.0-69-generic-x86_64-with-glibc2.35

Python version

3.10.6

Git repository

git clone git@github.com:AlignmentResearch/tuned-lens.git

Git state

git checkout -b "pythia-1.4-deduped-single-gpu" 433b352998e753a22ec0f1b0dcdc9752213c2fda

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-1.4b-deduped --data.name val.jsonl --per_gpu_batch_size=1 --wandb pythia-1.4-deduped-single-gpu

System Hardware

CPU count	30
Logical CPU count	30
GPU count	1
GPU type	NVIDIA A10

W&B CLI Version

0.15.0

Group

pythia-1.4b-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    false
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  null
- per_gpu_batch_size:
  1
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "pythia-1.4-deduped-single-gpu"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 72 keys
- bias_norm/0.ffn:
  0.08372334390878677
- bias_norm/1.ffn:
  0.06214439496397972
- bias_norm/2.ffn:
  0.05227738246321678
- bias_norm/3.ffn:
  0.04865264520049095
- bias_norm/4.ffn:
  0.04446972534060478
- bias_norm/5.ffn:
  0.03945485129952431
- bias_norm/6.ffn:
  0.039387769997119904
- bias_norm/7.ffn:
  0.03687728941440582
- bias_norm/8.ffn:
  0.03577563911676407
- bias_norm/9.ffn:
  0.03530735522508621
- bias_norm/10.ffn:
  0.03419721871614456
- bias_norm/11.ffn:
  0.032297227531671524
- bias_norm/12.ffn:
  0.02894881926476955
- bias_norm/13.ffn:
  0.026139503344893456
- bias_norm/14.ffn:
  0.0233419518917799
- bias_norm/15.ffn:
  0.02351752109825611
- bias_norm/16.ffn:
  0.01972641423344612
- bias_norm/17.ffn:
  0.01868392899632454
- bias_norm/18.ffn:
  0.017026657238602638
- bias_norm/19.ffn:
  0.014134662225842476
- bias_norm/20.ffn:
  0.012253792025148869
- bias_norm/21.ffn:
  0.011058310978114603
- bias_norm/22.ffn:
  0.007139739580452442
- bias_norm/input:
  1.7042267322540283
- loss/translator_0:
  2.0088207721710205
- loss/translator_1:
  1.6403244733810425
- loss/translator_2:
  1.5832446813583374
- loss/translator_3:
  1.516425609588623
- loss/translator_4:
  1.4180680513381958
- loss/translator_5:
  1.3300464153289795
- loss/translator_6:
  1.2729867696762085
- loss/translator_7:
  1.261189579963684
- loss/translator_8:
  1.1238908767700195
- loss/translator_9:
  1.0786733627319336
- loss/translator_10:
  1.0532126426696775
- loss/translator_11:
  0.9130973815917968
- loss/translator_12:
  0.8445327281951904
- loss/translator_13:
  0.7552800178527832
- loss/translator_14:
  0.6698911190032959
- loss/translator_15:
  0.5943631529808044
- loss/translator_16:
  0.4817017614841461
- loss/translator_17:
  0.411723792552948
- loss/translator_18:
  0.3563392162322998
- loss/translator_19:
  0.29559123516082764
- loss/translator_20:
  0.2341240495443344
- loss/translator_21:
  0.1824176609516144
- weight_norm/20.ffn:
  1.2338875532150269
- weight_norm/21.ffn:
  1.1057438850402832
- weight_norm/22.ffn:
  0.790013313293457
- weight_norm/input:
  2.5308187007904053

This run produced these artifacts as outputs. Learn more

wandb-history

run-4pimw50l-history:v0