levmckinney

Levmckinney's group workspace

Group: pythia-6.9b-deduped

1-1

of 1

Tags

Notes

Author

levmckinney

State

Finished

Start time

May 5th, 2023 4:54:04 AM

Runtime

6h 1m 32s

Tracked hours

6h 1m 25s

Run path

levmckinney/tuned-lens/n9pp9pwy

Linux-5.4.0-146-generic-x86_64-with-glibc2.35

Python version

3.10.6

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-6.9b-deduped --data.name /datasets/val.jsonl --per_gpu_batch_size=1 --output /output/EleutherAI/pythia-6.9b-deduped-1683261951 --wandb EleutherAI/pythia-6.9b-deduped-1683261951 --fsdp

System Hardware

CPU count	128
Logical CPU count	255
GPU count	2
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.15.0

Group

pythia-6.9b-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    true
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  "/output/EleutherAI/pythia-6.9b-deduped-1683261951"
- per_gpu_batch_size:
  1
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "EleutherAI/pythia-6.9b-deduped-1683261951"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 96 keys
- bias_norm/0.ffn:
  0.022592246532440186
- bias_norm/1.ffn:
  0.016763588413596153
- bias_norm/2.ffn:
  0.01404152438044548
- bias_norm/3.ffn:
  0.012916706502437592
- bias_norm/4.ffn:
  0.011516517028212547
- bias_norm/5.ffn:
  0.010814509354531763
- bias_norm/6.ffn:
  0.00945410318672657
- bias_norm/7.ffn:
  0.008815479464828968
- bias_norm/8.ffn:
  0.007719195447862148
- bias_norm/9.ffn:
  0.00635061040520668
- bias_norm/10.ffn:
  0.005455582868307829
- bias_norm/11.ffn:
  0.005304891150444746
- bias_norm/12.ffn:
  0.00489181699231267
- bias_norm/13.ffn:
  0.004511257633566856
- bias_norm/14.ffn:
  0.004033615812659264
- bias_norm/15.ffn:
  0.00377705623395741
- bias_norm/16.ffn:
  0.003524661296978593
- bias_norm/17.ffn:
  0.003254247596487403
- bias_norm/18.ffn:
  0.0030989437364041805
- bias_norm/19.ffn:
  0.0029369830153882504
- bias_norm/20.ffn:
  0.002834836021065712
- bias_norm/21.ffn:
  0.002828738885000348
- bias_norm/22.ffn:
  0.002754144836217165
- bias_norm/23.ffn:
  0.002762648742645979
- bias_norm/24.ffn:
  0.002773342188447714
- bias_norm/25.ffn:
  0.002785065909847617
- bias_norm/26.ffn:
  0.0027570140082389116
- bias_norm/27.ffn:
  0.002713581547141075
- bias_norm/28.ffn:
  0.002585683250799775
- bias_norm/29.ffn:
  0.0024507795460522175
- bias_norm/30.ffn:
  0.00243382272310555
- bias_norm/input:
  1.070804476737976
- loss/translator_0:
  2.1260461807250977
- loss/translator_1:
  1.7633315324783323
- loss/translator_2:
  1.6882742643356323
- loss/translator_3:
  1.5948361158370972
- loss/translator_4:
  1.5128463506698608
- loss/translator_5:
  1.4147065877914429
- loss/translator_6:
  1.298643946647644
- loss/translator_7:
  1.2270056009292605
- loss/translator_8:
  1.176915407180786
- loss/translator_9:
  1.0440255403518677
- loss/translator_10:
  0.9681597352027892
- loss/translator_11:
  0.8884387016296387
- loss/translator_12:
  0.8615212440490723
- loss/translator_13:
  0.8117616176605225
- weight_norm/28.ffn:
  0.6933315396308899
- weight_norm/29.ffn:
  0.6350148320198059
- weight_norm/30.ffn:
  0.5280448794364929
- weight_norm/input:
  1.6963543891906738

This run consumed these artifacts as inputs. Learn more

job

job-.io_levmck_hofvarpnir-jobs_sha256_32558218fa5514b21ac2622051e4506a2515ac657969ffbbcdb635623919170:v0

This run produced these artifacts as outputs. Learn more

wandb-history

run-n9pp9pwy-history:v0