levmckinney

Levmckinney's group workspace

Group: pythia-12b-deduped

1-1

of 1

Tags

Notes

Author

levmckinney

State

Finished

Start time

May 5th, 2023 3:19:42 PM

Runtime

5h 14m 37s

Tracked hours

5h 14m 29s

Run path

levmckinney/tuned-lens/ovvz4w3c

Linux-5.4.0-148-generic-x86_64-with-glibc2.35

Python version

3.10.6

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-12b-deduped --data.name /datasets/val.jsonl --per_gpu_batch_size=1 --output /output/EleutherAI/pythia-12b-deduped-1683299166 --wandb EleutherAI/pythia-12b-deduped-1683299166 --fsdp

System Hardware

CPU count	128
Logical CPU count	255
GPU count	4
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.15.0

Group

pythia-12b-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    true
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  "/output/EleutherAI/pythia-12b-deduped-1683299166"
- per_gpu_batch_size:
  1
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "EleutherAI/pythia-12b-deduped-1683299166"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 108 keys
- bias_norm/0.ffn:
  0.015914397314190865
- bias_norm/1.ffn:
  0.013314316980540752
- bias_norm/2.ffn:
  0.011105027049779892
- bias_norm/3.ffn:
  0.009572981856763365
- bias_norm/4.ffn:
  0.008649605326354504
- bias_norm/5.ffn:
  0.008007501251995564
- bias_norm/6.ffn:
  0.006929654162377119
- bias_norm/7.ffn:
  0.006142694503068924
- bias_norm/8.ffn:
  0.004967795684933662
- bias_norm/9.ffn:
  0.0043098400346934795
- bias_norm/10.ffn:
  0.003883617464452982
- bias_norm/11.ffn:
  0.0035951549652963877
- bias_norm/12.ffn:
  0.003284243866801262
- bias_norm/13.ffn:
  0.0029783418867737055
- bias_norm/14.ffn:
  0.0026997854001820087
- bias_norm/15.ffn:
  0.002420594450086355
- bias_norm/16.ffn:
  0.002192040905356407
- bias_norm/17.ffn:
  0.0020343526266515255
- bias_norm/18.ffn:
  0.0018384386785328388
- bias_norm/19.ffn:
  0.0017588514601811769
- bias_norm/20.ffn:
  0.0017427551792934537
- bias_norm/21.ffn:
  0.0017666786443442106
- bias_norm/22.ffn:
  0.0017473469488322737
- bias_norm/23.ffn:
  0.0017323915380984545
- bias_norm/24.ffn:
  0.001731615629978478
- bias_norm/25.ffn:
  0.0017512693302705884
- bias_norm/26.ffn:
  0.0016811741515994072
- bias_norm/27.ffn:
  0.0016638393281027677
- bias_norm/28.ffn:
  0.0016301656141877174
- bias_norm/29.ffn:
  0.00157680653501302
- bias_norm/30.ffn:
  0.001570948516018689
- bias_norm/31.ffn:
  0.001520850113593042
- bias_norm/32.ffn:
  0.0014232916291803122
- bias_norm/33.ffn:
  0.001282525947317481
- bias_norm/34.ffn:
  0.001195451826788485
- bias_norm/input:
  0.775674045085907
- loss/translator_0:
  2.139585256576538
- loss/translator_1:
  1.7683042287826538
- loss/translator_2:
  1.6840815544128418
- loss/translator_3:
  1.5712485313415527
- loss/translator_4:
  1.465919852256775
- loss/translator_5:
  1.392081379890442
- loss/translator_6:
  1.2736033201217651
- loss/translator_7:
  1.1397497653961182
- loss/translator_8:
  1.0946130752563477
- loss/translator_9:
  1.0028672218322754
- weight_norm/32.ffn:
  0.5218281149864197
- weight_norm/33.ffn:
  0.47037163376808167
- weight_norm/34.ffn:
  0.4201824367046356
- weight_norm/input:
  1.3950512409210205

This run consumed these artifacts as inputs. Learn more

job

job-.io_levmck_hofvarpnir-jobs_sha256_6d2d8f60fbe64ce74f782c181b2f9f4c7e4372afa6820c50245a6208fbab67df:v0

This run produced these artifacts as outputs. Learn more

wandb-history

run-ovvz4w3c-history:v0