levmckinney

Levmckinney's group workspace

Group: pythia-2.8b-deduped-v0

1-2

of 2

Tags

Notes

Author

levmckinney

State

Finished

Start time

May 5th, 2023 8:01:45 PM

Runtime

3h 54m 55s

Tracked hours

3h 54m 49s

Run path

levmckinney/tuned-lens/6wngha4s

Linux-5.4.0-107-generic-x86_64-with-glibc2.35

Python version

3.10.6

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-2.8b-deduped-v0 --data.name /datasets/val.jsonl --per_gpu_batch_size=1 --output /output/EleutherAI/pythia-2.8b-deduped-v0-1683316697 --wandb EleutherAI/pythia-2.8b-deduped-v0-1683316697 --fsdp

System Hardware

CPU count	128
Logical CPU count	255
GPU count	2
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.15.0

Group

pythia-2.8b-deduped-v0

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    true
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  "/output/EleutherAI/pythia-2.8b-deduped-v0-1683316697"
- per_gpu_batch_size:
  1
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "EleutherAI/pythia-2.8b-deduped-v0-1683316697"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 96 keys
- bias_norm/0.ffn:
  0.03782312944531441
- bias_norm/1.ffn:
  0.03233238682150841
- bias_norm/2.ffn:
  0.0319121815264225
- bias_norm/3.ffn:
  0.028152519837021828
- bias_norm/4.ffn:
  0.025158891454339027
- bias_norm/5.ffn:
  0.023298269137740135
- bias_norm/6.ffn:
  0.021414052695035934
- bias_norm/7.ffn:
  0.019969290122389793
- bias_norm/8.ffn:
  0.018401101231575012
- bias_norm/9.ffn:
  0.0167826097458601
- bias_norm/10.ffn:
  0.015507766976952553
- bias_norm/11.ffn:
  0.014823013916611671
- bias_norm/12.ffn:
  0.013741719536483288
- bias_norm/13.ffn:
  0.012744450010359287
- bias_norm/14.ffn:
  0.012785433791577816
- bias_norm/15.ffn:
  0.012054089456796646
- bias_norm/16.ffn:
  0.010810396634042265
- bias_norm/17.ffn:
  0.00963456928730011
- bias_norm/18.ffn:
  0.008703671395778656
- bias_norm/19.ffn:
  0.008072645403444767
- bias_norm/20.ffn:
  0.007798327133059502
- bias_norm/21.ffn:
  0.007650479674339294
- bias_norm/22.ffn:
  0.00801115669310093
- bias_norm/23.ffn:
  0.00789344497025013
- bias_norm/24.ffn:
  0.007599088363349438
- bias_norm/25.ffn:
  0.007295246701687574
- bias_norm/26.ffn:
  0.0070247165858745575
- bias_norm/27.ffn:
  0.0067007350735366344
- bias_norm/28.ffn:
  0.0063269478268921375
- bias_norm/29.ffn:
  0.005746887531131506
- bias_norm/30.ffn:
  0.004358953796327114
- bias_norm/input:
  1.4625439643859863
- loss/translator_0:
  2.0649847984313965
- loss/translator_1:
  1.7440794706344604
- loss/translator_2:
  1.6690800189971924
- loss/translator_3:
  1.566443681716919
- loss/translator_4:
  1.4060754776000977
- loss/translator_5:
  1.3671038150787354
- loss/translator_6:
  1.3018323183059692
- loss/translator_7:
  1.254406452178955
- loss/translator_8:
  1.2331786155700684
- loss/translator_9:
  1.2165470123291016
- loss/translator_10:
  1.1854616403579712
- loss/translator_11:
  1.1653034687042236
- loss/translator_12:
  1.0620527267456057
- loss/translator_13:
  0.9753857851028442
- weight_norm/28.ffn:
  1.0086007118225098
- weight_norm/29.ffn:
  0.9076520800590516
- weight_norm/30.ffn:
  0.6269403696060181
- weight_norm/input:
  2.1561594009399414

This run consumed these artifacts as inputs. Learn more

job

job-.io_levmck_hofvarpnir-jobs_sha256_ce2301aff1a519f2be958eb1e32422b2f7efca67c5a250f5a800fd6fa988be01:v0

This run produced these artifacts as outputs. Learn more

wandb-history

run-6wngha4s-history:v0