levmckinney

Levmckinney's group workspace

Group: pythia-2.8b-deduped

1-2

of 2

Tags

Notes

Author

levmckinney

State

Finished

Start time

May 5th, 2023 12:54:05 AM

Runtime

3h 51m 9s

Tracked hours

3h 51m 1s

Run path

levmckinney/tuned-lens/0dy877rj

Linux-5.4.0-146-generic-x86_64-with-glibc2.35

Python version

3.10.6

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-2.8b-deduped --data.name /datasets/val.jsonl --per_gpu_batch_size=1 --output /output/EleutherAI/pythia-2.8b-deduped-1683247839 --wandb EleutherAI/pythia-2.8b-deduped-1683247839 --fsdp

System Hardware

CPU count	128
Logical CPU count	255
GPU count	2
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.15.0

Group

pythia-2.8b-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    true
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  "/output/EleutherAI/pythia-2.8b-deduped-1683247839"
- per_gpu_batch_size:
  1
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "EleutherAI/pythia-2.8b-deduped-1683247839"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 96 keys
- bias_norm/0.ffn:
  0.0425301194190979
- bias_norm/1.ffn:
  0.036475300788879395
- bias_norm/2.ffn:
  0.03302118554711342
- bias_norm/3.ffn:
  0.02692563086748123
- bias_norm/4.ffn:
  0.026232492178678513
- bias_norm/5.ffn:
  0.02368577197194099
- bias_norm/6.ffn:
  0.02091137133538723
- bias_norm/7.ffn:
  0.01912717893719673
- bias_norm/8.ffn:
  0.017886720597743988
- bias_norm/9.ffn:
  0.01687142439186573
- bias_norm/10.ffn:
  0.01585715264081955
- bias_norm/11.ffn:
  0.01547165773808956
- bias_norm/12.ffn:
  0.01505132857710123
- bias_norm/13.ffn:
  0.013655302114784718
- bias_norm/14.ffn:
  0.014104748144745828
- bias_norm/15.ffn:
  0.012901515699923038
- bias_norm/16.ffn:
  0.012405884452164171
- bias_norm/17.ffn:
  0.010662960819900036
- bias_norm/18.ffn:
  0.009389943443238735
- bias_norm/19.ffn:
  0.007946131750941277
- bias_norm/20.ffn:
  0.007072828710079193
- bias_norm/21.ffn:
  0.0060940939001739025
- bias_norm/22.ffn:
  0.005554928909987211
- bias_norm/23.ffn:
  0.005253300536423922
- bias_norm/24.ffn:
  0.005220202263444662
- bias_norm/25.ffn:
  0.005169644020497799
- bias_norm/26.ffn:
  0.005122469272464514
- bias_norm/27.ffn:
  0.004999740514904261
- bias_norm/28.ffn:
  0.004881990607827902
- bias_norm/29.ffn:
  0.0045368000864982605
- bias_norm/30.ffn:
  0.0034977279137820005
- bias_norm/input:
  1.5146443843841553
- loss/translator_0:
  2.081482172012329
- loss/translator_1:
  1.7693002223968506
- loss/translator_2:
  1.6962895393371582
- loss/translator_3:
  1.5932772159576416
- loss/translator_4:
  1.4757094383239746
- loss/translator_5:
  1.4448288679122925
- loss/translator_6:
  1.2893112897872925
- loss/translator_7:
  1.2367652654647827
- loss/translator_8:
  1.218070387840271
- loss/translator_9:
  1.1963986158370972
- loss/translator_10:
  1.1773234605789185
- loss/translator_11:
  1.1381564140319824
- loss/translator_12:
  1.0542317628860474
- loss/translator_13:
  1.0166372060775757
- weight_norm/28.ffn:
  0.8676188588142395
- weight_norm/29.ffn:
  0.7780693173408508
- weight_norm/30.ffn:
  0.545457661151886
- weight_norm/input:
  2.072133779525757

This run consumed these artifacts as inputs. Learn more

job

job-.io_levmck_hofvarpnir-jobs_sha256_5cb66cad0842bf3f74237de9a1b8c57f13bc43c4ee36313dbfd2f4bc73a989cf:v0

This run produced these artifacts as outputs. Learn more

wandb-history

run-0dy877rj-history:v0