levmckinney

Levmckinney's group workspace

Group: pythia-160m-deduped

1-1

of 1

Tags

Notes

Author

levmckinney

State

Finished

Start time

April 28th, 2023 10:09:29 PM

Runtime

3h 38m 52s

Tracked hours

3h 38m 46s

Run path

levmckinney/tuned-lens/8di18thy

Linux-5.15.0-69-generic-x86_64-with-glibc2.35

Python version

3.10.6

Git repository

git clone git@github.com:AlignmentResearch/tuned-lens.git

Git state

git checkout -b "pythia-160m-deduped-single-gpu" 433b352998e753a22ec0f1b0dcdc9752213c2fda

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-160m-deduped --data.name val.jsonl --per_gpu_batch_size=2 --wandb pythia-160m-deduped-single-gpu

System Hardware

CPU count	30
Logical CPU count	30
GPU count	1
GPU type	NVIDIA A10

W&B CLI Version

0.15.0

Group

pythia-160m-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    false
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  null
- per_gpu_batch_size:
  2
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "pythia-160m-deduped-single-gpu"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 36 keys
- bias_norm/0.ffn:
  2.0609652996063232
- bias_norm/1.ffn:
  1.2509818077087402
- bias_norm/2.ffn:
  0.9316051602363586
- bias_norm/3.ffn:
  0.9871343970298768
- bias_norm/4.ffn:
  0.7916807532310486
- bias_norm/5.ffn:
  0.4731448292732239
- bias_norm/6.ffn:
  0.6521783471107483
- bias_norm/7.ffn:
  0.673930287361145
- bias_norm/8.ffn:
  0.4875253140926361
- bias_norm/9.ffn:
  0.3288973271846771
- bias_norm/10.ffn:
  0.23421116173267365
- bias_norm/input:
  4.2680511474609375
- loss/translator_0:
  1.8504103422164917
- loss/translator_1:
  1.514171838760376
- loss/translator_2:
  1.422130465507507
- loss/translator_3:
  1.2573373317718506
- loss/translator_4:
  1.1472371816635132
- loss/translator_5:
  0.8442860841751099
- loss/translator_6:
  0.7154824733734131
- loss/translator_7:
  0.6381961703300476
- loss/translator_8:
  0.5528746843338013
- loss/translator_9:
  0.438175231218338
- loss/translator_10:
  0.3387037515640259
- loss/translator_11:
  0.2384711652994156
- weight_norm/0.ffn:
  9.424782752990724
- weight_norm/1.ffn:
  8.642461776733398
- weight_norm/2.ffn:
  8.519474983215332
- weight_norm/3.ffn:
  8.950639724731445
- weight_norm/4.ffn:
  8.356016159057617
- weight_norm/5.ffn:
  7.1603827476501465
- weight_norm/6.ffn:
  7.122384548187256
- weight_norm/7.ffn:
  6.808546543121338
- weight_norm/8.ffn:
  6.184028625488281
- weight_norm/9.ffn:
  5.526681900024414
- weight_norm/10.ffn:
  4.9110822677612305
- weight_norm/input:
  6.7801337242126465

This run produced these artifacts as outputs. Learn more

wandb-history

run-8di18thy-history:v0