levmckinney

Levmckinney's group workspace

Group: pythia-410m-deduped

1-20

of 46

Tags

Notes

Author

levmckinney

State

Finished

Start time

April 29th, 2023 3:49:23 AM

Runtime

8h 23m 35s

Tracked hours

8h 23m 30s

Run path

levmckinney/tuned-lens/a7wmlqz2

Linux-5.15.0-69-generic-x86_64-with-glibc2.35

Python version

3.10.6

Git repository

git clone git@github.com:AlignmentResearch/tuned-lens.git

Git state

git checkout -b "pythia-160m-deduped-single-gpu" 433b352998e753a22ec0f1b0dcdc9752213c2fda

Command

-m tuned_lens.__main__ train --model.name EleutherAI/pythia-410m-deduped --data.name val.jsonl --per_gpu_batch_size=2 --wandb pythia-160m-deduped-single-gpu

System Hardware

CPU count	30
Logical CPU count	30
GPU count	1
GPU type	NVIDIA A10

W&B CLI Version

0.15.0

Group

pythia-410m-deduped

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 17 keys
- constant:
  false
- ▶
  data:{} 5 keys
- ▶
  dist:{} 2 keys
  - cpu_offload:
    false
  - fsdp:
    false
- lens_name_or_path:
  null
- loss:
  "LossChoice.KL"
- ▶
  model:{} 5 keys
- num_steps:
  250
- ▶
  opt:{} 6 keys
- output:
  null
- per_gpu_batch_size:
  2
- pre_ln:
  false
- seed:
  42
- separate_unembeddings:
  false
- token_shift:
  null
- tokens_per_step:
  262,144
- wandb:
  "pythia-160m-deduped-single-gpu"
- wandb_upload_checkpoints:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 72 keys
- bias_norm/0.ffn:
  0.3666253685951233
- bias_norm/1.ffn:
  0.294188529253006
- bias_norm/2.ffn:
  0.21356502175331116
- bias_norm/3.ffn:
  0.18856598436832428
- bias_norm/4.ffn:
  0.16113899648189545
- bias_norm/5.ffn:
  0.1872699409723282
- bias_norm/6.ffn:
  0.18308046460151672
- bias_norm/7.ffn:
  0.17345373332500458
- bias_norm/8.ffn:
  0.1625095158815384
- bias_norm/9.ffn:
  0.15996475517749786
- bias_norm/10.ffn:
  0.13301661610603333
- bias_norm/11.ffn:
  0.12773564457893372
- bias_norm/12.ffn:
  0.1276129186153412
- bias_norm/13.ffn:
  0.10227931290864944
- bias_norm/14.ffn:
  0.10428910702466963
- bias_norm/15.ffn:
  0.10334904491901398
- bias_norm/16.ffn:
  0.08965490013360977
- bias_norm/17.ffn:
  0.09269517660140993
- bias_norm/18.ffn:
  0.08181064575910568
- bias_norm/19.ffn:
  0.08016320317983627
- bias_norm/20.ffn:
  0.054338809102773666
- bias_norm/21.ffn:
  0.06413818150758743
- bias_norm/22.ffn:
  0.04949090629816055
- bias_norm/input:
  1.892978310585022
- loss/translator_0:
  1.9245588779449463
- loss/translator_1:
  1.7185463905334473
- loss/translator_2:
  1.6680587530136108
- loss/translator_3:
  1.5741310119628906
- loss/translator_4:
  1.4736759662628174
- loss/translator_5:
  1.406898856163025
- loss/translator_6:
  1.3218592405319214
- loss/translator_7:
  1.259551167488098
- loss/translator_8:
  1.212220549583435
- loss/translator_9:
  1.1607027053833008
- loss/translator_10:
  1.1177096366882324
- loss/translator_11:
  1.0058774948120115
- loss/translator_12:
  0.8567332625389099
- loss/translator_13:
  0.782529890537262
- loss/translator_14:
  0.7041617035865784
- loss/translator_15:
  0.6262714862823486
- loss/translator_16:
  0.5804436802864075
- loss/translator_17:
  0.4735834300518036
- loss/translator_18:
  0.3858170211315155
- loss/translator_19:
  0.31610462069511414
- loss/translator_20:
  0.24517187476158145
- loss/translator_21:
  0.19552823901176453
- weight_norm/20.ffn:
  1.658342719078064
- weight_norm/21.ffn:
  1.4119179248809814
- weight_norm/22.ffn:
  0.9917079210281372
- weight_norm/input:
  2.770873546600342

This run produced these artifacts as outputs. Learn more

wandb-history

run-a7wmlqz2-history:v0