Levmckinney's group workspace
Group: pythia-2.8b-deduped
State
Notes
User
Tags
Created
Runtime
Sweep
constant
data
data.max_length
data.name
data.split
data.text_column
dist
dist.cpu_offload
dist.fsdp
loss
model
model.name
model.revision
model.slow_tokenizer
num_steps
opt
opt.lr_scale
opt.momentum
opt.optimizer
opt.weight_decay
opt.zero
output
per_gpu_batch_size
pre_ln
seed
separate_unembeddings
tokens_per_step
wandb
wandb_upload_checkpoints
bias_only
checkpoint_dir
checkpoint_freq
data.shuffle_seed
dist.nccl_timeout
dist.per_gpu_batch_size
model.precision
opt.warmup_steps
data.dataset_shuffle
data.dataset_shuffle_seed
dist.dataloader_shuffle
w_ce
w_kl
data.max_seq_len
bias_norm/0.ffn
Finished
-
levmckinney
3h 51m 9s
-
false
-
2048
["/datasets/val.jsonl"]
validation
text
-
false
true
LossChoice.KL
-
EleutherAI/pythia-2.8b-deduped
main
false
250
-
1
0.9
OptimizerOption.SGD
0.001
false
/output/EleutherAI/pythia-2.8b-deduped-1683247839
1
false
42
false
262144
EleutherAI/pythia-2.8b-deduped-1683247839
false
-
-
-
-
-
-
-
-
-
-
-
-
-
-
0.04253
Crashed
-
levmckinney
1h 5m 27s
-
false
-
2048
["/datasets/val.jsonl"]
validation
text
-
false
true
LossChoice.KL
-
EleutherAI/pythia-2.8b-deduped
main
false
250
-
1
0.9
OptimizerOption.SGD
0.001
false
/output/EleutherAI/pythia-2.8b-deduped-1683243602
1
false
42
false
262144
EleutherAI/pythia-2.8b-deduped-1683243602
false
-
-
-
-
-
-
-
-
-
-
-
-
-
-
0.032578
1-2
of 2