Skip to main content

654 Scaling Laws with soft metrics

Created on January 16|Last edited on January 16

500G1T1.5Tthroughput/total_gflops0.850.90.95
10k20k30k40k50kStep0.80.850.90.951
Run set
18970
State
Notes
User
Tags
Created
Runtime
Sweep
_attn_implementation_autoset
_name_or_path
accelerator_config.even_batches
accelerator_config.non_blocking
accelerator_config.split_batches
accelerator_config.use_seedable_sampler
adafactor
adam_beta1
adam_beta2
adam_epsilon
add_cross_attention
allow_nondivisible_batch_size
allow_partial_checkpoint
architectures
attention_bias
attention_dropout
attention_probs_dropout_prob
auto_find_batch_size
auto_map.AutoConfig
auto_map.AutoModel
auto_map.AutoModelForMaskedLM
auto_map.AutoModelForMultipleChoice
auto_map.AutoModelForQuestionAnswering
auto_map.AutoModelForSequenceClassification
auto_map.AutoModelForTokenClassification
average_tokens_across_devices
batch_axis
batch_eval_metrics
bf16
bf16_full_eval
bos_token_id
chat_train_urls
checkpoint_path
checkpointer.append_run_id_to_base_path
checkpointer.base_path
checkpointer.delete_old_temp_checkpoints
checkpointer.keep
checkpointer.save_interval
chunk_size_feed_forward
classifier_activation
classifier_bias
classifier_dropout
classifier_pooling
cli_configs.batch_size
Crashed
percyliang
28m 45s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Crashed
percyliang
16m 16s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Finished
percyliang
19m 9s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Crashed
percyliang
6m 30s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Crashed
percyliang
0s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Finished
percyliang
7m 20s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Finished
percyliang
7m 33s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Crashed
percyliang
0s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Finished
percyliang
7m 1s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
Finished
percyliang
7m 53s
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
auto
1-10
of 28