ai2-llm

Akshitab's group workspace

Group: OLMo-7B-Tulu

1-1

of 1

Tags

Notes

Author

epwalsh

State

Finished

Start time

February 3rd, 2024 9:09:03 AM

Runtime

8m 37s

Tracked hours

8m 6s

Run path

ai2-llm/OLMo-7B/2whz7a3j

Linux-5.15.0-60-generic-x86_64-with-glibc2.31

Python version

3.10.9

Git repository

git clone https://github.com/allenai/LLM.git

Git state

git checkout -b "OLMo-7B-Tulu" 94b4539eb2e372c2dc52ea730a7f896b7e7bf482

Command

/home/petew/LLM/scripts/combine_wandb_runs.py

System Hardware

CPU count	64
Logical CPU count	128

W&B CLI Version

0.16.2

Group

OLMo-7B-Tulu

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 52 keys
- activation_checkpointing:
  null
- canceled_check_interval:
  50
- ▶
  compile:{} 3 keys
  - backend:
    "inductor"
  - fullgraph:
    false
  - mode:
    null
- console_log_interval:
  1
- ▶
  data:{} 9 keys
- device_eval_batch_size:
  4
- device_train_batch_size:
  64
- device_train_grad_accum:
  16
- device_train_microbatch_size:
  4
- dry_run:
  false
- early_stopping_factor:
  null
- epoch:
  0
- eval_interval:
  100
- eval_on_load:
  false
- eval_subset_num_batches:
  -1
- ▶
  evaluators:[] 12 items
- fast_forward_batches:
  null
- force_save_unsharded:
  false
- ▶
  fsdp:{} 4 keys
  - precision:
    "mixed"
  - sharding_strategy:
    "ShardingStrategy.FULL_SHARD"
  - use_orig_params:
    true
  - wrapping_strategy:
    "by_block"
- global_train_batch_size:
  2,048
- load_path:
  "/net/nfs.cirrascale/allennlp/petew/checkpoints/v1_5-mix-mitch-ish/step556000-unsharded"
- load_path_sharded_checkpointer:
  null
- max_duration:
  476,837
- max_grad_norm:
  1
- max_grad_norm_ratio:
  null
- ▶
  model:{} 35 keys
- new_style_checkpoints:
  null
- no_pre_train_checkpoint:
  false
- ▶
  optimizer:{} 8 keys
- precision:
  "amp_bf16"
- python_profiling:
  false
- remote_save_folder:
  "s3://ai2-llm/checkpoints/7b/v1_5-mix-mitch-ish-final-tulu"
- reset_optimizer_state:
  false
- restore_dataloader:
  false
- run_name:
  "mitchish-tulu"
- save_data_indices:
  true
- save_folder:
  "runs/mitchish-tulu"
- save_interval:
  500
- save_interval_ephemeral:
  null
- save_interval_unsharded:
  null
- save_num_checkpoints_to_keep:
  -1
- save_num_unsharded_checkpoints_to_keep:
  -1
- save_overwrite:
  true
- ▶
  scheduler:{} 6 keys
- seed:
  6,198
- sharded_checkpointer:
  "torch_legacy"
- stop_at:
  558,223
- time_limit:
  null
- ▶
  tokenizer:{} 2 keys
  - identifier:
    "tokenizers/allenai_eleuther-ai-gpt-neox-20b-pii-special.json"
  - truncate_direction:
    "right"
- torch_profiling:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 2127 keys
- eval/4chan-validation/CrossEntropyLoss:
  null
- eval/4chan-validation/Perplexity:
  null
- eval/c4_100_domains-validation/CrossEntropyLoss:
  null
- eval/c4_100_domains-validation/Perplexity:
  null
- eval/c4_en-validation/CrossEntropyLoss:
  null
- eval/c4_en-validation/Perplexity:
  null
- eval/downstream/arc_easy_acc:
  null
- eval/downstream/commitment_bank_acc:
  null
- eval/downstream/copa_acc:
  null
- eval/downstream/hellaswag_len_norm:
  null
- eval/downstream/mrpc_f1:
  null
- eval/downstream/openbook_qa_len_norm:
  null
- eval/downstream/piqa_len_norm:
  null
- eval/downstream/rte_len_norm:
  null
- eval/downstream/sciq_acc:
  null
- eval/downstream/sst2_acc:
  null
- eval/downstream/winogrande_acc:
  null
- eval/gab-validation/CrossEntropyLoss:
  null
- eval/gab-validation/Perplexity:
  null
- eval/ice-validation/CrossEntropyLoss:
  null
- eval/ice-validation/Perplexity:
  null
- eval/m2d2_s2orc-validation/CrossEntropyLoss:
  null
- eval/m2d2_s2orc-validation/Perplexity:
  null
- eval/m2d2_wiki-validation/CrossEntropyLoss:
  null
- eval/m2d2_wiki-validation/Perplexity:
  null
- eval/manosphere-validation/CrossEntropyLoss:
  null
- eval/manosphere-validation/Perplexity:
  null
- eval/mc4_en-validation/CrossEntropyLoss:
  null
- eval/mc4_en-validation/Perplexity:
  null
- eval/pile-validation/CrossEntropyLoss:
  null
- eval/pile-validation/Perplexity:
  null
- eval/ptb-validation/CrossEntropyLoss:
  null
- eval/ptb-validation/Perplexity:
  null
- eval/twitterAEE-validation/CrossEntropyLoss:
  null
- eval/twitterAEE-validation/Perplexity:
  null
- eval/wikitext_103-validation/CrossEntropyLoss:
  null
- eval/wikitext_103-validation/Perplexity:
  null
- optim/clipping_rate:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.avg:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.max:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.min:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.norm:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.avg:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.max:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.min:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.norm:
  null
- throughput/device/tokens_per_second:
  3,896.482302963216
- throughput/total_tokens:
  9,323,937,792
- train/CrossEntropyLoss:
  1.9563008546829224
- train/Perplexity:
  7.073114135298286

This run consumed these artifacts as inputs. Learn more

job

job-https___github.com_allenai_LLM.git_scripts_combine_wandb_runs.py:v80

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-2whz7a3j-history:v0