ai2-llm

Akshitab's group workspace

Group: OLMo-7B-Twin-2T

1-20

of 24

Tags

mosaicml-7b-v1

Author

epwalsh

State

Finished

Start time

February 6th, 2024 3:05:19 AM

Runtime

20m 27s

Tracked hours

17m 53s

Run path

ai2-llm/OLMo-7B/fcy9j4fw

Linux-5.15.0-60-generic-x86_64-with-glibc2.31

Python version

3.10.9

Git repository

git clone https://github.com/allenai/LLM.git

Git state

git checkout -b "OLMo-7B-Twin-2T-run-024" 584907622a4a6c0435614ec27717d264b9780510

Command

/home/petew/LLM/scripts/combine_wandb_runs.py

System Hardware

CPU count	64
Logical CPU count	128

W&B CLI Version

0.16.2

Group

OLMo-7B-Twin-2T

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 52 keys
- activation_checkpointing:
  "fine_grained"
- canceled_check_interval:
  50
- compile:
  null
- console_log_interval:
  1
- ▶
  data:{} 9 keys
- device_eval_batch_size:
  2
- device_train_batch_size:
  2
- device_train_grad_accum:
  1
- device_train_microbatch_size:
  2
- dry_run:
  false
- early_stopping_factor:
  null
- epoch:
  0
- eval_interval:
  10,000
- eval_on_load:
  false
- eval_subset_num_batches:
  -1
- ▶
  evaluators:[] 12 items
- fast_forward_batches:
  null
- force_save_unsharded:
  false
- ▶
  fsdp:{} 4 keys
  - precision:
    "mixed"
  - sharding_strategy:
    "ShardingStrategy.FULL_SHARD"
  - use_orig_params:
    true
  - wrapping_strategy:
    "one_in_four"
- global_train_batch_size:
  2,048
- load_path:
  "/scratch/project_462000229/checkpoints/5307103/latest"
- load_path_sharded_checkpointer:
  null
- max_duration:
  "2e12T"
- max_grad_norm:
  1
- max_grad_norm_ratio:
  null
- ▶
  model:{} 35 keys
- new_style_checkpoints:
  null
- no_pre_train_checkpoint:
  false
- ▶
  optimizer:{} 8 keys
- precision:
  "amp_bf16"
- python_profiling:
  false
- remote_save_folder:
  "s3://ai2-llm/checkpoints/7b/mitchish-lumi"
- reset_optimizer_state:
  false
- restore_dataloader:
  true
- run_name:
  "5375378"
- save_data_indices:
  true
- save_folder:
  "/pfs/lustref1/flash/project_462000229/checkpoints/mitchish-lumi"
- save_interval:
  10,000
- save_interval_ephemeral:
  1,000
- save_interval_unsharded:
  null
- save_num_checkpoints_to_keep:
  -1
- save_num_unsharded_checkpoints_to_keep:
  -1
- save_overwrite:
  true
- ▶
  scheduler:{} 6 keys
- seed:
  6,198
- sharded_checkpointer:
  "local"
- stop_at:
  null
- time_limit:
  171,000
- ▶
  tokenizer:{} 2 keys
  - identifier:
    "tokenizers/allenai_eleuther-ai-gpt-neox-20b-pii-special.json"
  - truncate_direction:
    "right"
- torch_profiling:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 2127 keys
- eval/4chan-validation/CrossEntropyLoss:
  null
- eval/4chan-validation/Perplexity:
  null
- eval/c4_100_domains-validation/CrossEntropyLoss:
  null
- eval/c4_100_domains-validation/Perplexity:
  null
- eval/c4_en-validation/CrossEntropyLoss:
  null
- eval/c4_en-validation/Perplexity:
  null
- eval/downstream/arc_easy_acc:
  null
- eval/downstream/commitment_bank_acc:
  null
- eval/downstream/copa_acc:
  null
- eval/downstream/hellaswag_len_norm:
  null
- eval/downstream/mrpc_f1:
  null
- eval/downstream/openbook_qa_len_norm:
  null
- eval/downstream/piqa_len_norm:
  null
- eval/downstream/rte_len_norm:
  null
- eval/downstream/sciq_acc:
  null
- eval/downstream/sst2_acc:
  null
- eval/downstream/winogrande_acc:
  null
- eval/gab-validation/CrossEntropyLoss:
  null
- eval/gab-validation/Perplexity:
  null
- eval/ice-validation/CrossEntropyLoss:
  null
- eval/ice-validation/Perplexity:
  null
- eval/m2d2_s2orc-validation/CrossEntropyLoss:
  null
- eval/m2d2_s2orc-validation/Perplexity:
  null
- eval/m2d2_wiki-validation/CrossEntropyLoss:
  null
- eval/m2d2_wiki-validation/Perplexity:
  null
- eval/manosphere-validation/CrossEntropyLoss:
  null
- eval/manosphere-validation/Perplexity:
  null
- eval/mc4_en-validation/CrossEntropyLoss:
  null
- eval/mc4_en-validation/Perplexity:
  null
- eval/pile-validation/CrossEntropyLoss:
  null
- eval/pile-validation/Perplexity:
  null
- eval/ptb-validation/CrossEntropyLoss:
  null
- eval/ptb-validation/Perplexity:
  null
- eval/twitterAEE-validation/CrossEntropyLoss:
  null
- eval/twitterAEE-validation/Perplexity:
  null
- eval/wikitext_103-validation/CrossEntropyLoss:
  null
- eval/wikitext_103-validation/Perplexity:
  null
- optim/clipping_rate:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.avg:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.max:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.min:
  null
- optim/exp_avg_sq/transformer.blocks.0.att_proj.weight.norm:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.avg:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.max:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.min:
  null
- optim/exp_avg_sq/transformer.blocks.0.attn_out.weight.norm:
  null
- throughput/device/tokens_per_second:
  695.5960616067973
- throughput/total_tokens:
  1,912,841,699,328
- train/CrossEntropyLoss:
  2.0855228900909424
- train/Perplexity:
  8.048799018813016

This run consumed these artifacts as inputs. Learn more

job

job-https___github.com_allenai_LLM.git_scripts_combine_wandb_runs.py:v104

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-fcy9j4fw-history:v0