ai2-llm

Akshitab's group workspace

Group: OLMo-1.7-7B-anneal

1-1

of 1

Tags

Notes

Author

epwalsh

State

Finished

Job

job-https___github.com_allenai_OLMo.git_scripts_copy_wandb_runs.py:v23

Start time

April 18th, 2024 3:12:53 PM

Runtime

18m 6s

Tracked hours

15m 39s

Run path

ai2-llm/OLMo-7B/lkwbcao1

Linux-5.15.0-60-generic-x86_64-with-glibc2.31

Python version

3.10.14

Git repository

git clone https://github.com/allenai/OLMo.git

Git state

git checkout -b "OLMo-1.7-7B-anneal-50B" 43a155faf0c41c849fc2e14fe375a6bc57ed89fb

Command

/home/petew/OLMo/scripts/copy_wandb_runs.py

System Hardware

CPU count	56
Logical CPU count	112
GPU count	4
GPU type	NVIDIA A100-PCIE-40GB

W&B CLI Version

0.16.6

Group

OLMo-1.7-7B-anneal

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 57 keys
- activation_checkpointing:
  "fine_grained"
- canceled_check_interval:
  50
- compile:
  null
- console_log_interval:
  1
- ▶
  data:{} 12 keys
- device_eval_batch_size:
  2
- device_train_batch_size:
  16
- device_train_grad_accum:
  8
- device_train_microbatch_size:
  2
- dry_run:
  false
- early_stopping_factor:
  null
- epoch:
  null
- eval_interval:
  500
- eval_on_load:
  false
- eval_subset_num_batches:
  -1
- ▶
  evaluators:[] 21 items
- extra_steps_after_cancel:
  10
- fast_forward_batches:
  null
- force_save_unsharded:
  false
- ▶
  fsdp:{} 4 keys
  - precision:
    "mixed"
  - sharding_strategy:
    "ShardingStrategy.SHARD_GRAD_OP"
  - use_orig_params:
    true
  - wrapping_strategy:
    "by_block_and_size"
- fused_loss:
  true
- gen1_gc_interval:
  8
- global_train_batch_size:
  1,024
- load_path:
  "s3://ai2-llm/checkpoints/OLMo-medium/mitchish7/step477000-unsharded"
- load_path_sharded_checkpointer:
  null
- max_duration:
  "50e9T"
- max_grad_norm:
  1
- max_grad_norm_ratio:
  null
- ▶
  model:{} 37 keys
- new_style_checkpoints:
  null
- no_pre_train_checkpoint:
  true
- ▶
  optimizer:{} 8 keys
- precision:
  "amp_bf16"
- python_profiling:
  false
- remote_save_folder:
  s3://ai2-llm/checkpoints/davidw/annealing/v1.7-step_2T-resume_optimizer-steps_50B
- reset_optimizer_state:
  false
- reset_trainer_state:
  true
- restore_dataloader:
  false
- run_name:
  "v1.7-step_2T-resume_optimizer-steps_50B"
- save_data_indices:
  true
- save_folder:
  "/data"
- save_interval:
  500
- save_interval_ephemeral:
  null
- save_interval_unsharded:
  null
- save_num_checkpoints_to_keep:
  -1
- save_num_unsharded_checkpoints_to_keep:
  -1
- stop_at:
  null
- time_limit:
  null
- ▶
  tokenizer:{} 2 keys
  - identifier:
    "tokenizers/allenai_gpt-neox-olmo-dolma-v1_5.json"
  - truncate_direction:
    "right"
- torch_profiling:
  false

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 2135 keys
- eval/4chan-validation/CrossEntropyLoss:
  null
- eval/4chan-validation/Perplexity:
  null
- eval/c4_100_domains-validation/CrossEntropyLoss:
  null
- eval/c4_100_domains-validation/Perplexity:
  null
- eval/c4_en-validation/CrossEntropyLoss:
  null
- eval/c4_en-validation/Perplexity:
  null
- eval/downstream/arc_challenge_len_norm:
  null
- eval/downstream/arc_easy_acc:
  null
- eval/downstream/boolq_acc:
  null
- eval/downstream/hellaswag_len_norm:
  null
- eval/downstream/mmlu_humanities_len_norm:
  null
- eval/downstream/mmlu_humanities_mc_5shot_len_norm:
  null
- eval/downstream/mmlu_humanities_var_len_norm:
  null
- eval/downstream/mmlu_other_len_norm:
  null
- eval/downstream/mmlu_other_mc_5shot_len_norm:
  null
- eval/downstream/mmlu_other_var_len_norm:
  null
- eval/downstream/mmlu_social_sciences_len_norm:
  null
- eval/downstream/mmlu_social_sciences_mc_5shot_len_norm:
  null
- eval/downstream/mmlu_social_sciences_var_len_norm:
  null
- eval/downstream/mmlu_stem_len_norm:
  null
- eval/downstream/mmlu_stem_mc_5shot_len_norm:
  null
- eval/downstream/mmlu_stem_var_len_norm:
  null
- eval/downstream/openbook_qa_len_norm:
  null
- eval/downstream/piqa_len_norm:
  null
- eval/downstream/sciq_acc:
  null
- eval/downstream/winogrande_acc:
  null
- eval/gab-validation/CrossEntropyLoss:
  null
- eval/gab-validation/Perplexity:
  null
- eval/ice-validation/CrossEntropyLoss:
  null
- eval/ice-validation/Perplexity:
  null
- eval/m2d2_s2orc-validation/CrossEntropyLoss:
  null
- eval/m2d2_s2orc-validation/Perplexity:
  null
- eval/m2d2_wiki-validation/CrossEntropyLoss:
  null
- eval/m2d2_wiki-validation/Perplexity:
  null
- eval/manosphere-validation/CrossEntropyLoss:
  null
- eval/manosphere-validation/Perplexity:
  null
- eval/mc4_en-validation/CrossEntropyLoss:
  null
- eval/mc4_en-validation/Perplexity:
  null
- eval/pile-validation/CrossEntropyLoss:
  null
- eval/pile-validation/Perplexity:
  null
- eval/ptb-validation/CrossEntropyLoss:
  null
- eval/ptb-validation/Perplexity:
  null
- eval/twitterAEE-validation/CrossEntropyLoss:
  null
- eval/twitterAEE-validation/Perplexity:
  null
- eval/wikitext_103-validation/CrossEntropyLoss:
  null
- eval/wikitext_103-validation/Perplexity:
  null
- throughput/device/tokens_per_second:
  7,574.633074295682
- throughput/total_tokens:
  54,664,364,032
- train/CrossEntropyLoss:
  1.7184362411499023
- train/Perplexity:
  5.575802432948954

This run consumed these artifacts as inputs. Learn more

job

job-https___github.com_allenai_OLMo.git_scripts_copy_wandb_runs.py:v23

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-lkwbcao1-history:v2