asap-zzhou

Asap-zzhou's group workspace

Group: sft-reproduction

1-3

of 3

Tags

Notes

Author

asap-zzhou

State

Finished

Start time

October 21st, 2025 5:06:43 AM

Runtime

16h 45m 36s

Tracked hours

16h 45m 31s

Run path

asap-zzhou/dllm/1z7kqb2q

Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.17

Python version

CPython 3.10.18

Git repository

git clone git@github.com:ZHZisZZ/dllm.git

Git state

git checkout -b "Dream-7B-SFT-tulu3-fsdp-bs3-len2048-ep5-lr1e-5-gbl" f3e64f95ad3d3fa2611d777786f7529959ee0656

Command

/mnt/lustrenew/mllm_aligned/dongzhichen/dllm/examples/dream/sft.py --dataset_args data/sft/dream/tulu-3-sft-mixture --load_preprocessed_data True --output_dir models/Dream-7B-SFT-tulu3-fsdp-bs3-len2048-ep5-lr1e-5-gbl --max_length 2048 --truncation right --group_by_length True --num_train_epochs 5 --learning_rate 1e-5 --per_device_train_batch_size 2 --gradient_accumulation_steps 2 --per_device_eval_batch_size 2 --eval_on_start False --eval_steps 0.1 --save_steps 0.05

System Hardware

CPU count	64
Logical CPU count	128
GPU count	8
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.22.2

Group

sft-reproduction

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 218 keys
- _name_or_path:
  "/mnt/lustrenew/mllm_aligned/shared/models/huggingface/Dream-org/Dream-v0-Base-7B"
- ▶
  accelerator_config:{} 6 keys
- adafactor:
  false
- adam_beta1:
  0.9
- adam_beta2:
  0.999
- adam_epsilon:
  0.00000001
- add_cross_attention:
  false
- ▶
  architectures:[] 1 item
  - 0:
    "DreamModel"
- attention_dropout:
  0
- auto_find_batch_size:
  false
- ▶
  auto_map:{} 2 keys
  - AutoConfig:
    "configuration_dream.DreamConfig"
  - AutoModel:
    "modeling_dream.DreamModel"
- average_tokens_across_devices:
  true
- bad_words_ids:
  null
- batch_eval_metrics:
  false
- begin_suppress_tokens:
  null
- bf16:
  true
- bf16_full_eval:
  false
- bias:
  "none"
- bos_token_id:
  151,665
- chunk_size_feed_forward:
  0
- cross_attention_hidden_size:
  null
- data_seed:
  null
- dataloader_drop_last:
  false
- dataloader_num_workers:
  0
- dataloader_persistent_workers:
  false
- dataloader_pin_memory:
  true
- dataloader_prefetch_factor:
  null
- ddp_backend:
  null
- ddp_broadcast_buffers:
  null
- ddp_bucket_cap_mb:
  null
- ddp_find_unused_parameters:
  null
- ddp_timeout:
  1,800
- debug:[] 0 items
- decoder_start_token_id:
  null
- deepspeed:
  null
- disable_tqdm:
  false
- diversity_penalty:
  0
- do_eval:
  true
- do_predict:
  false
- do_sample:
  false
- do_train:
  false
- dtype:
  "bfloat16"
- early_stopping:
  false
- encoder_no_repeat_ngram_size:
  0
- eos_token_id:
  151,643
- eval_accumulation_steps:
  null
- vocab_size:
  152,064
- warmup_ratio:
  0.1
- warmup_steps:
  0
- weight_decay:
  0

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 14 keys
- eval/loss:
  0.1407092809677124
- eval/runtime:
  315.7667
- eval/samples_per_second:
  288.897
- eval/steps_per_second:
  0.754
- total_flos:
  564,184,193,811,087,360
- train_loss:
  0.2889569140949837
- train_runtime:
  60,254.9897
- train_samples_per_second:
  68.24
- train_steps_per_second:
  0.089
- train/epoch:
  5
- train/global_step:
  5,355
- train/grad_norm:
  0.9333163499832152
- train/learning_rate:
  0.00000000003824968373
- train/loss:
  0.2077

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-1z7kqb2q-history:v0