asap-zzhou

Asap-zzhou's group workspace

Group: bkup

1-5

of 5

Tags

Notes

Author

asap-zzhou

State

Crashed

Start time

October 25th, 2025 6:15:45 PM

Runtime

9h 9m 46s

Tracked hours

9h 9m 11s

Run path

asap-zzhou/dllm/eci7ed5u

Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.17

Python version

CPython 3.10.18

Git repository

git clone git@github.com:ZHZisZZ/dllm.git

Git state

git checkout -b "ModernBERT-large/openwebtext/steps-200000-bs-1024-len-1024" 199845c93a5162f7635d7caa087cc017605da26a

Command

/mnt/lustrenew/mllm_aligned/dongzhichen/dllm/examples/bert/pt.py --model_name_or_path answerdotai/ModernBERT-large --dataset_args dylanebert/openwebtext --text_field text --streaming True --insert_eos True --max_steps 200000 --max_length 1024 --per_device_train_batch_size 32 --warmup_steps 2000 --eval_strategy no --eval_on_start False --save_steps 0.01 --output_dir models/ModernBERT-large/openwebtext/steps-200000-bs-1024-len-1024

System Hardware

CPU count	64
Logical CPU count	128
GPU count	8
GPU type	NVIDIA A100-SXM4-80GB

W&B CLI Version

0.22.2

Group

bkup

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 219 keys
- _name_or_path:
  "/mnt/lustrenew/mllm_aligned/shared/models/huggingface/answerdotai/ModernBERT-large"
- ▶
  accelerator_config:{} 6 keys
- adafactor:
  false
- adam_beta1:
  0.9
- adam_beta2:
  0.999
- adam_epsilon:
  0.00000001
- add_cross_attention:
  false
- ▶
  architectures:[] 1 item
  - 0:
    "ModernBertForMaskedLM"
- attention_bias:
  false
- attention_dropout:
  0
- auto_find_batch_size:
  false
- average_tokens_across_devices:
  true
- bad_words_ids:
  null
- batch_eval_metrics:
  false
- begin_suppress_tokens:
  null
- bf16:
  true
- bf16_full_eval:
  false
- bos_token_id:
  null
- chunk_size_feed_forward:
  0
- classifier_activation:
  "gelu"
- classifier_bias:
  false
- classifier_dropout:
  0
- classifier_pooling:
  "mean"
- cls_token_id:
  50,281
- cross_attention_hidden_size:
  null
- data_seed:
  null
- dataloader_drop_last:
  false
- dataloader_num_workers:
  0
- dataloader_persistent_workers:
  false
- dataloader_pin_memory:
  true
- dataloader_prefetch_factor:
  null
- ddp_backend:
  null
- ddp_broadcast_buffers:
  null
- ddp_bucket_cap_mb:
  null
- ddp_find_unused_parameters:
  null
- ddp_timeout:
  1,800
- debug:[] 0 items
- decoder_bias:
  true
- decoder_start_token_id:
  null
- deepspeed:
  null
- deterministic_flash_attn:
  false
- disable_tqdm:
  false
- diversity_penalty:
  0
- do_eval:
  false
- do_predict:
  false
- do_sample:
  false
- vocab_size:
  50,368
- warmup_ratio:
  0.1
- warmup_steps:
  2,000
- weight_decay:
  0

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 5 keys
- train/epoch:
  0.0949
- train/global_step:
  18,980
- train/grad_norm:
  0.35546875
- train/learning_rate:
  0.00009819654469608436
- train/loss:
  2.8825

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-eci7ed5u-history:v0