Skip to main content

Kastan's group workspace

Timestamps visible
2022-08-05 19:00:03
Wandb run id:  3nsgkzdv
2022-08-05 19:00:03
[08/05/22 13:59:51] INFO     colossalai - colossalai - INFO:
2022-08-05 19:00:03
                             /u/kastanday/LLM-Distributed-Quantization/wandb_log
2022-08-05 19:00:03
                             s/custom_wandb_log_hook.py:122 before_train
2022-08-05 19:00:03
                    INFO     colossalai - colossalai - INFO: training starts
2022-08-05 19:00:03
wandb: WARNING Saving files without folders. If you want to preserve sub directories pass base_path to wandb.save, i.e. wandb.save("/mnt/folder/file.h5", base_path="/mnt")
2022-08-05 19:00:23
❌ Error logging layer norms -- check message: 'NaiveAMPModel' object has no attribute 'embed'
2022-08-05 19:12:30
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.117022481678963 seconds), retrying request
2022-08-05 19:12:30
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.760765696975028 seconds), retrying request
2022-08-05 19:19:23
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.2328531305964003 seconds), retrying request
2022-08-05 19:19:27
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.882083303393472 seconds), retrying request
2022-08-05 19:19:33
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 8.345803798408085 seconds), retrying request
2022-08-05 19:26:48
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.0028602301420504 seconds), retrying request