Skip to main content
haok
Projects
flame-moe
Run groups
ablation-1e18
Logs
Group
Workspace
Runs
Logs
Files
Artifacts

Annavettoruzzo's group workspace

Timestamps visible
2025-04-27 04:31:39
 [2025-04-27 00:31:39] iteration       40/      46 | consumed samples:        40960 | elapsed time per iteration (ms): 31744.8 | throughput per GPU (TFLOP/s/GPU): 85.4 | learning rate: 3.000000E-04 | global batch size:  1024 | lm loss: 7.192156E+00 | load_balancing_loss: 1.320723E+00 | loss scale: 1.0 | grad norm: 1.254 | num zeros: 0 | number of skipped iterations:   0 | number of nan iterations:   0 |
2025-04-27 04:34:20
 [2025-04-27 00:34:20] iteration       45/      46 | consumed samples:        46080 | elapsed time per iteration (ms): 32252.7 | throughput per GPU (TFLOP/s/GPU): 84.1 | learning rate: 8.108592E-05 | global batch size:  1024 | lm loss: 7.058396E+00 | load_balancing_loss: 1.208231E+00 | loss scale: 1.0 | grad norm: 0.930 | num zeros: 0 | number of skipped iterations:   0 | number of nan iterations:   0 |
2025-04-27 04:34:53
/project/flame/zichunyu/code/cxcscmu/MoE-Research/Megatron-LM/megatron/core/transformer/transformer_layer.py:339: UserWarning: TransformerLayer._get_layer_offset is deprecated.Please use get_transformer_layer_offset instead.
2025-04-27 04:34:53
  warnings.warn(
2025-04-27 05:05:19
(min, max) time across ranks (ms):
2025-04-27 05:05:19
    evaluate .......................................: (1732172.24, 1732173.22)
2025-04-27 05:05:20
----------------------------------------------------------------------------------------------------------------
2025-04-27 05:05:20
 validation loss at iteration 46 on validation set | lm loss value: 6.971900E+00 | lm loss PPL: 1.066246E+03 |
2025-04-27 05:05:20
----------------------------------------------------------------------------------------------------------------
2025-04-27 05:33:34
(min, max) time across ranks (ms):
2025-04-27 05:33:34
    evaluate .......................................: (1694190.19, 1694190.80)
2025-04-27 05:33:34
----------------------------------------------------------------------------------------------------------
2025-04-27 05:33:34
 validation loss at iteration 46 on test set | lm loss value: 6.973315E+00 | lm loss PPL: 1.067756E+03 |
2025-04-27 05:33:34
----------------------------------------------------------------------------------------------------------