eleutherai

Schoelkopf's group workspace

Group: mamba-tp1-bs16-fusedinner_r7kwr8b2_11cu2i6r

2768

1-4

of 4

Timestamps visible

2024-03-13 11:32:23

[2024-03-13 11:32:21,443] [WARNING] [comm.py:152:init_deepspeed_backend] NCCL backend in DeepSpeed not yet implemented

2024-03-13 11:32:23

[2024-03-13 11:32:21,443] [INFO] [comm.py:594:init_distributed] cdb=None

2024-03-13 11:32:27

WARNING: APEX not installed - defaulting to deepspeed's fused adam

2024-03-13 11:32:29

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:32:29

Loading extension module fused_adam...

2024-03-13 11:32:29

Time to load fused_adam op: 1.01784348487854 seconds

2024-03-13 11:32:29

[2024-03-13 11:32:28,025] [WARNING] [config_utils.py:69:_process_deprecated_field] Config parameter cpu_offload is deprecated use offload_optimizer instead

2024-03-13 11:32:33

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:32:33

Loading extension module utils...

2024-03-13 11:32:33

Time to load utils op: 1.0041322708129883 seconds

2024-03-13 11:32:35

Rank: 16 partition count [32, 32] and sizes[(5220864, False), (22488, False)]

2024-03-13 11:32:39

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:32:39

No modifications detected for re-loaded extension module utils, skipping build step...

2024-03-13 11:32:39

Loading extension module utils...

2024-03-13 11:32:39

Time to load utils op: 0.0051877498626708984 seconds

2024-03-13 11:33:33

> RANK 16 elapsed time for building blendable dataset indices: 1.06 (sec)

2024-03-13 11:33:33

> RANK 16 elapsed time for building blendable dataset indices: 1.06 (sec)

2024-03-13 11:33:35

> RANK 16 elapsed time for building blendable dataset indices: 1.06 (sec)