eleutherai

Schoelkopf's group workspace

Group: mamba-tp2-bs16-fusedinner_3yv14en0_7pcsp4pn

5176

1-4

of 4

Timestamps visible

2024-03-13 11:25:13

[2024-03-13 11:25:11,331] [WARNING] [comm.py:152:init_deepspeed_backend] NCCL backend in DeepSpeed not yet implemented

2024-03-13 11:25:13

[2024-03-13 11:25:11,331] [INFO] [comm.py:594:init_distributed] cdb=None

2024-03-13 11:25:17

WARNING: APEX not installed - defaulting to deepspeed's fused adam

2024-03-13 11:25:18

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:25:18

Loading extension module fused_adam...

2024-03-13 11:25:19

Time to load fused_adam op: 1.0043566226959229 seconds

2024-03-13 11:25:19

[2024-03-13 11:25:17,420] [WARNING] [config_utils.py:69:_process_deprecated_field] Config parameter cpu_offload is deprecated use offload_optimizer instead

2024-03-13 11:25:20

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:25:21

Time to load utils op: 1.0051591396331787 seconds

2024-03-13 11:25:22

Loading extension module utils...

2024-03-13 11:25:25

Rank: 8 partition count [16, 16] and sizes[(5227008, False), (23088, False)]

2024-03-13 11:25:27

Time to load utils op: 0.0034363269805908203 seconds

2024-03-13 11:25:28

Using ./extensions/ as PyTorch extensions root...

2024-03-13 11:25:28

No modifications detected for re-loaded extension module utils, skipping build step...

2024-03-13 11:25:28

Loading extension module utils...

2024-03-13 11:26:45

> RANK 8 elapsed time for building blendable dataset indices: 1.03 (sec)

2024-03-13 11:26:45

> RANK 8 elapsed time for building blendable dataset indices: 1.01 (sec)

2024-03-13 11:26:47

> RANK 8 elapsed time for building blendable dataset indices: 1.00 (sec)