Skip to main content

Igoro's group workspace

Timestamps visible
2021-11-02 23:00:12
[2021-11-02 23:00:07,005] [INFO] [distributed.py:46:init_distributed] Initializing torch distributed with backend: nccl
2021-11-02 23:00:12
make: Entering directory '/home/mchorse/gpt-neox/megatron/data'
2021-11-02 23:00:12
make: Nothing to be done for 'default'.
2021-11-02 23:00:12
make: Leaving directory '/home/mchorse/gpt-neox/megatron/data'
2021-11-02 23:00:16
[2021-11-02 23:00:14,444] [WARNING] [config.py:77:_sanity_check] DeepSpeedConfig: cpu_offload is deprecated. Please use offload_optimizer.
2021-11-02 23:00:20
Using /home/mchorse/.cache/torch_extensions/py38_cu111 as PyTorch extensions root...
2021-11-02 23:00:20
Emitting ninja build file /home/mchorse/.cache/torch_extensions/py38_cu111/utils/build.ninja...
2021-11-02 23:00:20
Building extension module utils...
2021-11-02 23:00:20
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
2021-11-02 23:00:20
ninja: no work to do.
2021-11-02 23:00:20
Loading extension module utils...
2021-11-02 23:00:20
Time to load utils op: 0.3602328300476074 seconds
2021-11-02 23:00:20
[2021-11-02 23:00:19,412] [INFO] [stage1.py:160:__init__] ZeRO Elastic Checkpoint = True
2021-11-02 23:00:22
Using /home/mchorse/.cache/torch_extensions/py38_cu111 as PyTorch extensions root...
2021-11-02 23:00:22
No modifications detected for re-loaded extension module utils, skipping build step...
2021-11-02 23:00:22
Loading extension module utils...
2021-11-02 23:00:22
Time to load utils op: 0.0008463859558105469 seconds