eleutherai

Preetham-gali's group workspace

Group: WmQb7XN9Q7AZW7Jp4GjYNg_1r8qwr27

2511

1-3

of 3

Timestamps visible

2021-08-27 00:25:50

[2021-08-27 00:25:48,838] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 8388608.0, reducing to 4194304.0

2021-08-27 00:25:54

[2021-08-27 00:25:52,925] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 4194304.0, reducing to 2097152.0

2021-08-27 00:25:59

[2021-08-27 00:25:57,014] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 2097152.0, reducing to 1048576.0

2021-08-27 00:26:03

[2021-08-27 00:26:01,102] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 1048576.0, reducing to 524288.0

2021-08-27 00:26:07

[2021-08-27 00:26:05,257] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 524288.0, reducing to 262144.0

2021-08-27 00:26:11

[2021-08-27 00:26:09,342] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 262144.0, reducing to 131072.0

2021-08-27 00:26:15

[2021-08-27 00:26:13,468] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 131072.0, reducing to 65536.0

2021-08-27 00:26:19

[2021-08-27 00:26:17,568] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 65536.0, reducing to 32768.0

2021-08-27 00:26:23

[2021-08-27 00:26:21,645] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 32768.0, reducing to 16384.0

2021-08-27 00:26:27

[2021-08-27 00:26:25,752] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 16384.0, reducing to 8192.0

2021-08-27 01:50:44

[2021-08-27 01:50:42,840] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 16384.0, reducing to 16384.0

2021-08-27 01:52:34

[2021-08-27 01:52:32,163] [INFO] [stage1.py:695:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 16384.0, reducing to 8192.0