Skip to main content
eleutherai
Projects
pythia
Run groups
v2-12B-deduped_8iyl2s8o
Logs
Group
Workspace
Runs
Logs
Files
Artifacts

Atmallen8's group workspace

Timestamps visible
2023-02-24 14:32:55
ip-26-0-143-178:3896069:3897865 [0] NCCL INFO Connected all trees
2023-02-24 14:32:55
ip-26-0-143-178:3896069:3897865 [0] NCCL INFO threadThresholds 8/8/64 | 512/8/64 | 512 | 512
2023-02-24 14:32:55
ip-26-0-143-178:3896069:3897865 [0] NCCL INFO 4 coll channels, 4 p2p channels, 2 p2p channels per peer
2023-02-24 14:32:55
ip-26-0-143-178:3896069:3897865 [0] NCCL INFO comm 0x4f12b400 rank 60 nranks 64 cudaDev 0 busId 101c0 - Init COMPLETE
2023-02-24 14:32:55
Time to load utils op: 0.3044149875640869 seconds
2023-02-24 14:32:55
[2023-02-24 14:32:55,606] [INFO] [stage1.py:160:__init__] ZeRO Elastic Checkpoint = True
2023-02-24 14:32:55
Using ./new_extensions/ as PyTorch extensions root...
2023-02-24 14:32:55
Loading extension module utils...
2023-02-24 14:32:57
Time to load utils op: 0.0017242431640625 seconds
2023-02-24 14:32:57
NCCL version 2.14.3+cuda11.7
2023-02-24 14:32:57
ip-26-0-143-178:3896069:3898165 [0] NCCL INFO Using network AWS Libfabric
2023-02-24 14:32:57
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.480826430992361 seconds), retrying request
2023-02-24 14:32:57
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.384385734278023 seconds), retrying request
2023-02-24 14:32:57
Using ./new_extensions/ as PyTorch extensions root...
2023-02-24 14:32:57
No modifications detected for re-loaded extension module utils, skipping build step...
2023-02-24 14:32:57
Loading extension module utils...
2023-02-24 14:33:01
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 9.105675874886739 seconds), retrying request