Skip to main content
eleutherai
Projects
pythia
Run groups
v2-12B-deduped_5qq847kz
Logs
Overview
Workspace
Runs
Logs
Files
Artifacts

Atmallen8's group workspace

Timestamps visible
2023-02-24 14:36:23
ip-26-0-136-16:200303:201937 [0] NCCL INFO Connected all trees
2023-02-24 14:36:23
ip-26-0-136-16:200303:201937 [0] NCCL INFO threadThresholds 8/8/64 | 512/8/64 | 512 | 512
2023-02-24 14:36:23
ip-26-0-136-16:200303:201937 [0] NCCL INFO 4 coll channels, 4 p2p channels, 2 p2p channels per peer
2023-02-24 14:36:23
ip-26-0-136-16:200303:201937 [0] NCCL INFO comm 0x4f54e280 rank 12 nranks 64 cudaDev 0 busId 101c0 - Init COMPLETE
2023-02-24 14:36:23
Time to load utils op: 0.5043907165527344 seconds
2023-02-24 14:36:23
[2023-02-24 14:36:22,956] [INFO] [stage1.py:160:__init__] ZeRO Elastic Checkpoint = True
2023-02-24 14:36:23
Using ./new_extensions/ as PyTorch extensions root...
2023-02-24 14:36:23
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.195255972487674 seconds), retrying request
2023-02-24 14:36:23
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.975278290890667 seconds), retrying request
2023-02-24 14:36:23
Loading extension module utils...
2023-02-24 14:36:25
Time to load utils op: 0.0009489059448242188 seconds
2023-02-24 14:36:25
NCCL version 2.14.3+cuda11.7
2023-02-24 14:36:25
ip-26-0-136-16:200303:202237 [0] NCCL INFO Using network AWS Libfabric
2023-02-24 14:36:25
Using ./new_extensions/ as PyTorch extensions root...
2023-02-24 14:36:25
No modifications detected for re-loaded extension module utils, skipping build step...
2023-02-24 14:36:25
Loading extension module utils...
2023-02-24 14:36:27
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 9.742407944693143 seconds), retrying request