Skip to main content
eleutherai
Projects
pythia
Run groups
v2-12B-deduped_2rfw9acn
Logs
Overview
Workspace
Runs
Logs
Files
Artifacts

Atmallen8's group workspace

Timestamps visible
2023-02-24 14:39:28
Added key: store_based_barrier_key:320 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:320 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:321 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:321 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:322 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:322 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:323 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:323 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:324 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:324 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:325 to store for rank: 88
2023-02-24 14:39:28
Rank 88: Completed store-based barrier for key:store_based_barrier_key:325 with 256 nodes.
2023-02-24 14:39:28
Added key: store_based_barrier_key:326 to store for rank: 88
2023-02-24 14:39:30
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.262323522880723 seconds), retrying request
2023-02-24 14:39:38
Waiting in store based barrier to initialize process group for rank: 88, key: store_based_barrier_key:326 (world_size=256, worker_count=252, timeout=0:30:00)
2023-02-24 14:39:40
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.2571714454084315 seconds), retrying request
2023-02-24 14:39:46
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.917783341008535 seconds), retrying request