Skip to main content
kastan
Projects
col_ai
Run groups
Jul-27__17:41
Logs
W&B is scheduled for routine maintenance on Friday, Nov 14th at 6:00 PM PST. The UI and API may be intermittently unavailable during this time. Thank you for your patience and visit https://status.wandb.com for updates.
Group
Workspace
Runs
Logs
Files
Artifacts

Kastan's group workspace

Timestamps visible
2022-07-27 23:12:57
frame #6: c10d::PrefixStore::get(std::string const&) + 0x31 (0x7f40324718d1 in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so)
2022-07-27 23:12:57
frame #7: c10d::ProcessGroupNCCL::broadcastUniqueNCCLID(ncclUniqueId*, bool, std::string const&, int) + 0xab (0x7f4047f9833b in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so)
2022-07-27 23:12:57
frame #8: c10d::ProcessGroupNCCL::getNCCLComm(std::string const&, std::vector<c10::Device, std::allocator<c10::Device> > const&, c10d::OpType, int, bool) + 0x1fe (0x7f4047f9c0de in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so)
2022-07-27 23:12:57
frame #9: <unknown function> + 0x1ff1d6 (0x7f4047fa31d6 in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so)
2022-07-27 23:12:57
frame #10: c10d::ProcessGroupNCCL::allreduce_impl(std::vector<at::Tensor, std::allocator<at::Tensor> >&, c10d::AllreduceOptions const&) + 0x10 (0x7f4047fa45d0 in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so)
2022-07-27 23:12:57
frame #11: c10d::ProcessGroupNCCL::allreduce(std::vector<at::Tensor, std::allocator<at::Tensor> >&, c10d::AllreduceOptions const&) + 0x2ac (0x7f4047fa631c in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cpp.so)
2022-07-27 23:12:57
frame #12: <unknown function> + 0x9f9f93 (0x7f4055a63f93 in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_python.so)
2022-07-27 23:12:57
frame #13: <unknown function> + 0x36bc3d (0x7f40553d5c3d in /u/kastanday/.conda/envs/nice_base/envs/col_ai_old_v5/lib/python3.9/site-packages/torch/lib/libtorch_python.so)
2022-07-27 23:12:57
<omitting python frames>
2022-07-27 23:12:57
frame #54: __libc_start_main + 0xf3 (0x7f4083c22493 in /lib64/libc.so.6)