Skip to main content
kimsehun725
Projects
my_vae
Run groups
20221017_2205
Logs
Group
Workspace
Runs
Logs
Files
Artifacts

Kimsehun725's group workspace

Timestamps visible
2022-10-17 13:05:42
TPU available: False, using: 0 TPU cores
2022-10-17 13:05:42
IPU available: False, using: 0 IPUs
2022-10-17 13:05:42
HPU available: False, using: 0 HPUs
2022-10-17 13:05:50
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/4
2022-10-17 13:05:52
----------------------------------------------------------------------------------------------------
2022-10-17 13:05:52
distributed_backend=nccl
2022-10-17 13:05:52
All distributed processes registered. Starting with 4 processes
2022-10-17 13:05:52
----------------------------------------------------------------------------------------------------
2022-10-17 13:06:02
Sanity Checking: 0it [00:00, ?it/s]
2022-10-17 13:06:02
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
2022-10-17 13:06:02
  | Name      | Type          | Params
2022-10-17 13:06:02
--------------------------------------------
2022-10-17 13:06:02
0 | encoder   | Conv1DEncoder | 1.8 M
2022-10-17 13:06:02
1 | fc1       | Linear        | 23.1 K
2022-10-17 13:06:02
2 | fc2       | Linear        | 23.1 K
2022-10-17 13:06:02
3 | fc3       | Linear        | 23.4 K
2022-10-17 13:06:02
4 | decoder_0 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
5 | decoder_1 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
6 | decoder_2 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
7 | decoder_3 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
8 | decoder_4 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
9 | decoder_5 | Conv1DDecoder | 1.8 M
2022-10-17 13:06:02
--------------------------------------------
2022-10-17 13:06:02
13.0 M    Trainable params
2022-10-17 13:06:02
0         Non-trainable params
2022-10-17 13:06:02
13.0 M    Total params
2022-10-17 13:06:02
51.971    Total estimated model params size (MB)
2022-10-17 13:06:06
Traceback (most recent call last):
2022-10-17 13:06:06
  File "src/train.py", line 65, in <module>
2022-10-17 13:06:06
    main(kwargs)
2022-10-17 13:06:06
  File "src/train.py", line 23, in main
2022-10-17 13:06:06
    step2(kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/src/step2.py", line 149, in step2
2022-10-17 13:06:06
    train(kwargs, now, test_num, train_data_list,
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/src/step2.py", line 119, in train
2022-10-17 13:06:06
    trainer.fit(model, train_dataloaders=train_loader,
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit
2022-10-17 13:06:06
    self._call_and_handle_interrupt(
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 648, in _call_and_handle_interrupt
2022-10-17 13:06:06
    return self.strategy.launcher.launch(trainer_fn, *args, trainer=self, **kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 93, in launch
2022-10-17 13:06:06
    return function(*args, **kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 735, in _fit_impl
2022-10-17 13:06:06
    results = self._run(model, ckpt_path=self.ckpt_path)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1166, in _run
2022-10-17 13:06:06
    results = self._run_stage()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1252, in _run_stage
2022-10-17 13:06:06
    return self._run_train()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1274, in _run_train
2022-10-17 13:06:06
    self._run_sanity_check()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1343, in _run_sanity_check
2022-10-17 13:06:06
    val_loop.run()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run
2022-10-17 13:06:06
    self.advance(*args, **kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 155, in advance
2022-10-17 13:06:06
    dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run
2022-10-17 13:06:06
    self.advance(*args, **kwargs)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 127, in advance
2022-10-17 13:06:06
    batch = next(data_fetcher)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/utilities/fetching.py", line 184, in __next__
2022-10-17 13:06:06
    return self.fetching_function()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/utilities/fetching.py", line 263, in fetching_function
2022-10-17 13:06:06
    self._fetch_next_batch(self.dataloader_iter)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/pytorch_lightning/utilities/fetching.py", line 277, in _fetch_next_batch
2022-10-17 13:06:06
    batch = next(iterator)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 681, in __next__
2022-10-17 13:06:06
    data = self._next_data()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
2022-10-17 13:06:06
    return self._process_data(data)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
2022-10-17 13:06:06
    data.reraise()
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/_utils.py", line 461, in reraise
2022-10-17 13:06:06
    raise exception
2022-10-17 13:06:06
NameError: Caught NameError in DataLoader worker process 0.
2022-10-17 13:06:06
Original Traceback (most recent call last):
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
2022-10-17 13:06:06
    data = fetcher.fetch(index)
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
2022-10-17 13:06:06
    data = [self.dataset[idx] for idx in possibly_batched_index]
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/venv/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
2022-10-17 13:06:06
    data = [self.dataset[idx] for idx in possibly_batched_index]
2022-10-17 13:06:06
  File "/data/group1/z44543r/vae_separation/src/step2.py", line 56, in __getitem__
2022-10-17 13:06:06
    x = np.load(self.data_list[idx]+ "mic_" + feature_type + ".npy")
2022-10-17 13:06:06
NameError: name 'feature_type' is not defined