Kastan's group workspace
gpt2_1d_manual
What makes this group special?
Tags
gpt2_1d_manual
Notes
Author
kastanState
Finished
Start time
July 15th, 2022 6:06:11 PM
Runtime
1m 22s
Tracked hours
1m 19s
Run path
kastan/col_ai/2rt3t910
OS
Linux-4.18.0-305.49.1.el8_4.x86_64-x86_64-with-glibc2.28
Python version
3.9.12
Git repository
git clone https://github.com/hpcaitech/ColossalAI-Examples.git
Git state
git checkout -b "gpt2_1d_manual" f743872c2089d6bb5e593db6a8a48d427e6b2b1e
Command
/u/kastanday/new_colossal_ai/ColossalAI/examples/language/gpt/train_gpt.py --config gpt2_configs/gpt2_1d_manual.py --host gpub020 --world_size 2 --rank 8 --port 29500
System Hardware
| CPU count | 64 |
| GPU count | 4 |
| GPU type | NVIDIA A40 |
W&B CLI Version
0.12.21
Group
gpt2_1d_manualConfig
Config parameters are your model's inputs. Learn more
- {} 20 keys▶
- "nccl"
- 1
- "gpt2_configs/gpt2_1d_manual.py"
- "gpt2_configs/gpt2_1d_manual.py"
- "/u/kastanday/colossal_ai/raw_json_backup/train_data_FINAL.json"
- {} 1 key▶
- "AMP_TYPE.NAIVE"
- false
- "titans.model.gpt.gpt.gpt2_small"
- "gpub020"
- null
- {} 1 key▶
- "titans.loss.lm_loss.gpt_lmloss.GPTLMLoss"
- {} 2 keys▶
- true
- "titans.model.gpt.gpt.gpt2_small"
- 60
- {} 3 keys▶
- 0.00015
- "torch.optim.adam.Adam"
- 0.01
- {} 2 keys▶
- 1
- {} 2 keys▶
- "1d"
- 2
- 29,500
- 8
- 1,024
- 2
- 2
Summary
Summary metrics are your model's outputs. Learn more
- {} 6 keys▶
- "Goodbye, we finished training"
- "Hi, we got started"
- 27.39257049560547
- 27.39257049560547
- 1.857
- 1.723
Artifact Outputs
This run produced these artifacts as outputs. Total: 1. Learn more
Type
Name
Consumer count
Loading...