Kastan's group workspace
gpt2_2d_TP4_PP2_8B
What makes this group special?
Tags
gpt2_2d_TP4_PP2_8B
Notes
Author
kastanState
Crashed
Start time
July 15th, 2022 7:03:11 PM
Runtime
32m 26s
Tracked hours
31m 2s
Run path
kastan/col_ai/ii8kmyg4
OS
Linux-4.18.0-305.49.1.el8_4.x86_64-x86_64-with-glibc2.28
Python version
3.9.12
Git repository
git clone https://github.com/hpcaitech/ColossalAI-Examples.git
Git state
git checkout -b "gpt2_2d_TP4_PP2_8B" f743872c2089d6bb5e593db6a8a48d427e6b2b1e
Command
/u/kastanday/new_colossal_ai/ColossalAI/examples/language/gpt/train_gpt.py --config gpt2_configs/gpt2_2d_TP4_PP2_8B.py --host gpub020 --port 29500 --world_size 8 --rank 2
System Hardware
| CPU count | 64 |
| GPU count | 4 |
| GPU type | NVIDIA A40 |
W&B CLI Version
0.12.21
Group
gpt2_2d_TP4_PP2_8BConfig
Config parameters are your model's inputs. Learn more
- {} 25 keys▶
- "nccl"
- 16
- "gpt2_configs/gpt2_2d_TP4_PP2_8B.py"
- "gpt2_configs/gpt2_2d_TP4_PP2_8B.py"
- "/u/kastanday/colossal_ai/raw_json_backup/train_data_FINAL.json"
- {} 1 key▶
- "AMP_TYPE.NAIVE"
- true
- false
- "titans.model.gpt.gpt.gpt2_8B"
- "titans.model.gpt.gpt.gpt2_small"
- "gpub020"
- null
- {} 1 key▶
- "titans.loss.lm_loss.gpt_lmloss.GPTLMLoss"
- "2d"
- {} 2 keys▶
- true
- "titans.model.gpt.gpt.gpt2_8B"
- 60
- 4
- {} 3 keys▶
- 0.00015
- "torch.optim.adam.Adam"
- 0.01
- {} 2 keys▶
- 2
- {} 2 keys▶
- "2d"
- 4
- 2
- 29,500
- 2
- 1,024
- 4
- 8
Summary
Summary metrics are your model's outputs. Learn more
- {} 3 keys▶
- "Hi, we got started"
- null
- 0.000025
Artifact Outputs
This run produced these artifacts as outputs. Total: 1. Learn more
Type
Name
Consumer count
Loading...