Skip to main content
kastan
Projects
LLM-Distributed-Quantization
Run groups
Aug-05__13:18
Group
Workspace
Runs
Logs
Files
Artifacts

Kastan's group workspace

Aug-05__13:18

What makes this group special?
Tags

gpt2_2p5d

Notes
Tags
Aug-05__13:18
BATCH_SIZE32
MICRO_BATCH_SIZE=4
NUM_EPOCHS=4
NUM_MICRO_BATCHES=8
PP=2
SLURM=513872
TP=4
WORLD_SIZE=32
Author
kastan
State
Crashed
Start time
August 5th, 2022 6:18:57 PM
Runtime
2m 24s
Tracked hours
-
Run path
kastan/LLM-Distributed-Quantization/28uokbhn
OS
Linux-4.18.0-305.49.1.el8_4.x86_64-x86_64-with-glibc2.28
Python version
3.9.12
Command
/u/kastanday/LLM-Distributed-Quantization/benchmarks/gpt/v2_train.py --config /u/kastanday/LLM-Distributed-Quantization/benchmarks/gpt/configs/gpt2_2p5d.py --host gpub005 --port 29500 --world_size 32 --rank 15
System Hardware
CPU count64
GPU count4
GPU typeNVIDIA A40
W&B CLI Version
0.13.0
Group
Aug-05__13:18
Config

Config parameters are your model's inputs. Learn more

  • ▶
    {} 27 keys
    • 32
    • 1
    • "col_ai_quant"
    • "/u/kastanday/LLM-Distributed-Quantization/datasets/small-gpt-dataset.json"
    • ▶
      {} 1 key
      • "AMP_TYPE.NAIVE"
    • "titans.model.gpt.gpt.gpt2_8B"
    • "titans.model.gpt.gpt.gpt2_xl"
    • 2
    • 0.00015
    • "./gpt2_2.5d_tp4_bs32_lr0.00015_accum2_clip_grad1.0/"
    • ▶
      {} 1 key
      • "titans.loss.lm_loss.gpt_lmloss.GPTLMLoss"
    • 4
    • ▶
      {} 4 keys
      • false
      • "torch.float16"
      • 1,024
      • 50,304
    • 4
    • "4"
    • 8
    • ▶
      {} 2 keys
      • 0.00015
      • 0.01
    • ▶
      {} 2 keys
      • 4
      • ▶
        {} 3 keys
        • 1
        • "2.5d"
        • 4
    • 2
    • 1,024
    • "2.5d"
    • 4
    • 64
    • "32"
    • 50,304
    • 1
    • 0.01
Summary

Summary metrics are your model's outputs. Learn more

No summary metrics saved for this run.

Check the summary metrics documentation for more information.