Skip to main content
kastan
Projects
LLM-Distributed-Quantization
Run groups
Aug-05__13:35
Group
Workspace
Runs
Logs
Files
Artifacts

Kastan's group workspace

Aug-05__13:35

What makes this group special?
Tags

gpt2_1d

Notes
Tags
Aug-05__13:35
BATCH_SIZE32
NUM_EPOCHS=20
SLURM=513895
TP=4
WORLD_SIZE=4
Author
kastan
State
Crashed
Start time
August 5th, 2022 6:35:44 PM
Runtime
29m 30s
Tracked hours
29m 20s
Run path
kastan/LLM-Distributed-Quantization/ekuuoopx
OS
Linux-4.18.0-305.49.1.el8_4.x86_64-x86_64-with-glibc2.28
Python version
3.9.12
Command
/u/kastanday/LLM-Distributed-Quantization/benchmarks/gpt/v2_train.py --config /u/kastanday/LLM-Distributed-Quantization/benchmarks/gpt/configs/gpt2_1d.py --host gpub003 --port 29500 --world_size 4 --rank 3
System Hardware
CPU count64
GPU count4
GPU typeNVIDIA A40
W&B CLI Version
0.13.0
Group
Aug-05__13:35
Config

Config parameters are your model's inputs. Learn more

  • ▶
    {} 25 keys
    • 32
    • 1
    • "col_ai_quant"
    • "/u/kastanday/LLM-Distributed-Quantization/datasets/small-gpt-dataset.json"
    • ▶
      {} 1 key
      • "AMP_TYPE.NAIVE"
    • "titans.model.gpt.gpt.gpt2_8B"
    • "titans.model.gpt.gpt.gpt2_large"
    • "titans.model.gpt.gpt.gpt2_medium"
    • "titans.model.gpt.gpt.gpt2_xl"
    • 1
    • 0.00015
    • ▶
      {} 1 key
      • "titans.loss.lm_loss.gpt_lmloss.GPTLMLoss"
    • ▶
      {} 4 keys
      • true
      • "torch.float16"
      • 1,024
      • 50,304
    • 20
    • "4"
    • ▶
      {} 2 keys
      • 0.00015
      • 0.01
    • ▶
      {} 2 keys
      • 1
      • ▶
        {} 2 keys
        • "1d"
        • 4
    • 1,024
    • "1d"
    • 4
    • 32
    • "4"
    • 50,304
    • 1
    • 0.01
Summary

Summary metrics are your model's outputs. Learn more

  • ▶
    {} 4 keys
    • 0
    • 27.016342163085938
    • 3.1468
    • 8.3307
Artifact Outputs

This run produced these artifacts as outputs. Total: 1. Learn more

1-1
of 1
Type
Name
Consumer count
wandb-history
run-ekuuoopx-history:v0
0
Loading...