embedding-gradient-shrink | gpt Overview – Weights & Biases

Skip to main content

W&B is scheduled for routine maintenance on Friday, Nov 14th at 6:00 PM PST. The UI and API may be intermittently unavailable during this time. Thank you for your patience and visit https://status.wandb.com for updates.

Clashluke's group workspace

Group: embedding-gradient-shrink

1-4

of 4

Tags

Notes

Author

State

Crashed

Start time

December 30th, 2022 8:40:39 PM

Runtime

23h 55m 35s

Tracked hours

23h 54m 17s

Run path

homebrewnlp/gpt/4yjnleczht4eyne1ztykmwzcqbp11zwc

OS

Linux-5.4.0-1043-gcp-x86_64-with-glibc2.29

Python version

3.8.5

Git repository

git clone https://github.com/HomebrewNLP/HomebrewNLP-Jax

Git state

git checkout -b "embedding-gradient-shrink-3" 5a89473260f5bc4851f84dbd95a8de44cdfb9635

Command

main.py

System Hardware

CPU count	24
Logical CPU count	48

W&B CLI Version

0.13.7

Group

embedding-gradient-shrink

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 13 keys
- add_depth:
  false
- ▶
  data:{} 8 keys
- depth:
  0
- ▶
  dims:{} 11 keys
- fail_on_missing_parameter:
  true
- global_prefix:
  ""
- ▶
  model:{} 6 keys
- name_cache_offsets:{} 0 keys
- ▶
  optimizer:{} 11 keys
- parameter_usages:{} 0 keys
- seed:
  0
- ▶
  training:{} 10 keys
- ▶
  wandb:{} 6 keys

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 13 keys
- Accuracy/Current:
  0.7559683322906494
- Accuracy/Median64:
  0.7537310123443604
- Accuracy/Median256:
  0.7531439065933228
- Accuracy/Median1024:
  0.7526698112487793
- Loss/Current:
  0.7960879004836698
- Loss/Median64:
  0.8041418928683686
- Loss/Median256:
  0.8051246684271094
- Loss/Median1024:
  0.8072759046957019
- Optimizer/Learning Rate:
  0.00875752791762352
- Speed/Batches per Second:
  0.07293057966967031
- Speed/Parameters * Tokens per Day:
  2,677,117,153,970,706,432
- Speed/Tokens per Day:
  6,607,289,275.865644
- Speed/Tokens Seen:
  19,873,660,928

This run produced these artifacts as outputs. Total: 1. Learn more

run-4yjnleczht4eyne1ztykmwzcqbp11zwc-history:v0

Loading...