alpha_sched_2.0 | vat Overview – Weights & Biases

Skip to main content

Arampacha's group workspace

Group: alpha_sched_2.0

eval/accuracy

-

-

-

1-3

of 3

Tags

Notes

Testing different alpha schedulers

Tags

alpha_sched_exp

Author

State

Finished

Start time

April 28th, 2021 11:54:35 AM

Runtime

37m 58s

Tracked hours

37m 53s

Run path

fastai_community/vat/8mqkmun1

OS

Linux-5.8.11-050811-lowlatency-x86_64-with-debian-buster-sid

Python version

3.7.7

Git repository

git clone https://github.com/aikindergarten/vat.git

Git state

git checkout -b "apha_sched_4983275" 9a37f7c813f7f670c21161345b7eb16624f91322

Command

vat/nbs/_alum_exps_lm_finetuning_fastai_blurr_imdb.ipynb

System Hardware

CPU count	20
GPU count	1
GPU type	GeForce RTX 2080 Ti

W&B CLI Version

0.10.27

Group

alpha_sched_2.0

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 23 keys
- alpha_max:
  2
- ▶
  ALUMCallback:{} 3 keys
  - alpha:
    1
  - m:
    "Embeddings( (word_embeddings): Embedding(30522, 768, padding_idx=0) (position_embeddings): Embedding(512, 768) (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True) (dropout): Dropout(p=0.1, inplace=False) )"
  - start_epoch:
    0
- batch per epoch:
  3,125
- batch size:
  8
- dataset.tfms:
  "[Pipeline: read_text, Pipeline: parent_label -> Categorize -- {'vocab': None, 'sort': True, 'add_na': False}]"
- device:
  "cuda"
- dls.after_batch:
  "Pipeline: HF_AfterBatchTransform"
- dls.after_item:
  "Pipeline: ToTensor"
- dls.before_batch:
  "Pipeline: HF_BeforeBatchTransform"
- exp_seed:
  4,983,275
- frozen:
  false
- frozen idx:
  0
- ▶
  GradientAccumulation:{} 1 key
  - n_acc:
    8
- HF_BaseModelCallback:
  true
- ▶
  Learner:{} 12 keys
- MixedPrecision:
  true
- model parameters:
  66,955,010
- n_inp:
  1
- ParamScheduler:
  true
- ProgressCallback:
  true
- ▶
  Recorder:{} 3 keys
  - add_time:
    true
  - train_metrics:
    false
  - valid_metrics:
    true
- TrainEvalCallback:
  true
- ▶
  WandbCallback:{} 9 keys

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 37 keys
- accuracy:
  0.9409199953079224
- alpha_0:
  0.00002005611077815495
- alpha_1:
  0.00002005611077815495
- alpha_2:
  0.00002005611077815495
- alpha_3:
  0.00002005611077815495
- alpha_max_0:
  2
- alpha_max_1:
  2
- alpha_max_2:
  2
- alpha_max_3:
  2
- beta_0:
  0
- beta_1:
  0
- beta_2:
  0
- beta_3:
  0
- epoch:
  4
- eps_0:
  0.00001
- eps_1:
  0.00001
- eps_2:
  0.00001
- eps_3:
  0.00001
- lr_0:
  0.00000000020056110778
- lr_1:
  0.00000000020056110778
- lr_2:
  0.00000000020056110778
- lr_3:
  0.00000000020056110778
- mom_0:
  0.949999997194433
- mom_1:
  0.949999997194433
- mom_2:
  0.949999997194433
- mom_3:
  0.949999997194433
- raw_loss:
  0.07460817694664001
- sqr_mom_0:
  0.99
- sqr_mom_1:
  0.99
- sqr_mom_2:
  0.99
- sqr_mom_3:
  0.99
- train_loss:
  0.08746294677257538
- valid_loss:
  0.1676061451435089
- wd_0:
  0
- wd_1:
  0
- wd_2:
  0
- wd_3:
  0

This run produced these artifacts as outputs. Total: 1. Learn more

run-8mqkmun1-history:v0

Loading...