snli-distilroberta-base-hf-3e-05 | vat Workspace – Weights & Biases

Skip to main content

Arampacha's group workspace

Group: snli-distilroberta-base-hf-3e-05

eval/accuracy

0.89433

0.87848

0.81721

0.90835

1-4

of 4

Tags

Notes

HF finetuning distilroberta-base with AdamW lr=3e-05

Tags

adamw

alum

distilroberta-base

snli

Author

State

Finished

Start time

April 22nd, 2021 12:26:51 AM

Runtime

8h 20m 28s

Tracked hours

2h 14m 56s

Run path

fastai_community/vat/2hh91a6n

OS

Linux-4.15.0-123-generic-x86_64-with-debian-buster-sid

Python version

3.7.6

Command

<python with no main file> -f /workspace/.local/share/jupyter/runtime/kernel-35fc04fd-86fc-47d0-adae-b843e09e0751.json

System Hardware

CPU count	56
GPU count	1
GPU type	Tesla V100S-PCIE-32GB

W&B CLI Version

0.10.12

Group

snli-distilroberta-base-hf-3e-05

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 130 keys
- _n_gpu:
  1
- _name_or_path:
  "distilroberta-base"
- adafactor:
  false
- adam_beta1:
  0.9
- adam_beta2:
  0.999
- adam_epsilon:
  0.00000001
- add_cross_attention:
  false
- ▶
  architectures:[] 1 item
  - 0:
    "RobertaForMaskedLM"
- attention_probs_dropout_prob:
  0.1
- bad_words_ids:
  null
- bos_token_id:
  0
- chunk_size_feed_forward:
  0
- dataloader_drop_last:
  false
- dataloader_num_workers:
  4
- dataloader_pin_memory:
  true
- ddp_find_unused_parameters:
  "None"
- debug:
  false
- decoder_start_token_id:
  null
- deepspeed:
  "None"
- disable_tqdm:
  false
- diversity_penalty:
  0
- do_eval:
  "None"
- do_predict:
  false
- do_sample:
  false
- do_train:
  false
- early_stopping:
  false
- encoder_no_repeat_ngram_size:
  0
- eos_token_id:
  2
- eval_accumulation_steps:
  "None"
- eval_batch_size:
  128
- eval_steps:
  200
- evaluation_strategy:
  "epoch"
- finetuning_task:
  null
- forced_bos_token_id:
  null
- forced_eos_token_id:
  null
- fp16:
  true
- fp16_backend:
  "auto"
- fp16_full_eval:
  false
- fp16_opt_level:
  "O1"
- gradient_accumulation_steps:
  1
- gradient_checkpointing:
  false
- greater_is_better:
  "None"
- group_by_length:
  true
- hidden_act:
  "gelu"
- hidden_dropout_prob:
  0.1
- hidden_size:
  768
- vocab_size:
  50,265
- warmup_ratio:
  0.2
- warmup_steps:
  0
- weight_decay:
  0

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 11 keys
- eval/accuracy:
  0.8943304206462102
- eval/loss:
  0.3233351707458496
- eval/runtime:
  5.2036
- eval/samples_per_second:
  1,891.369
- train/epoch:
  5
- train/global_step:
  42,920
- train/learning_rate:
  0.00000010484622553588
- train/loss:
  0.5583
- train/total_flos:
  38,199,174,122,031,200
- train/train_runtime:
  7,799.8248
- train/train_samples_per_second:
  5.503

This run produced these artifacts as outputs. Total: 1. Learn more

run-2hh91a6n-history:v0

Loading...