fastai_community

Arampacha's group workspace

Group: snli-distilroberta-base-hf-3e-05

eval/accuracy

0.89433

0.87848

0.81721

0.90835

1-4

of 4

Timestamps visible

2021-04-22 00:26:55

Some weights of the model checkpoint at distilroberta-base were not used when initializing RobertaForSequenceClassification: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight', 'roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']

2021-04-22 00:26:55

- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).

2021-04-22 00:26:55

- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

2021-04-22 00:26:55

Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at distilroberta-base and are newly initialized: ['classifier.dense.weight', 'classifier.dense.bias', 'classifier.out_proj.weight', 'classifier.out_proj.bias']

2021-04-22 00:26:55

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

2021-04-22 01:10:12

Starting virtual adversarial training at epoch 2.0