BigBird base NER fine-tuning results
A comparison of training results for the Feedback Prize competition
Created on December 19|Last edited on December 21
Comment
Training comparison
| parameter | bb-base | bb-base-idpt | |
|---|---|---|---|
| model name | google/bigbird-roberta-base | google/bigbird-roberta-base | |
| in-domain pre-training | No | Yes | |
| learning rate | 2e-5 | 2e-5 | |
| weight decay | 0.0095 | 0.0095 | |
| train batch size | 4 | 4 | |
| gradient accumulation steps | 8 | 8 | |
| fp16 | True | True | |
| seed | 18 | 18 |
Add a comment