LayoutT5 fine-tuning graphics
Experiments on fine-tuning with different settings, hyperparameters and OCRs.
Created on February 16|Last edited on May 11
Comment
Fine-tuning graphicsValTrainTestHyperparametersPre-training effectValHyperparametersChange of OCRValHyperparameters
Fine-tuning graphics
All fine-tuning experiment have been performed by using the same hyperparameters. The difference between each run relies on the pre-training performed before.
- Red: No pretrain
- Green: Pretrain 31k (32x512), lr 1e-5
- Blue: Pretrain 31k (16x1024), lr 1e-5
- Brown: Pretrain 31k (16x1024), lr 2e-4
- Light blue: Pretrain 31k (16x1024), lr 2e-4. This is the only run where I change the fine-tunning configuration, which is the same as in pretrain (16x1024, lr 2e-4).
Val
Train
Test
Hyperparameters
Pre-training effect
Val
Hyperparameters
Change of OCR
Val
Hyperparameters
Add a comment