Experiments of LayoutCT5 trained on DocCVQA with different configurations. Trained using different amount of [PAGE] tokens and trained and tested on different number of pages.

Experiments on fine-tuning with different settings, hyperparameters and OCRs.

1 - Problem at initializing Iterations after pretraining 2 - Slight differences w and w/o pertaining 3 - Seems that could increase more (but not much). Purple: T5 Base Garnet: T5 Pretrain+Finetune Green: LayoutT5 Base Red: LayoutT5 Pretrain+Finetune

Blue: T5 Base (BS 4) Purple: T5 Base (BS 16) Green: LayoutT5 Base (BS16) * Green and purple overlaps 100% Layout = Sum(le,up,ri,bot,w,h) ==> + Semantic

Difference pretraining with BS 4 (Blue) or BS 16 (Purple)