Training V6a (#2 Fine Tuning)

Data: V1 (18M Training Samples) Model: Custom DeBERTaV2-base (87M Parameters, Vocab Size 500) Task: Sequence Classification Fine-Tuning @ 18M Samples (1.0 Epoch) Started: 2022-09-19 Total Runtime: 16h

Jonathan Rahn

Created on October 6|Last edited on October 6

Comment

﻿
Results:Validation Accuracy: 33,27%*
Illegal Moves: 1,5%**
Chart:﻿
eval/accuracy
eval/accuracy
20k40k60ktrain/global_step0.050.10.150.20.250.3
train/loss, eval/loss
train/loss, eval/loss
20k40k60ktrain/global_step3456
﻿
* Literature Benchmark: ~38% [1], ~60% (AGZ-cnn @ KGS Test), [2]

** Self-play 200 games until result or illegal, top_k=3. Literature Benchmark: ~2% [1, 2]
﻿
﻿

Add a comment