Training V6b (#2 Fine Tuning)

Data: V2 (258M Training Samples) Model: Custom DeBERTaV2-base (87M Parameters, Vocab Size 500) Task: Sequence Classification Fine-Tuning @ 29M Samples (0.11 Epoch) Started: 2022-09-19 Total Runtime: 56h [In Progress]

Jonathan Rahn

Created on September 23|Last edited on October 7

Comment

﻿
Results:Validation Accuracy: 41,4%* [In Progress]
Illegal Moves: 1,7%**
Chart:﻿
eval/accuracy
eval/accuracy
50k100k150k200ktrain/global_step0.10.20.30.4
train/loss, eval/loss
train/loss, eval/loss
50k100k150k200ktrain/global_step34567
﻿
* Literature Benchmark: ~38% [1], ~60% (AGZ-cnn @ KGS Test), [2]

** Self-play 200 games until result or illegal, top_k=3. Literature Benchmark: ~2% [1, 2]
﻿
﻿

Add a comment