First Positive Result - NER on SciERC Benfits
Created on April 25|Last edited on April 25
Comment
In this run, we get Coref + Pruner working on Ontonotes, and get NER working on SciERC. As you can see in the panels below, doing so benefits NER.
We can't conclude if it benefits Coref+Pruner because of the problem we're facing.
I'll see what happens there. Soon.
Train Metrics
Run set
2
Here Span Recall chart represents the following computation:
- ignore what label is predicted. Only look at whether we predicted the span to have any label, or not a valid span label. And compare this to the gold annotations. As you can imagine, getting a high recall here would be difficult. For instance, its common in Ontonotes documents to have about 5000ish spans and only 100 or less named entities.
Valid Metrics
Run set
2
Similar things in train and valid. Which is nice.
We notice that the model both converges faster (in epochs) and converges better. Faster convergence is understandable and not that interesting since previously, we'd see only 350 docs in one epoch, and now we're seeing about 3000. So maybe the model doesn't converge faster in pure gradient update terms, only in epochs.
But what happens to Coref?
Sadly, coref is still not working. But we can do a comparison between just Ontonotes Coref (+Pruner) and this Coref (MTL with SciERC NER and Ontonotes Pruner)
Run set
2
Here, the performance is in fact worse than Vanilla Coref Ontonotes. It shold be said however, that I wouldn't trust either of these things because its possible that something is infact wrong here.
Add a comment