Practical crowdsourcing (w/ stopping criterion)
Practical crowdsourcing should monitor the confidence of the label and set the stopping criterion.
Created on July 22|Last edited on July 27
Comment
Desiderata: Higher accuracy & higher convergence rate
- kcenter doesn't quite help still
-
k-NN performance drops a lot when using dimension reduced features
-
k-NN has a long tail -> not calibrated?
p(yi∣Z,xi)∝p(yi∣xi,Z/i)p(yi∣zi)p(y_i|Z, x_i) \propto p(y_i|x_i, Z_{/i}) p(y_i|z_i)
- p(yi∣xi,Z/i)=∏j∼Nip(yj∣zj)s(i,j)p(y_i|x_i, Z_{/i}) = \prod_{\mathcal{j \sim N_i}} p(y_j|z_j)^{s(i, j)}
- p(yi∣xi,Z/i)=∏j∼Nip(yj∣zj)p(eij∣xi,xj)=∏j∼Nip(yj∣zj)s(i,j)p(y_i|x_i, Z_{/i}) = \prod_{\mathcal{j \sim N_i}} p(y_j|z_j) p(e_{ij}| x_i, x_j) = \prod_{\mathcal{j \sim N_i}} p(y_j|z_j) s(i, j)
Random Sample
13
13
Random Sample
14
Kcenter
14
Add a comment