Worker calibration

Created on September 10|Last edited on September 10
Comment
﻿
Is worker estimation well-calibrated?10 classes, 400 images per class
﻿
﻿
﻿
worker/all/ece
worker/all/ece
Showing first 10 runs
0500m11.522.5cost-per-image0.10.20.30.4
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=false
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=true
09-11-20-aggregate.algo=bayes+lp-worker.known=false
09-11-20-aggregate.algo=bayes+lp-worker.known=true
09-10-20-online.sample=kcenter-worker.known=true
09-10-20-online.sample=kcenter-worker.known=false
09-10-20-online.sample=random-worker.known=false
09-10-20-online.sample=random-worker.known=true
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.9
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.8
worker/invalid/ece
worker/invalid/ece
Showing first 10 runs
0500m11.522.5cost-per-image0.10.20.30.40.50.6
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=false
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=true
09-11-20-aggregate.algo=bayes+lp-worker.known=false
09-11-20-aggregate.algo=bayes+lp-worker.known=true
09-10-20-online.sample=kcenter-worker.known=true
09-10-20-online.sample=kcenter-worker.known=false
09-10-20-online.sample=random-worker.known=false
09-10-20-online.sample=random-worker.known=true
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.9
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.8
worker/valid/ece
worker/valid/ece
Showing first 10 runs
0500m11.522.5cost-per-image0.10.20.30.40.5
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=false
09-11-20-aggregate.algo=bayes+dist-lp-worker.known=true
09-11-20-aggregate.algo=bayes+lp-worker.known=false
09-11-20-aggregate.algo=bayes+lp-worker.known=true
09-10-20-online.sample=kcenter-worker.known=true
09-10-20-online.sample=kcenter-worker.known=false
09-10-20-online.sample=random-worker.known=false
09-10-20-online.sample=random-worker.known=true
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.9
09-10-20-risk_thres=0.9-aggregate.learner_thres=0.8
Run set2559
﻿
Distributional label results in poorly-calibrated model
﻿
﻿
﻿
Run set6
﻿
﻿
Add a comment