Best Model Evaluation Study
Evaluating best model (dauntless-sweep-2) with both NMS and MOB bounding box aggregation and both evaluation schemes: standard PASCAL VOC2012 and our SAR-APD (HERIDAL) evaluation
Created on March 5|Last edited on March 5
Comment
Run label mapping
- swift-frog -> MOB with VOC2012 eval
- woven-waterfall -> MOB with SAR-APD eval
- easy-sound -> NMS with SAR-APD eval
- serene-energy -> NMS with VOC2012 eval
Findings
As evident from the bar charts, VOC2012 evaluation literally destroys MOB bounding box aggregation performance metrics, albeit upon visual inspection it's coarse object localisation makes a lot of sense in high-resolution images as an auxiliary support for human visual inspector during SAR missions.
On the other hand, NMS performance metrics stay roughly the same when switching from VOC2012 to SAR-APD evaluation, meaning for near perfect localisation the the two evaluation schemes converge.
Run set
4
Add a comment