Final data and model evaluation for the Duckietown object detection task
Created on April 8|Last edited on April 8
Comment
Following the initial exploratory data analysis, the label distribution table of the Duckietown object detection dataset indicated an unbalanced class distribution:
- duckie: 51.7% of images have 0 duckies
- cone: 92.8% of images have 0 cones
- duckiebot: 11.5% of images have 0 duckiebots
This observation is confirmed by the following labels analysis:
Run: playful-plasma-1
1
The dataset was therefore split into training/validation/test subsets (80%/10%/10%) stratified by cone, duckie, and duckiebot fields resulting in the following balanced subsets distributions:
With that stratified partition, YOLOv5 model was trained/fine-tuned resulting in the following grid that summarizes the model fine-tuning into 6 insights:
- Losses (training/validation) as well as 4 tracked metrics table: precision/recall/mAP_0.5/mAP_0.5:0.95 - picked as typical metrics for the given object detection task
- Confusion matrix for predicted classes
- F1-Confidence curve
- Precision-Recall curve
- Precision-Confidence curve
- Recall-Confidence curve
Run: playful-plasma-1
1
A closer inspection of the class-specific metrics extracted from the logs shows
Class Images Instances P R mAP50 mAP50-95:
all 176 479 0.91 0.923 0.961 0.617
cone 176 30 0.93 0.886 0.975 0.549
duckie 176 259 0.838 0.92 0.925 0.595
duckiebot 176 190 0.96 0.963 0.985 0.706
that the model had the hardest time with duckie class - the relatively lower precision for this class can be explained by the increased false positive rate for that class - a further inspection of images confirms however that some of these "false positive" predictions are indeed just positive - the detected duckies are correct but were simply not labelled as such in the ground truth as they were probably considered too far away / too small.
Overall the following insights indicate a good result of fine-tuning YOLOv5 on the duckietown object detection dataset. This concludes the evaluation.
💡
Add a comment