Final results - MSc Thesis
Which Feature Rankers are best? In this experiment, we test feature rankers on various datasets to find out. Both real-world and synthetic datasets are included. This dashboard shows the results.
Created on June 16|Last edited on July 2
Comment
Experimental results for the `Synclf hard' dataset
ReliefF performance
This first experiment shows the results for just one ranker, on one dataset: ReliefF on Synclf hard.
Synclf hard
1
Performance of multiple rankers
- FeatBoost
- Infinite Selection
- MultiSURF
- ReliefF
- Stability Selection
Synclf hard
5
Experimental results for all datasets
Experiments run on all feature rankers and all datasets. Overview plots.
Mean validation scores
Plot for both regression- and classification. Use the wheel to select different validators or tasks:
^ click this wheel in the chart
^ choose between different validators and dataset tasks.
Note the color scale is currently broken for regression: ignore the color in that case. This is due to the negative values in the R2 score. Also, the validators considered are DT and k-NN. k-NN is classification-only so is not available for regression datasets.
See the overview:
All datasets
415
Synthetic datasets
For these datasets, the ground-truth feature importance is available. This allows us to compute the and Log loss scores.
Synthetic datasets
233
Time complexity
Empirical experiment measurements. In seconds.
Run set
389
Learning curve and time complexity
In this separate experiment, the accuracy and fitting time was recorded with increasing sample sizes.
Synclf hard
120
Evaluation metric correlation
Extra section. Examines the correlation between evaluation metrics.
In this chart, we examine the correlation between Log loss, and the mean validation score.
Synclf medium w/ DT
9
For the above dataset, the metrics correlate pretty well. This is not always the case, however:
Synclf very hard w/ DT
9
Peregrine HPC processing time
This set of panels contains runs from a private project, which cannot be shown in this report
Add a comment