"Brute force feature selection"

ammargh

Shouldn't brute force feature selection return the best performance? The performance I had using features I have selected manually was better than the performance using features returned from brute force selection component.

Is this normal?

MariusHelf

Hi,

how big is the difference between your manual performance and the performance of the features found by brute force? Are you using a Cross Validation?

Please keep in mind that by default the X-Validation always uses random splits, and thus small performance changes can be produced by randomness. To enforce the same splits in all X-Validations in all iterations, and also in your manual evaluation, you can set the local random seed of all X-Validation operators to a constant. Then only the performance of the algorithms and the features is compared, and the factor "random" is eliminated.

Best regards,
Marius