"How to evaluate feature weighting ?"
Hi RapidMiner team,
Its Phong from UniGE (e-LICO partner).
I am trying to model a feature weighting (say ReliefF) where I want to do a dimensionality reduction (like top-k), then learn a classification model, and validate it on a test set. All of these should be done in a 10CV.
So, I have tried the standard XValidation operator, modeling in the training phase, the feature weighting + dim. reduction and the classification model, then pass everything to the testing phase, to apply the model and validate it.
However, I encountered a strange thing; after the first fold (ie after the first dim. reduction), the second fold present me a train set, WHICH IS ALREADY REDUCED !! And it continues with the rest of folds... It seems to be a data memory error, or am I wrong ?
Then, I have tried the Wrapper-XValidation operator, which seems to be here especially for feature weighting / selection, but by default, after the attribute weighting phase, when the operator builds the learner, it seems that the training set is reduced automatically by removing features that have zero weight... So how can I specify that I want to apply another rule like top-k ?
Hope that my questions are clear enough..
Thanks for your help.
Cheers
Phong
