"weighted nearest neighbor crossvalidation"

Question

Hi, I cannot figure out how to integrate learning feature weights in a nearest
neighbor algorithm using 10 fold cross-validation. 
Nearest Neighbor and cross validation alone is no problem. But the 
usage of weights complicate this a lot. The weights should be learned on the 
training data and using the cross-validation operator applied on the evaluation data.
Is this possible to do this with the GUI or do I have to write the cross validation
myself without employing the cross validation operator?

Thank you for any help.

IngoRM · Answer

Hi Ulli, no coding is necessary for this (actually, problems like these were the reason for the modular operator concept or RapidMiner). This can actually be done with nested cross validations, i.e. an outer cross valivation where the learner is embedded into a feature weighting scheme like EvolutionaryWeighting containing an inner cross validation for optimizing the weights. However, it is even more comfortable to use the operator "WrapperXValidation" as outer cross validation for this task. From the operator info dialog (F1) of this operator: This operator evaluates the performance of feature weighting and selection algorithms. The first inner operator is the algorithm to be evaluated itself. It must return an attribute weights vector which is applied on the test data. This fold is used to create a new model using the second inner operator and retrieve a performance vector using the third inner operator. This performance vector serves as a performance indicator for the actual algorithm. This implementation of a MethodValidationChain works similar to the XValidation. And here are the inner conditions (also from the operator info dialog:* Operator 1 (Wrapper) must be able to handle [ExampleSet] and must deliver [AttributeWeights]. * Operator 2 (Training) must be able to handle [ExampleSet] and must deliver [Model]. * Operator 3 (Testing) must be able to handle [ExampleSet, Model] and must deliver [PerformanceVector]. So this is how could setup a process for Nearest Neighbors together with evolutionary attribute weighting: The process will run several minutes. After the process has finished, the performance is delivered together with an averaged weight vector from all runs. This vector for example could be saved and applied on new data sets for application. In the example above, the found weights should be something like att1 0.9181578856281039 att3 0.8079093341177875 att2 0.5669022824248217 random1 0.4395652799419607 random2 0.25727249709958755 random 0.047672333763268744 Cheers, Ingo