SVM Weighting

lexusboy

Hello,

I would be grateful to anyone who could explain me the working of SVM Weighting, I have used RapidMiner in extensively in my thesis, and I have to explain these operators in my document. I read the description of SVM Weighting in the RapidMIner manual but I could not understand it well.

Thanks!

Regards

Ben

Hi,
a linear SVM (support vector machine), in very short, builds a linear model R^d -> {+1, -1} given by the equation
Y = sign(w*X+b)
Y = sign( sum^d_i=1(w_i*X_i) +b).
The absolute value of the ith entrance in the weight-vector w resembles the influence of the ith feature, given the all features are normalized.
The SVMWeighting operator now calculates such a linear model and returns the absolutes values of the weight vector.

You should note, that two features which are highly correlated share the importance score, giving them a lower score even if they are "important" for the class variable. For this case, you should better try Recursive Feature Elimination (RFE)

For a better description of SVM see http://research.microsoft.com/en-us/um/people/cburges/papers/svmtutorial.pdf
and for Recursive Feature Elimination using the weight vector of an SVM see
@article{Guyon/etal/2002,
author = "Isabelle Guyon and Jason Weston and Stephen Barnhill and Vladimir Vapnik",
title = "Gene Selection for Cancer Classification using Support Vector Machines",
journal = "Machine Learning",
volume = "46",
number = "1-3",
publisher = "Kluwer Academic Publishers, Boston",
pages = "389--422",
year = "2002",
}

Greetings
Ben

B_Miner

lexusboy,

A very readable book on SVM is by Dr. Lutz Hamel, published by Wiley.

HTH

lexusboy

Thank you Ben for your explanation & B_Miner for your suggestion, I will look into that book