"SVM vs Logistic Regression"

goku

I'd like some help.
I'm performing document classifiaction using SVM and Simple Logistic and I want to choose which one is better for my case.
From what I see, SVM model is build faster than simple logistic, but simple logistic seems to have better precision and recall than SVM.
So, I have some silly questions that I want to ask, because I need to explain why simple  logistic can be chosen.

SVM avoids overfitting using hyperplanes but I don't know if simple logistic does something to avoid overfitting.
And my second question is if simple logistic is linear logistic regression model or gets a linear model and fits it in a logistic regression model.

Any advice would be helpful because simple logistic is getting me confused  ???

kasper2304

Hi goku.

To decide what which method are the best one need to know a bit more about your dataset. So if you can please explain a bit more. I dont really know what you mean by "Linear" logistic regression. The name "log"istic regression implies that the regression is based on a line that comes from the logarithm rather than being linear. The only three regressions i ever heard about is "simple" linear regression = 1 DV vs 1 IV,  "multiple" regression = 1 DV vs. + 2 IV's and "logistic" regression = 1 categorical DV vs. + 1 IV.

I eralise that my answer comes late but if you still need help i would provide you with some hints.

Kasper

MariusHelf

Hi,

if you cross validated your results, then in your case probably the logistic regression works better than the svm... \

Btw, in RapidMiner the logistic regression internally uses an SVM with a modified loss function. There is a paper which proves that both methods are equivalent. If you are interested, I can search for the title and the author of the paper.

Best regards,
Marius

earmijo

Marius:

Could you post the reference please?

I have never quite understood what RM fits when one uses the Logistic Regression Operator.

I'm a statistician and for me Logistic Regression is what the Weka operator W-Logistic fits.

Thanks in advance,

\E.

MariusHelf

Hi E.,

as the documentation of the Logistic Regression states, it is based on myKLR and the respective paper by Keerthi. Here you go:
http://www-ai.cs.uni-dortmund.de/SOFTWARE/MYKLR/index.html
http://link.springer.com/content/pdf/10.1007%2Fs10994-005-0768-5

I have to admit that I have never read the paper, so I can't give you any details about its contents

Best regards,
Marius