Logistic regression with and without regularisation
hi all,
I have a classification case, wherein I use Logistic regression.
At first instant - I get accuracy of 68.52% with 100% recall.
subsequently with regularisation - I get accuracy of 98.77% with 100 % recall.
1. can you elaborate , how regularisation leads to this much jump in better accuracy. can you explain the basics behind this rapidminer option.
2. I couldn't see lamda value in result. Is there any way to get it displayed.
3. Both the cases , I get 100% recall. ( zero false negative, which is desirable in my case.).
But Im not sure, whether it is a good model.
I achieved above after normalisation and cross validation. Im enclosing
both results. thanks
regds
thiru
I have a classification case, wherein I use Logistic regression.
At first instant - I get accuracy of 68.52% with 100% recall.
subsequently with regularisation - I get accuracy of 98.77% with 100 % recall.
1. can you elaborate , how regularisation leads to this much jump in better accuracy. can you explain the basics behind this rapidminer option.
2. I couldn't see lamda value in result. Is there any way to get it displayed.
3. Both the cases , I get 100% recall. ( zero false negative, which is desirable in my case.).
But Im not sure, whether it is a good model.
I achieved above after normalisation and cross validation. Im enclosing
both results. thanks
regds
thiru
Find more posts tagged with
Sort by:
1 - 4 of
41
Sort by:
1 - 2 of
21
Hello @Thiru
Regularization is used for better generalization. It does not always improve results as some times the increased bias dominates the reduced variance. In your case, the reduction in variance might have dominated the bias increase (due to regularization) and gave a better result. So, in simple words, your model might have overfitted without regularization and gave less performance on test data.
Actually the 100% recall you were observing is only for 1 class (positive class), you need to check recall for both classes. If a model is blindly classifying all the observations as a positive class, then also recall will be 100%. Take a trade-off between all performance metrics (AUC, accuracy etc.). Please let us know if you need anything else
Just my 2c
Regularization is used for better generalization. It does not always improve results as some times the increased bias dominates the reduced variance. In your case, the reduction in variance might have dominated the bias increase (due to regularization) and gave a better result. So, in simple words, your model might have overfitted without regularization and gave less performance on test data.
I couldn't see lamda value in result. Is there any way to get it displayed.
Did you check the "Description" tab of the logistic regression model? Lambda value is there. Is this what you were asking?Both the cases , I get 100% recall. ( zero false negative, which is desirable in my case.).
But Im not sure, whether it is a good model.
Actually the 100% recall you were observing is only for 1 class (positive class), you need to check recall for both classes. If a model is blindly classifying all the observations as a positive class, then also recall will be 100%. Take a trade-off between all performance metrics (AUC, accuracy etc.). Please let us know if you need anything else
Just my 2c
Regularization is used for better generalization. It does not always improve results as some times the increased bias dominates the reduced variance. In your case, the reduction in variance might have dominated the bias increase (due to regularization) and gave a better result. So, in simple words, your model might have overfitted without regularization and gave less performance on test data.
Actually the 100% recall you were observing is only for 1 class (positive class), you need to check recall for both classes. If a model is blindly classifying all the observations as a positive class, then also recall will be 100%. Take a trade-off between all performance metrics (AUC, accuracy etc.). Please let us know if you need anything else
Just my 2c