bias variance trade off

Thiru

hi all,

My query is with respect to the classification models.

Is there any way to  identify the soft spot where the bias and variance meet. I do understand there is no metric for bias variance trade off.  
 However Im interested in knowing how to approach the issue.  

a.  How to analyse and  is related to confusion matrix / Precision/recall. 
b.  Any specific or set of operators in rapid miner to analyse.
c.  Do you suggest any typical process to be followed to arrive the balance 
authentically and quickly.
d.  any resource/video on the topic.


thanks in advance for your reply.

regds
thiru

PaulMSimpson

Hi thiru, I think this is a great resource on the bias-variance tradeoff. http://scott.fortmann-roe.com/docs/BiasVariance.html Also, considering that High Variance means you have low Precision, and High Bias means you have low Recall, it may help to use the confusion matrix for binary classification. Note in the images below (one with a high Recall, the next with a high Precision), which have Predicted values as rows and Actual values as columns, that Precision is looking at values going across, and Recall looks at values going down (with TP as the numerator, and TP + FP or TP + FN as the denominator). I think that if your model's results give you a confusion matrix where the numbers let you get a good balance between Precision and Recall, you have a good model that neither overfits nor underfits.
Regarding the confusion matrix, you can alter it simply by adjusting your threshold (the cutoff point at which you decide whether to call a predicted probability a "Yes" or a "No"). My R Shiny app, Predictive Thresholds, Performance Measures and the Confusion Matrix, as described in this article, allows you to examine the effect on the confusion matrix of adjusting your threshold up or down from the standard 50% threshold.