Learning Curves: Cost Functions

Hyram

Hi

I am trying to assess bias and variance for my classification problem. As such, I am trying to plot the cost functions J(Theta) for testing and training vs. number of examples to determine if I have high variance or high bias. How can I do this in RapidMiner? Alternatively, are there operators that let me visualise bias vs. variance?

Thanks,
Hyram

Telcontar120

There isn't anything automatic to produce this.  You can do it somewhat manually by using the Sample operator and using different values/percentages of your total dataset and then outputting the model performance results of each of these different scenarios.   You could automate this further by using a macro for sample value and putting it into a Loop.  But there isn't any built-in operator that would easily output this all for you. 

Telcontar120

There isn't anything automatic to produce this.  You can do it somewhat manually by using the Sample operator and using different values/percentages of your total dataset and then outputting the model performance results of each of these different scenarios.   You could automate this further by using a macro for sample value and putting it into a Loop.  But there isn't any built-in operator that would easily output this all for you. 

Hyram

Thanks @Telcontar120!