How does confidence works in Rapidminer

tahsin

Hello,

I am doing modeling from rapidminer and everything works good but I am not understanding the confidence and prediction that It generates. I have YES and NO as outcome variable. This is how it looks like,



How does the confidence relates to the prediction? As you can see even with higher percentage of NO confidence it is predicting YES. After 0.42 it starting to predict NO. Isn't it like, if the confidence(NO) is above 0.5 then it should predict NO otherwise YES. What is the threshold value for the confidence to predict YES or NO? 

Any help is appreciated. Thanks

BalazsBarany

Hi!

Confidence is calculated by the modeling algorithm, depending on how the algorithm works. E. g. in Naive Bayes it's a set likelihoods scaled to 1, in a decision tree the ratio in the final node, etc. 

By default the decision threshold is 0.5 but it can be changed with threshold-related operators. 

It is your process that changes the default. Can you set a breakpoint after the Apply Model to see its output? Are you seeing this effect in the resulting table? What operators are after Apply Model in the process?

Regards,
Balázs

BalazsBarany

Hi!

Confidence is calculated by the modeling algorithm, depending on how the algorithm works. E. g. in Naive Bayes it's a set likelihoods scaled to 1, in a decision tree the ratio in the final node, etc. 

By default the decision threshold is 0.5 but it can be changed with threshold-related operators. 

It is your process that changes the default. Can you set a breakpoint after the Apply Model to see its output? Are you seeing this effect in the resulting table? What operators are after Apply Model in the process?

Regards,
Balázs

tahsin

@BalazsBarany
Thank you for your response. I was able to solve the problem with an Apply Threshold operator. It takes care of the confidence issue by assigning the threshold score for YES and NO class.