Confidence of a predeiction

Papad
New Altair Community Member
Hi everyone,
When I see a prediction in my result like this

in line 4 I have confidence 1 for loyal , but the prediction is wrong. Does that mean that my model is not so good?
Thanks in advance.
When I see a prediction in my result like this

in line 4 I have confidence 1 for loyal , but the prediction is wrong. Does that mean that my model is not so good?
Thanks in advance.
Tagged:
0
Best Answer
-
Hello @Papad
The predicted output of a model is based on the confidence value. The model checks the confidence and assigns a label (churn or loyal) based on a threshold of 0.5. In your line 4 we can see that your model thinks that the sample belongs to loyal as the confidence is 1. We can see that your model also gave wrong predictions for line 1 and 3.
A model goodness depends on its performance metrics like AUC, Kappa etc. Its difficult to determine worth of a model based on single sample.1
Answers
-
Hello @Papad
The predicted output of a model is based on the confidence value. The model checks the confidence and assigns a label (churn or loyal) based on a threshold of 0.5. In your line 4 we can see that your model thinks that the sample belongs to loyal as the confidence is 1. We can see that your model also gave wrong predictions for line 1 and 3.
A model goodness depends on its performance metrics like AUC, Kappa etc. Its difficult to determine worth of a model based on single sample.1 -
Hi,my 2 cents: https://towardsdatascience.com/why-you-should-ignore-data-science-performance-measures-785e9b56e941BR,Martin2