why the accuracy and AUC values ​​are so different?

Dhiii12
Dhiii12 New Altair Community Member
edited November 5 in Community Q&A
Please help me,

Why does the 65% accuracy value get an AUC value of 0.727, while the accuracy of 73.54% get an AUC value of 0.711. what affects both, why higher accuracy has lower AUC?





I thank you in advance for your help! 

Best regards, 

Dhiii
Tagged:

Answers

  • BalazsBarany
    BalazsBarany New Altair Community Member
    Hi!

    This often happens with an imbalanced label distribution, e. g. if one value of the label is 80 % and the other is 20 % of the data. But there can be other reasons, too.

    AUC and accuracy measure different things. 

    Accuracy is just the percentage of the correct predictions. In an extreme case, when the simplest possible model just predicts the majority class, it can be right as often as the majority class is in the data. But this is not a good model (a model you'd want to use).

    AUC measures true positive to false positive rates at each confidence level. The red line that you see in the chart goes up for correct predictions and to the right for wrong ones. The line describes a "curve" and the AUC is the area under that curve. So the best AUC (1.0) is achieved by just making correct predictions with the highest confidence. 

    So AUC also measures the confidence levels, not just the (more or less arbitrary) decision boundary applied by Apply Model. It is a more complex but more reliable measure, and it's not affected by imbalanced label distributions.

    Check the confusion matrix. Do you have a situation where one class is being overpredicted (has many false positivies)? That would be a situation where AUC is a better indicator of the model performance than Accuracy.

    Regards,
    Balázs

Categories