Specificity, sensitivity and AUC measures via RM v9.1
CindyDK
New Altair Community Member
Best Answers
-
ROC / AUC and its related measures are designed for binominal label classification problems only.
The best way to approach if you have multiple classes is to encode with a one-vs-all-others approach. So if you have possible label values {1,2,3} then encode that as 1 vs not 1, and generate the ROC. You can repeat for the other classes as needed, store the performances, and average them as well if you want.2 -
Sensitivity is the true positive rate, and specificity is the true negative rate. Both are also available in the Performance (Binominal classification) operator in RapidMiner so you should be able to get them in the same way you get the AUC above, by using the 1 vs all others comparison method for a multi-class problem.
Here's a basic description of the concepts in case you need a refresher: https://en.wikipedia.org/wiki/Sensitivity_and_specificity5
Answers
-
ROC / AUC and its related measures are designed for binominal label classification problems only.
The best way to approach if you have multiple classes is to encode with a one-vs-all-others approach. So if you have possible label values {1,2,3} then encode that as 1 vs not 1, and generate the ROC. You can repeat for the other classes as needed, store the performances, and average them as well if you want.2 -
Thanks for the sharing. It seems to utilize the loop to cover the number of the classes (Re: 1 vs all others). Any other thoughts about "specificity, sensitivity" measures?0
-
Sensitivity is the true positive rate, and specificity is the true negative rate. Both are also available in the Performance (Binominal classification) operator in RapidMiner so you should be able to get them in the same way you get the AUC above, by using the 1 vs all others comparison method for a multi-class problem.
Here's a basic description of the concepts in case you need a refresher: https://en.wikipedia.org/wiki/Sensitivity_and_specificity5