Multiclass evaluation problem

sgmeire

Hi!

I´m using a cross-validation for a multiclass classification problem and I'm using a performance evaluation model for multiclass. I'd like to obtain the AUC measure but it is only possible for binary class. How can I do this? Can anybody help me?

Thank you,
Silvana.

wessel

Hey,

I assume you referring to a nominal classification problem.
As opposed to a bi-nominal classification problem.
The term multi-class is sometimes used to refer to problems where an instances can belong to multiple classes, e.g. some instances belonging to both the sports and football class.

AUC, area under ROC curve.
ROC, Receiver Operating Characteristic, true positive rate plotted against the false positive rate.
True positive rate, and true negative rate are only defined for bi-nominal classification problems.

You could transform your problem into a bi-nominal classification problem.
Lets say you have three classes A, B and C.
You can then make 3 different datasets with classes A and not A, B and not B, C and not C.
And then obtain the AUC score on each dataset.
I doubt this is a useful measure though.

Have you considered the Cohen's kappa coefficient as a measure of performance?

Or maybe area under a learning curve?