[SOLVED] Accuracy performance

StaryVena

Hello,
how can I create accuracy performance, if I have TP, TN, FN, FP values? I created precision performance using BinaryClassificationPerformance, but there isn't accuracy measure. There is also MultiClassificationPerformance class, but it can be created only from ExampleSet (I think). I would like to create accuracy performance from image, so I don't have ExampleSet.
Can somebody help me?

Best,
Václav

MariusHelf

Hi Václav,

the accuracy is the probability that new examples are classified correctly, so it's #correct_predictions / #total_prediction = (TP + TN) / (TP + TN + FN + FP)

Best regards,
Marius

StaryVena

Hi Marius,
thank you for answer. I know the formula, but I don't know, how to implement it in RapidMiner so that can be used for optimization.
Example:
I have two binary images - one is reference and one is generated in RM. I need reference image to  be same as reference. I have operator which have this two images on the input ports and I need accuracy performance on the output port. So I could use the Optimize Parameters operator to optimize parameters for image generation. How should the code for accuracy looks like if I have TP, TN, FN, FP values?
I hope it's now clearer.

Best,
Václav

MariusHelf

Hi Václav,

unfortunately you are right, to calculate the accuracy you need MultiClassificationPerformance, and that can in fact only be used with examples. So you would have to create an ExampleSet from your confusion matrix to resemble an example set which would create that matrix.
Basically you need only 4 examples, one simulating each of  TP, TN, FN, FP with weights corresponding to the value in the confusion matrix.
I.e. for TP = 10, FP = 20, FN = 15, TN = 5 you could create 4 examples:
label, weight, prediction(label)
true, 10, true
false, 20, true
true, 15, false
false, 5, false

Don't forget to assign the weight role to the weight attribute.

Best!
Marius

StaryVena

Hi Marius,
thank you 

Best,
Václav