logging Confusion matrix

Unknown

Hi there

is there a way to log (e.g. during optimization...)whole confusion matrix or its specific parts?

thx

MariusHelf

Hi,

if you have a X-Validation, it provides the first three performance measure of its inner Performance operator as log values (performance1, 2 and 3).
If you use the Performance (Binominal Classification) operator, you can select false positive, false negative etc. and thus get the values of the confusion matrix.
Currently, there is this limitation that you can only log three performance values from the X-Validation. The next release of RapidMiner, however, will feature a Performance to Data operator, which will convert the complete performance vector to an example set, such that you can get all the values of the Performance operator. The dataset will have the format Performance Measure - Value, where Performance Measure is one of the performance measures selected in the Performance operator.

Best, Marius

Unknown

Marius wrote:
Hi, if you have a X-Validation, it provides the first three performance measure of its inner Performance operator as log values (performance1, 2 and 3).

Yes thats nice, but there is no option to get directly  TP and FP for a certain class or percentages. One has to filter examples, generate attributes ,extract macros....that's how I calculate what I need... and it works fine ^_^

If you use the Performance (Binominal Classification) operator, you can select false positive, false negative etc. and thus get the values of the confusion matrix.

I have not tried that. Is this available only for the binary case then?

Currently, there is this limitation that you can only log three performance values from the X-Validation. The next release of RapidMiner, however, will feature a Performance to Data operator, which will convert the complete performance vector to an example set, such that you can get all the values of the Performance operator. The dataset will have the format Performance Measure - Value, where Performance Measure is one of the performance measures selected in the Performance operator.

Sounds good. Hope you squeeze in the option for an arbitrary field of the confusion m.

Best, Marius

cheers
f

MariusHelf

fritmore wrote:
Yes thats nice, but there is no option to get directly  TP and FP for a certain class or percentages. One has to filter examples, generate attributes ,extract macros....that's how I calculate what I need... and it works fine ^_^ (...) I have not tried that. Is this available only for the binary case then?

Yes, you can! While Performance (Binominal Classification) is obviously limited to the two-classes-case, it offers to output TP, FP etc. For the general classification case Performance (Classification) still offers some more control than the generic Performance operator. There are other specialized Performance operators, just have a look at the Performance operator group.

Sounds good. Hope you squeeze in the option for an arbitrary field of the confusion m. cheers f

The new operator will only output the values of the performance measures, not the confusion matrix. But as stated before, with the specialized performance operator you can calculate TP, FP etc as a measure.

Best, Marius