Test dataset (example set) not equal to sum of confusion matrix tabs.

ssv112

I have used 124 test datasets and used different ML operators but everytime my sum of test samples ( in confusion matrix) is not equal to example set (124) and comes different number with different ML technique. Can anyone explain why? 

lionelderkrikor

@ssv112,

A hypothesis : 

If you are using Auto-Model, Auto-Model first calculates the performance on  7- hold out sets of the test set. Then Auto-Model uses the operator called Performance Average (Robust)  to create the final confusion matrix.
This operator removes the performances with the highest and the lowest value for the main criterion from the input and calculates the average of the rest.
So there is a factor 5/7 (or 7/5) between the number of samples in the initial test set and the sum of test samples in the final confusion matrix.

Hope this helps.... if not please share your data and process.

Regards,

Lionel

  

lionelderkrikor

Hi @ssv112,

In order we can reproduce what you observe, can you share : 

 - your process
 - your data

Thanks,

Regards,

Lionel

lionelderkrikor

@ssv112,

A hypothesis : 

If you are using Auto-Model, Auto-Model first calculates the performance on  7- hold out sets of the test set. Then Auto-Model uses the operator called Performance Average (Robust)  to create the final confusion matrix.
This operator removes the performances with the highest and the lowest value for the main criterion from the input and calculates the average of the rest.
So there is a factor 5/7 (or 7/5) between the number of samples in the initial test set and the sum of test samples in the final confusion matrix.

Hope this helps.... if not please share your data and process.

Regards,

Lionel