Compare ROC and Normalization use - getting different results

inbariku

Hi,

I'd like to better understand the use of Compare ROC operator:

I'm using it to compare 3 classifiers: SVM, KNN and Logistic regression, on the Titanic dataset.

In the 1st scenario, I'm using normalize operator before the compare ROC input port.

In the 2nd scenario, I'm using the normalize operator inside the compare ROC operator. (only on the KNN and SVM, as the Logit don't require normalization. however, adding it to the Logit as well, changed it's ROC curve, and it shouldn't....)

In both cases, I'm setting the number of folds to be (-1), so I can look at it as a train/test split and not cross validation.

I'm getting different results for these 2 cases. why is that?

Also, from what I know, the best practice is to normalize the training data and testing data separately. how can I achieve that using the compare ROC operator?

attached are .rmp files for both scenarios described above.

Thanks!