Different performance between grid search and real test

YADAO

Hi guys,
I used optimize parameters to find the best parameters' values with 10 fold cross-validation, and it showed out that the best kappa was 0.397. But when I applied these parameter values to 10 fold cross-validation myself, the kappa was only 0.337.
What's wrong?
Thank you.

MartinLiebig

Hi @YADAO ,

can it just be sampling differences? What happens if you change the seed of your x-val?

Best,

Martin

swordf

Is there still someone on this issue now?

I have the same problem, different results are presented using the suggested parameter by grid optimizer,

I don't understand why, even if it's not the same it should be close at least

MartinLiebig

Hi @swordf ,

close in number of standard_deviations, yes.

Best,

Martin