Regarding Kappa Value in Cross Validation

varunm1

Hello,

Kappa values can be positive or negative depending on the type (level of agreement +ve, level of disagreement -ve). But in cross-validation how the final value of kappa is calculated? Are the negative values converted to positive and then the average kappa of all the folds are taken into consideration or are they are averaged directly?

Thanks,
Varun

MartinLiebig

Hi @varunm1 ,

this is the code we use:

double pa = accuracy;
double pe = 0.0d;
for (int i = 0; i < counter.length; i++) {
   double row = 0.0d;
   double column = 0.0d;
   for (int j = 0; j < counter[i].length; j++) {
      row += counter[i][j];
      column += counter[j][i];
   }
// pe += ((row * column) / Math.pow(total, counter.length));
   pe += row * column / (total * total);
}
return (pa - pe) / (1.0d - pe);

Where counter has the confusion matrix values.

Does this make sense to you?

BR,

Martin