"[SOLVED] Overlapping folds in cross validation?"
Hi forum,
Today, I read many helpful posts about cross validation (x-validation). But still I have one important question: Do the folds, which are constructed, "overlap" with each other? I mean do they have any duplicated data point or they are completely separated folds with no overlap?
You know in RM we have 3 types of cross validation sampling: "linear", "shuffled" and "stratified". I think choosing linear sampling makes non-overlapping folds but the other two may construct overlapping folds. But I experienced a very astonishing result: When I used 10 folds x-val with "linear sampling" I got the accuracy of 31% but when I just choose the "stratified sampling" I got 86% accuracy!!! I am really confused with the results. Does Anyone know how should I evaluate the performance of my model?
I would also really appreciate if someone explain the issue of overlapping folds for cross validation, from academic point of view.
regards,
Today, I read many helpful posts about cross validation (x-validation). But still I have one important question: Do the folds, which are constructed, "overlap" with each other? I mean do they have any duplicated data point or they are completely separated folds with no overlap?
You know in RM we have 3 types of cross validation sampling: "linear", "shuffled" and "stratified". I think choosing linear sampling makes non-overlapping folds but the other two may construct overlapping folds. But I experienced a very astonishing result: When I used 10 folds x-val with "linear sampling" I got the accuracy of 31% but when I just choose the "stratified sampling" I got 86% accuracy!!! I am really confused with the results. Does Anyone know how should I evaluate the performance of my model?
I would also really appreciate if someone explain the issue of overlapping folds for cross validation, from academic point of view.
regards,