Stratification: How to get the same number of examples for each class?
I have a data set of 2 labels, label A(6000 items), label B(500items).
I want to run a 10-fold cross validation but with sampling. For example: the 1st fold has 600 of label A and 50 of label B. we want to sample 50 label A out and create a new 1st fold with 50 label A and 50 label B. Same process for rest of 8 folds and we use 9 folds together to training and 1 fold of non-sampled data to testing. The process loop through for the entire data set and collect the performance.
So far I am able to do the above process one fold by one fold which is time consuming. I was hoping to set up a process to do them automatically.
Thanks in advance for your support
John Quest
I want to run a 10-fold cross validation but with sampling. For example: the 1st fold has 600 of label A and 50 of label B. we want to sample 50 label A out and create a new 1st fold with 50 label A and 50 label B. Same process for rest of 8 folds and we use 9 folds together to training and 1 fold of non-sampled data to testing. The process loop through for the entire data set and collect the performance.
So far I am able to do the above process one fold by one fold which is time consuming. I was hoping to set up a process to do them automatically.
Thanks in advance for your support
John Quest