Process help: Extract ID wise prediction performance after Cross Validation
Hello,
I currently have multiple observation predictions for each subject from a cross-validation method (Binary Classification). I am trying to extract subject wise prediction performances from the predictions made by CV. To do this, I am counting the number of prediction labels per subject based on the ID and then create attributes that have a number of predictions for label 1 and the number of predictions for label 2. Then the prediction per subject is assigned based on a threshold of 0.5, for example, if more than 50 percent of subject 1 samples are labeled as label 1, then that subject will be assigned label 1. Similarly for all the subjects based on the set threshold. Once I get the subject wise predictions, I try to calculate the performance using the performance operator.
Issue: Everything works well when I have predictions for both labels, but when I have only a single label predicted for all subjects (less accurate algorithm) based on a threshold, my process fails as my process design to calculate performance involved both classes. I am missing logic to bypass this issue and create an attribute with zero values for the other label for all subjects.
I attached repository files in this thread, you can run the process to check this error. Any help would be much appreciated.
@mschmitz @lionelderkrikor @yyhuang @kayman
I currently have multiple observation predictions for each subject from a cross-validation method (Binary Classification). I am trying to extract subject wise prediction performances from the predictions made by CV. To do this, I am counting the number of prediction labels per subject based on the ID and then create attributes that have a number of predictions for label 1 and the number of predictions for label 2. Then the prediction per subject is assigned based on a threshold of 0.5, for example, if more than 50 percent of subject 1 samples are labeled as label 1, then that subject will be assigned label 1. Similarly for all the subjects based on the set threshold. Once I get the subject wise predictions, I try to calculate the performance using the performance operator.
Issue: Everything works well when I have predictions for both labels, but when I have only a single label predicted for all subjects (less accurate algorithm) based on a threshold, my process fails as my process design to calculate performance involved both classes. I am missing logic to bypass this issue and create an attribute with zero values for the other label for all subjects.
I attached repository files in this thread, you can run the process to check this error. Any help would be much appreciated.
@mschmitz @lionelderkrikor @yyhuang @kayman