🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.
Join and Win

Process help: Extract ID wise prediction performance after Cross Validation

User: "varunm1"
New Altair Community Member
Updated by Jocelyn
Hello,

I currently have multiple observation predictions for each subject from a cross-validation method (Binary Classification). I am trying to extract subject wise prediction performances from the predictions made by CV. To do this, I am counting the number of prediction labels per subject based on the ID and then create attributes that have a number of predictions for label 1 and the number of predictions for label 2. Then the prediction per subject is assigned based on a threshold of 0.5, for example, if more than 50 percent of subject 1 samples are labeled as label 1, then that subject will be assigned label 1. Similarly for all the subjects based on the set threshold. Once I get the subject wise predictions, I try to calculate the performance using the performance operator.

Issue: Everything works well when I have predictions for both labels, but when I have only a single label predicted for all subjects (less accurate algorithm) based on a threshold, my process fails as my process design to calculate performance involved both classes. I am missing logic to bypass this issue and create an attribute with zero values for the other label for all subjects.

I attached repository files in this thread, you can run the process to check this error. Any help would be much appreciated.

@mschmitz @lionelderkrikor @yyhuang @kayman

Find more posts tagged with

Sort by:
1 - 1 of 11
    User: "lionelderkrikor"
    New Altair Community Member
    Accepted Answer
    Varun,

    Bug fixed !!!

    An error was raising because in your Generates attributes you try to calculate the Final Prediction according to other attributes
    but in the cases of the 2 datasets mentionned above , these attributes have missing values, so the calculus is impossible and RapidMiner is raising an error.
    So I added a Replace Missing Values operator (with replacement value = 0) before the Generate Attributes operator.

    Here the working process in attached file.

    Telle me if this process is answering to your requirements...

    Regards,

    Lionel