Removing Non-Duplicates using RapidMiner

novice_miner
novice_miner New Altair Community Member
edited November 5 in Community Q&A

I have a dataset of over 42000 records that has several duplicate and unique values. However, I would like to clean it up and remove ONLY non-duplicate values and leave duplicate records. I know the “remove duplicates” operator removes duplicates but in my case, I want to do the opposite. It’s quite easy to accomplish this task on excel but as you know, excel can’t seamlessly handle the size of my dataset. Is there anyway I can perform this task on RapidMiner?

Best Answer

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    There is already an output port on the Remove Duplicates operator that provides the duplicate examples, so you can simply use that as the input to your downstream process instead of the exampleset with the duplicates removed.

Answers

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    There is already an output port on the Remove Duplicates operator that provides the duplicate examples, so you can simply use that as the input to your downstream process instead of the exampleset with the duplicates removed.
  • novice_miner
    novice_miner New Altair Community Member
    That worked. Thank you