Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

Difference between result of Rapid miner and Excel removing duplicates function

Good day.
I am new in using RM.
I need to remove duplicates from my dataset within preprocessing step.
SO,

I have 7621 examples as original set.

I used "remove duplicates' function of excel and got 6830 rows ( examples) as a result.

Since, I` m runing the project in RM , I need to clean my data via its operator. Thus, I used "Remove Duplicates operator" , I have choosen "Project name" attribute and run process. As an outcome I got 6854 examples.
My question is why do I have difference between the resulting examples ( 6854 via RM & 6830 via Excel).
I attached my process to this message and asking support for dealing with this problem, please.

Thank you in advance.

Find more posts tagged with

AI Studio

Excel

Getting Started

Accepted answers

All comments

earmijo

Without you providing the dataset, there is no way of knowing for sure. Experiment checking/unchecking the option "Include Special Attributes".

sshabano

I attached xml file.

Telcontar120

The xml file doesn't allow us to actually see the data though, just that you retrieve it as part of your process...

Another thing to test is whether you have any leading or trailing spaces as part of the attribute you are trying to dedupe if it is polynominal. You can use the "Trim" operator in RapidMiner first to make sure it doesn't.