Setting Ranges in a Declare Missing Value Operator
I asked about this on the Support forum, but it seems what I might actually be after is a new feature.
I have an attribute in my data set containing millions of examples (observations). This attribute, Crime_Instances, has no missing values, but has about 25 distinct outlier values comprising less than 1% of the total examples. I would like to use a Declare Missing Values operator to submit an expression such as Crime_Instances > 8, so that all examples that have a value of more than 8 in the Crime_Instances attribute would be set to missing. It wouldn't matter if one example had 15 in this attribute, while another had 25 and another had 90, they would all be set to missing using a single expression in a single operator. This feature would be very useful to me.
As a side note, Sebastian suggested a method for accomplishing this and it did work, though it required six extra operators to achieve my desired result, so it's not as if I can't do it, I just can't do it very easily right now.
Thanks!
Matt
I have an attribute in my data set containing millions of examples (observations). This attribute, Crime_Instances, has no missing values, but has about 25 distinct outlier values comprising less than 1% of the total examples. I would like to use a Declare Missing Values operator to submit an expression such as Crime_Instances > 8, so that all examples that have a value of more than 8 in the Crime_Instances attribute would be set to missing. It wouldn't matter if one example had 15 in this attribute, while another had 25 and another had 90, they would all be set to missing using a single expression in a single operator. This feature would be very useful to me.
As a side note, Sebastian suggested a method for accomplishing this and it did work, though it required six extra operators to achieve my desired result, so it's not as if I can't do it, I just can't do it very easily right now.
Thanks!
Matt
Find more posts tagged with
Sort by:
1 - 3 of
31
This looks like it will work for what I need. I'll look forward to this feature being added!
The Declare Missing Operator has new mode "expression", this example works: if(Crime_Instances > 8, true, false)
This not function: Crime_Instances > 8
Excuseme for very bad english.
This not function: Crime_Instances > 8
Excuseme for very bad english.
the next RapidMiner version will have a constant MISSING which you can use in, e.g., "Generate Attributes". Then you just generate a new attribute
if(oldValue > 8, MISSING, oldValue)
Best,
Simon