Filtering ALL Examples (no. of missing data in the attribute = total no. of examples)

1705410G

I would like to filter out all missing data. Where the number of missing data in the attribute is equal to the total number of examples. 


I know Select Attribute can do the trick. However, as I have roughly 20-30 attributes where all examples missing, so I would not like to pick it out individually. 


I've tried using Filter Examples > no_missing_attributes, but I have no idea why I ended up with no examples at all from 2041 examples.

Thank you!

regards,
AY

lionelderkrikor

Hi again @1705410G,

A solution is to use Remove Useless Attributes operator...

Hope it helps,

Regards,

Lionel

lionelderkrikor

Hi @1705410G,

Could you share your dataset and eventually your process ?

Regards,

Lionel

1705410G

NIL - Sorry, had to remove due to privacy. 

lionelderkrikor

Hi again @1705410G,

A solution is to use Remove Useless Attributes operator...

Hope it helps,

Regards,

Lionel

1705410G

lionelderkrikor 

I'm not sure if this is the correct way to share a process. I achieve 0 examples after running this process when I only want to remove those attributes with ALL missing examples.


Regards,
AY

lionelderkrikor

Hi again @1705410G,

A solution is to use Remove Useless Attributes operator...

Hope it helps,

Regards,

Lionel

1705410G

@ lionelderkrikor 

Thank you!!

Regards,
AY

tftemme

Hi @1705410G,

Another solution is the operator Filter Attributes with Missing Values from the Operator Toolbox extension (which exactly does what you want and has further configurations to remove Attributes with Missing Values). The reason that you end up with 0 Examples when using Filter Examples is that the operator removes examples for which the attribute selected is missing. So as your attribute you actually want to remove has only missing values, all Examples are removed.

Best regards,
Fabian

1705410G

@ lionelderkrikor ,

What parameters should I set in Remove Useless Attributes then? Or should I leave it ? I read it briefly but i'm unsure how it works.

Regards,
AY

1705410G

@ lionelderkrikor ,

What parameters should I set in Remove Useless Attributes then? Or should I leave it ? I read it briefly but I don't understand how it works.

Regards, 
AY

1705410G

@ lionelderkrikor ,

What parameters should I set it too? I read it up briefly but am unsure how it works.

Regards,
AY

1705410G

@ lionelderkrikor ,

What parameters should I set it too? I read it up briefly but am unsure how it works.

Regards,
AY

Telcontar120

You can leave the default settings and it should eliminate any constant attributes.  Try running that and see if it gives you the results you are expecting.  If you then want to set a lower threshold then you can modify the min deviation for numericals or nominals.