Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

noise detection in RM?

Hi Forum,

I have a question about noise detection in RM. I dont want to use complex methods like wavelet filter or something like that. I just want to omit examples which have a low values in most of their attributes. Should I extend a new operator or the existing operators can do this? By the way, how does RM do in noise detection?

I should mention that I have seen the "Filter Example" but I couldn't find how I should use it to fulfill my request.

thanks in advance.

Find more posts tagged with

AI Studio

Accepted answers

All comments

MariusHelf

Maybe you can try some kind of outlier detection?

siamak_want

Hi again Marius,

Maybe you can try some kind of outlier detection?

I am not sure but I think outliers are something different from noise. Also there is a problem: Detecting outliers from high dimensional data (text data in my case) is very time consuming due to complex algorithms of this area (LOF,COF, ...). But I just want to remove examples which satisfy a simple condition (most their features have a low value) with scanning all the data for one time. I think I need to extend a new operator to remove these examples. Am I right Marius?

Again thanks.

MariusHelf

siamak_want wrote:
But I just want to remove examples which satisfy a simple condition (most their features have a low value)

If you can write the condition yourself, you could use Generate Attributes and Filter Attributes as described here: http://rapid-i.com/rapidforum/index.php/topic,5073.0.html

Best, Marius

siamak_want

Hi Marius,

I got the Idea, Thanks in advance:)