Scientific Notation for very small numbers 1E-12

Question

I have imported some data from a csv file using the AML operator. The data has columns of small E-12 valued data.

I noticed that in the results view all very small numbers are represented as zeros. Even in the meta data view the statistics is all zero. However, when you copy and paste the entry you see that the correct E-12 number is stored there.

Does rapid miner correctly use these numbers (E-10 - E-12 range) or does it assume zero for the processing operators. I suppose I could scale up by some constant but is that necessary?

Also is there any way to show scientific notation in the results view?  ;D

Thanks,
-Gagi

land · Answer

Hi,
in addition to what haddock said: The Remove Useless operator uses the standard deviation of the attribute values to determine if it's useless. If your numbers are very small, you will have to lower the threshold accordingly.
I think it would be smarter to use some mean weighted threshold, but anyway, the remove useless operator should be avoided for attributes having different values at all if possible. The usage of a learner based attribute selection will be far preferable.

Greetings,
  Sebastian

haddock · Answer

Hi there, In Rapido reals are really reals, they are only rounded up for display, according to the 'fractiondigits.number' preference setting. As for imposing scientific notation, or others ....

dragoljub · Answer

I have also noticed that this could be problematic when using the 'Remove Useless' operator. It seems like for very small numbers the statistics are not correctly calculated since they are always interpreted as zero rather than normalized values.  ???

-Gagi