memory issues

mjavier99

Hi, I am running a Clustering Model W-Clope with a table with about 1 million registers. RapiMiner never ends and a have a memory leak. I have configured MAX_JAVA_MEMORY to 2048 because I have 3 GB of memory to use. This RapidMiner build to be used with large table datasets ?

land

Hi,
weka learners generally needs to copy the data once if used inside RapidMiner. Hence you will have a waste of memory of factor two.

Then you will have to think that most Datamining algorithms need a runtime equivalent to the number of examples to power of three. So 1 million examples is simply much to much...Even if each computational step would only need 1 clock of your CPU, a 2 GHz CPU would need 2 years to solve such a problem...

Greetings,
  Sebastian