jave Heap Space

svpriyan

Hai
I have some doubts to ask from you.
When i run my below code in rapid miner i get the error heap space problem.
i have 2GB RAM only and my data set is like 2 regular attributes, 50000 example.

DO i have any other changes to be done to get the result like improve my code / do I need to upgrade my RAM .

<operator name="Root" class="Process" expanded="yes">     <operator name="CSVExampleSource" class="CSVExampleSource">         <parameter key="filename" value="C:\b9.csv"/>     </operator>     <operator name="Numerical2Polynominal" class="Numerical2Polynominal">     </operator>     <operator name="Aggregation" class="Aggregation">         <list key="aggregation_attributes">           <parameter key="item" value="count"/>         </list>         <parameter key="group_by_attributes" value="tid|item"/>     </operator>     <operator name="Example2AttributePivoting" class="Example2AttributePivoting">         <parameter key="group_attribute" value="tid"/>         <parameter key="index_attribute" value="item"/>     </operator>     <operator name="ChangeAttributeRole" class="ChangeAttributeRole">         <parameter key="name" value="tid"/>     </operator>     <operator name="UserBasedDiscretization" class="UserBasedDiscretization">         <list key="classes">           <parameter key="yes" value="1.0"/>           <parameter key="no" value="0.0"/>         </list>     </operator>     <operator name="FPGrowth" class="FPGrowth">         <parameter key="min_support" value="0.2"/>     </operator>     <operator name="AssociationRuleGenerator" class="AssociationRuleGenerator">         <parameter key="keep_frequent_item_sets" value="true"/>         <parameter key="min_confidence" value="0.4"/>     </operator>     <operator name="ResultWriter" class="ResultWriter">         <parameter key="result_file" value="C:\Documents and Settings\svpriyan\Desktop\7to13res.res"/>     </operator> </operator>

thanks
Priyan

land

Hi,
FPGrowth is a very fast, but very memory consuming algorithm for frequent item set mining. If your problem does not fit into your ram, you should increase min_support, because a great memory consumption indicates the creation of very many of item sets containing even rare and hence unimportant sets. Start with higher support and check if process then finishes successfully.

Greetings,
  Sebastian

svpriyan

Thanks,
I did it, but I could not get any FI Sets.  seems i have to try with RAM. but win 32 can go up to 3 GB RAM,  is that correct?
Greetings
Priyan

haddock

Hi Priyan,

You are right, if you are on Windows you will need either XP 64 or Vista 64, Rapido works fine on both.

Unknown

In general, have found that my maximum heap space for various versions of windows is around 1400 megs.

My understanding is that Java requires that the heap be allocated into a contiguous memory space and Windows allocates memory for DLLS somewhere in the middle of the allocated process memory space causing heap allocation requests in excess of the 1400 megs to fail.

http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx
http://forums.sun.com/thread.jspa?forumID=37&;threadID=201718
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4358809

It should be possible to allocate larger heaps with some special setup.  I'd be interested in hearing if anyone has accomplished this, and if so, how.

- Pat

keith

This has been discussed on the RM forums before, most recently at:  http://rapid-i.com/rapidforum/index.php/topic,646.msg2422.html

The short answer is that while there are tricks like the /3GB switch that may eek out a little more heap space, it's fundamentally a Windows 32-bit + Java limitation, and the best way to solve it is to upgrade to a 64-bit platform.  I made the switch to Windows XP 64-bit, and have had no further problems with Java heap space.