Why KNN in rapidminer is giving memory problem ?

kashif_khan
kashif_khan New Altair Community Member
edited November 5 in Community Q&A
Hi, i am working in text classification on 20NewsGroup dataset with 100 documents in each category. I classify text documents via Naive Bayes using 10-fold cross validation, It runs successfully and give me results at the end.

I tried same with KNN with 10-fold cross validation but it always ends in "Process Failure" which shows that it requires more memory than available. I increase heap space for rapidminer to 2.5G in build.xml as well as rapidminerGUI.bat but nothing improves and it always ends up in demanding more memory.

Kindly help, i am stuck out at it and tried every possible option i could think about

Platform Details:

OS: Windows 7(64 bit)
Software Version: Rapidminer 5.3 (64 bit)
Java: Java 1.7 (64 bit)
Tagged:

Answers

  • kashif_khan
    kashif_khan New Altair Community Member
    No Reply ? :(:(:(
  • Marco_Boeck
    Marco_Boeck New Altair Community Member
    Hi,

    some algorithms require more memory than others. How much memory does your system have and what is available to RapidMiner? You can check how much RapidMiner can access by selecting "View" -> "Show View" -> "System Monitor".

    Regards,
    Marco
  • kashif_khan
    kashif_khan New Altair Community Member
    System Monitor is showing

    Total: 1.2 G
    Max: 1.2G

    I have total RAM of 3G available in my system. I think Rapid miner use a format for vector which is too heavy. I wrote my example-set after calculating tf-idf in rapidminer and file size was 1.24G.

Categories