"Outof Memory Error loading big SPSS file"

Braulio

Hi there,

I am having a problem while loading a big SPSS file in RapidMiner. (pasted below)
The file is about 420 MB (over 1 million lines and 70 columns)

My question: what memory does the error refer to? RAM memory?

Thanks a lot

Braulio

Here is the error:

P May 26, 2008 2:24:18 PM: [NOTE] Process starts
P May 26, 2008 2:24:18 PM: Process:
  Root[0] (Process)
  +- SPSSExampleSource[0] (SPSSExampleSource)
P May 26, 2008 2:29:10 PM: [Fatal] OutOfMemoryError occured in 1st application of SPSSExampleSource (SPSSExampleSource)
P May 26, 2008 2:29:10 PM: [Fatal] Process failed: Java heap space
          Root[1] (Process)
here ==> +- SPSSExampleSource[1] (SPSSExampleSource)

TobiasMalbrecht

Hello Braulio,

this error occurs when the Java Virtual Machine (JVM) tries to occupy more memory than it has been made available to the JVM. If you use RapidMiner on a Windows system you may set the memory that is made available to the JVM by editing the file RapidMinerGUI.bat in the scripts subdirectory in the RapidMiner program directory.

The line you have to edit appears like:

if "%MAX_JAVA_MEMORY%"=="" set MAX_JAVA_MEMORY=1024

There you can specify how much memory the JVM may use (in megabyte). If you then start RapidMiner via the RapidMinerGUI.bat the JVM which executes RapidMiner should use the amount of memory you have specified. For a 420mb SPSS file 1024mb of memory however might eventually not be enough since SPSS files may be compressed and therefore the amount of memory used might be actually quite more than the size of the file.

Hope to have helped,
Tobias