Rapidminer AZURE image runs out of memory
prithvirajbiswas
New Altair Community Member
Answers
-
Hi @prithvirajbiswas - this is MarlaBot. I found these great videos on our RapidMiner Academy that you may find helpful:
MarlaBot
0 -
Hello @prithvirajbiswas
Looks like your post is not answered yet
I do change that on windows, you can see if this is similar or not in Linux. I dont use docker but this is based on some reading, In docker, the configurations are in volume/configuration and the parameter is JOBAGENT_CONTAINER_MEMORYLIMIT. You can change the memory limit here. I also think that the memory you can allocate is based on your license, if it is unlimited then no problem. You can see your limit in server page under Administration --> Manage License
One more thing related to Automodel, Auto model will downsample data and it is absolute downsampling, you can check that in the model. If you want to run 15 million then you need to remove this sample operator inside process. I am not sure how feasible it is to run such huge data on AM.
@Marco_Boeck or @Edin_Klapic might have something here about memory change.1 -
Just a quick correction ;-)...Auto model will downsample data to 250,000 samples...AM actually does not always sample down to that number. The sampling depends on the learning algorithm and can be even less (for slower algorithms like SVM) or much more (for faster ones like Naive Bayes). Just wanted to throw this out here :-)Those sample rates are chosen to keep AM within reasonable time limits. As @Varun has mentioned, you can manually change them in the process but often that does not make a lot of sense for most data sets so I would be a bit careful that you do not blow up the whole process...Cheers,
Ingo2
