🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.
Join and Win

Rapidminer to HPC

User: "icsdm16069"
icsdm16069
New Altair Community Member
Updated by Jocelyn
Hello,

i would like to ask if i could run the rapidminer (using linux distribution) in an HPC with slurm. I am really confused because i am trying to run it and have memory problems. My rapidminer process is working well and is inside a loop. Is anyone that have the same problem?

Find more posts tagged with

AI Studio
Sort by:
1 - 3 of 31
    User: "MartinLiebig"
    MartinLiebig
    Altair Employee
    how much ram did you give it?
    BR,
    Martin
    User: "icsdm16069"
    icsdm16069
    New Altair Community Member
    OP
    my slurm program is the following:

    #!/bin/bash -l

####################################
#      slurm script template   #
#                                  #
# Submit script: sbatch filename   #
#                                  #
####################################

#SBATCH --job-name=rapidminer    # Job name
#SBATCH --output=rapidminer.%j.out # Stdout (%j expands to jobId)
#SBATCH --error=rapidminer.%j.err # Stderr (%j expands to jobId)
#SBATCH --ntasks=3     # Number of tasks(processes)
#SBATCH --nodes=1     # Number of nodes requested
#SBATCH --ntasks-per-node=3     # Tasks per node
#SBATCH --cpus-per-task=1     # Threads per task
#SBATCH --time=10:10:30   # walltime
#SBATCH --mem=64G   # memory per NODE
#SBATCH --partition=task    # Partition
#SBATCH --account=pa    # Replace with your system project

if [ x$SLURM_CPUS_PER_TASK == x ]; then
  export OMP_NUM_THREADS=1
else
  export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
fi


## LOAD MODULES ##
module purge		# clean up loaded modules 

# load necessary modules
module load gnu/4.9.2
module load intel/15.0.3
module load intelmpi/5.0.3
module load cuda/8.0.61
module load java/1.8.0
module load python/3.5.0
## RUN YOUR PROGRAM ##
srun ./rapidminer-studio/scripts/rapidminer-batch.sh -f /rapidminercodes/xml.rmp

    User: "icsdm16069"
    icsdm16069
    New Altair Community Member
    OP
    ok i fixed it. The first thing that you need to modify is the max memory that can be used from rapidminer and the second is the making an mpi program in order to split your input data and run the rapidminer in parallel.
Quick Links