Autoscaling on Amazon with RapidMiner Server

JEdward

Hi guys,

 

I'm wondering if anyone has a handy guide already for autoscaling RapidMiner Server (both scale up & scale out) using AWS? 

 

Scaling out using a JBOSS cluster (deployment mode) is relatively straight forward & lots of guides already exist for it (although none RM specific, might write one and share if none exists)....  but I'd like to know if it's possible to scale up & down JBOSS without needing to restart the Instance or RM Server. 

Alternatively, what are the best practice approaches for identifying a potential high memory process to trigger creation of a high memory instance & run the desired process just on that instance? 

 

 

zprekopcsak

Hi @JEdward,

Even though JBoss has some scale-out support, RapidMiner Server does not support that as of today. We do support High-Availability with cold-swap failover as you can see in this guide: http://docs.rapidminer.com/server/advanced-topics/high-availability/

In theory, you could run two Servers in HA, a small primary server and spin up a bigger secondary server as necessary. Then you can kill the primary and do the failover to the bigger secondary instance. You essentially scale up then, but keep in mind that you will lose running and waiting processes and web UI sessions. See the HA guide for more details.

This is not a promise, but we are considering architectural changes in Server for later this year that would also add support for a scale-out solution.

 

Best,

zprekopcsak

Hi @JEdward,

Even though JBoss has some scale-out support, RapidMiner Server does not support that as of today. We do support High-Availability with cold-swap failover as you can see in this guide: http://docs.rapidminer.com/server/advanced-topics/high-availability/

In theory, you could run two Servers in HA, a small primary server and spin up a bigger secondary server as necessary. Then you can kill the primary and do the failover to the bigger secondary instance. You essentially scale up then, but keep in mind that you will lose running and waiting processes and web UI sessions. See the HA guide for more details.

This is not a promise, but we are considering architectural changes in Server for later this year that would also add support for a scale-out solution.

 

Best,

Nikouy

Hi @zprekopcsak

Have there been any architectural changes that enable autoscalign functions?

Thanks

MartinLiebig

Hi @Nikouy,

there have been a lot of change to this. The new JobAgent and HA infrastructure is vastly different.

Is this a commercial inquiry? If yes, we are of course happy to explain it to you in a call. Just connect with me via PM.

Best,
Martin