Starting a RapidMiner Server via AWS

felix_w
felix_w New Altair Community Member
edited November 5 in Community Q&A

Dear Rapidminer Community, 

 

first of all, I am a total noob when it comes to handling servers, sorry ;) There will maybe be some dumb questions in the next paragraphs. ;-) What I would like to do is to set up a Rapidminer Server via AWS and I have a few questions because I am already a little bit confused which steps I have to do to get this server running. (Yes, I have a Rapidminer Server license)

 

I will use the RapidminerServer for collaborative working with 4-6 people, how many GB RAM do I need, how many cores, how many GB space approximately? Is this offer succifient? 16GB RAM, 4 cores https://aws.amazon.com/marketplace/pp/B0742FWHKH . We don't have huge amounts of data atm and will also use our own laptops for calculations etc, the server functions mainly as a place to share processes and some data. That is why an offer, which only makes us pay if we actually use the server would be nice. 

 

I have watched the two videos provided by Rapidminer how to set up Rapidminer Server and just to make sure I would like to summarize the steps here, more or less a to-do list for me: 

 

  1. I have to install Java 8
  2. I have to set up a database, which one would you recommend? MySQL?
    1. I have to install this database on my local machine right?
  3. I would like to use the suggested offer from Rapidminer on AWS (https://aws.amazon.com/marketplace/pp/B0742FWHKH)
  4. Install RapidminerServer software on my laptop
  5. Connect my RapidMiner Studio with my AWS Rapidminer Server through a new repository 

Is this overall correct? 

 

I would use these sources as references for my installation: 

https://rapidminer.com/blog/installing-rapidminer-server-aws/

https://rapidminer.com/blog/installing-rapidminer-server-aws-part-2/

https://rapidminer.com/training/videos/

 

Further questions: 

.) People can work on the server even if my laptop is turned off?  Because if the database is on my laptop and its not turned on, the server should of course be accessible for everybody else.

.) Everybody else who would like to login to the server just needs to create a new repository in his/her RapidMiner Studio and login to the server? Or does everybody have to install RapidMiner Server and Java 8 etc?

.) Will the database on my laptop create a lot of data? 

.) I have RapidMiner Studio 8.1. installed, is this a problem? There is only 8.0 atm available for Server?

 

I know, a lot of questions but I hope you can still help me on my way to installing our own RapidminerStudio. :)

 

Best regards

Felix

Tagged:

Answers

  • felix_w
    felix_w New Altair Community Member

    I just found this article: https://docs.rapidminer.com/latest/server/installation/cloud_images.html 

     

    So, if I use the predefined image I don't have to install a database separately? Do I still have to install Java 8 then or does this all come as a bundle in the preset image?

     

    Best regards

    Felix

  • sgenzer
    sgenzer
    Altair Employee

    hello @felix_w - no worries about being a noob. We have ALL been there. :)  I did this a while ago and I think (?) it is much easier than all that. The database and the RM Server are all pre-configured on the instance. You literally just choose the size of your instance and press go.

     

    As for the size of the instance, I would recommend starting with a t2.large. It always worked well for me.

     

    Scott

     

     

     

  • sgenzer
    sgenzer
    Altair Employee

    oh you asked about your laptop...that does not matter at all. An instance on EC2 is literally a computer that you are renting by the minute in the cloud. It stands by itself. It will be there for however long you leave it on.

     

    Re: Server/Studio versioning. This is VERY important. Server and Studio must be the same version to "talk" to one another. Server and Studio 8.1 were released yesterday.

     

    Re: other users. Once you launch the instance, login as admin and create user accounts. Each user will have her/his own user repo ON THE SERVER (i.e. on that EC2 instance's MySQL database).

     

    Some other pieces of advice:

     

    • The beauty of EC2 is that you can turn on and off your server all the time - and you only pay for when it is ON. Hence mastering the AWS Lambda features is very good - setting CRON triggers to only have your server running when you need it.
    • I would also recommend setting up AWS Lambda scheduled snapshot backups.
    • In general on EC2, storage is pretty cheap; server time is NOT cheap. Turning things on and off is the key.
    • Make sure your security (IAM) is set up so that access to your servers is exactly what you need and nothing more.
    • I would recommend purchasing a dedicated IP address and associating it with your server. It makes accessing your server much easier. Otherwise every time you turn it on and off, you will get a new IP.

    Scott

  • SGolbert
    SGolbert New Altair Community Member

    Just a quick tip: You can set your database in Amazon RDS. You get 20 GB of storage for free and you are moving some of the processing and memory consumption out of your server.

  • sgenzer
    sgenzer
    Altair Employee

    absolutely correct. Thanks, @SGolbert. Is the setup with RDS any easier now than it has been in the past?

     

    Scott

     

  • Thomas_Ott
    Thomas_Ott New Altair Community Member

    @sgenzer what would be really great are some tips and tricks videos on how to spin up a RM Server on AWS. I did it but it I finally moved to a RM Server on a desktop.

  • sgenzer
    sgenzer
    Altair Employee

    are you reading my mind, @Thomas_Ott? I was thinking the exact same thing. :) OK they're on the list.

     

    Scott

     

     

  • felix_w
    felix_w New Altair Community Member

    Hi Scott, 

     

    thank you for your detailed reply!

     

    To summarize it for myself, please correct me if I am wrong

    • If I set up a predefined image on AWS I don't have to install a seperate database, Java 8 etc? I only need to decide which instance I take, e.g. t2.large, and AWS sets it up for me? Do I even need to download the RapidMiner Server software?
    • Subsequently, as soon as it is ready I can simply create a server repository in my RapidMiner Studio and connect it to the AWS Rapidminer server as well as everybody else who has access (after I set up their accounts)? That's it?
    • What do you understand by "server time"? Does this mean simply running the server without doing any actual work on it is costly? E.g. the server is online from 8am-8pm but I only do some work from 3pm to 5pm I have to pay the full 12h and not only the 2h I work on it? t2.large costs ~0,09$/h + 0,2$ infrastructure costs = 0,29$/h?
    • Can I turn on and off the server via RapidMiner Server or do I have to do that via my AWS account?
    • Can I configure the IAM and a buy a dedicated IP at AWS or where do I get that?
    • The AWS Lambda feature is accessible via an AWS acocunt?

     

    Best regards

    Felix

  • felix_w
    felix_w New Altair Community Member

    Anybody who can answer some of my questions above? :) 

     

    Best regards

    Felix

  • SGolbert
    SGolbert New Altair Community Member

    Hi Felix,

     

    I will try to answer some of the questions, but you will have to contact AWS for IP and billing details.

     

    • The RM Server image on AWS marketplace comes with Java8 and a database (I couldn't find out exactly which one, should be either MySQL or Postgresql). It is ready to go! More info: https://docs.rapidminer.com/latest/server/installation/cloud_images.html
    • T2 instances are charged by the hour. If they are running but not doing calculations, they accumulate "CPU credits". Then if they do heavy calculations later, they go into turbo mode and receive more processing power, but consume CPU credits. If you use the T2 unlimited option, they will always use full processing power when needed but you will get charged for the extra CPU credits that you use.

    Regards,

    Sebastian

  • felix_w
    felix_w New Altair Community Member

    Thank you for your answers Sebastian :)

     

    Best regards

    Felix

  • Nikouy
    Nikouy New Altair Community Member
    @sgenzer "mastering the AWS Lambda features is very good - setting CRON triggers to only have your server running when you need it."

    Hi Scott,

    How can CRON triggers be integrated with RM and Lambda to spin up an instance only when actually required?

    Thanks



  • sgenzer
    sgenzer
    Altair Employee
    hi @Nikouy try this https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/

Categories