Why would you use Python instead of RapidMiner?

Mariusz

Hello! I am doing degree in AI engineering and I've been assigned task of comparing RapidMiner to Python. While I'm not a Python beginner, I'm fairly new to the RapidMiner and I definitely lack industrial experience. Given my limited knowledge, I still find RapidMiner superior in carrying out tasks I would usually code in Python. So my question is: In what circumstances, or for what specific task you would use Python instead of RapidMiner, and why?
Cheers, Mariusz.

lionelderkrikor

Hi Mariusz,

First, in introduction, I recall that you can integrate Python scripts inside RapidMiner, via the Execute Python operator (via Python Scripting extension to install from the marketplace)  .
By this way, you can benefit of some Python feature(s) inside RapidMiner.
Now, to answer to your question, I personnaly envisage to use Python for 3 tasks for potential future projects : 

1.  Use models/algorithms which are not implemented in RapidMiner. I'm thinking especially to 2 performing algorithms : 

•   Light GBM
•   XGBoost

     2.  For a project involving time series, I envisage to use TSFRESH a python package which automatically calculates a large number (dozens) of time series caracteristics (including complex caracteristics), the so called features. In comparaison RapidMiner has the Extract Aggregates operator but it calculates around 12 features.

     3. For image mining, there is no "out of the box" solution implemented in RapidMiner, for example to read directly an image.
         So, for image/video processing ,  I'm using OpenCV a powerful python package dedicated to this task.

Hope this helps,

Regards,

Lionel

PS : If, of course, one or more feature(s) I have described in this post are already implemented in RapidMiner, thanks to correct me and explain how to  access it (them).

kypexin

Hi @Mariusz

I work in the field of credit risk and scoring, so that's exactly my case for now:

• Not all domain specific algorithms even exist in RM, for example weight of evidence and information value algorithms.
• In production, there's a legacy system based on AWS docker and specific Python code, which uses lambda function to deploy web services in production. 

That's why I use RM first for quick data cleaning and model prototyping, and then I move everything to Python, including training production models as well (which are serialised via pickle) and then creating a web service for deploying on AWS.

So in general, that's in many aspects a question of legacy production systems and a whole IT ecosystem in the organisation. When I was once creating a fraud detection engine from scratch using RM, that was a perfectly closed ecosystem which didn't need anything except RM itself. 

lionelderkrikor

Hi Mariusz,

First, in introduction, I recall that you can integrate Python scripts inside RapidMiner, via the Execute Python operator (via Python Scripting extension to install from the marketplace)  .
By this way, you can benefit of some Python feature(s) inside RapidMiner.
Now, to answer to your question, I personnaly envisage to use Python for 3 tasks for potential future projects : 

1.  Use models/algorithms which are not implemented in RapidMiner. I'm thinking especially to 2 performing algorithms : 

•   Light GBM
•   XGBoost

     2.  For a project involving time series, I envisage to use TSFRESH a python package which automatically calculates a large number (dozens) of time series caracteristics (including complex caracteristics), the so called features. In comparaison RapidMiner has the Extract Aggregates operator but it calculates around 12 features.

     3. For image mining, there is no "out of the box" solution implemented in RapidMiner, for example to read directly an image.
         So, for image/video processing ,  I'm using OpenCV a powerful python package dedicated to this task.

Hope this helps,

Regards,

Lionel

PS : If, of course, one or more feature(s) I have described in this post are already implemented in RapidMiner, thanks to correct me and explain how to  access it (them).

Mariusz

This is great! Thanks for taking the time to answer my question. I knew about the Python operator, but I've only re-used previously written code with it, instead of building process from scratch. It's actually the only thing in favour of Python I found myself: the amount of reusable materials on the internet. Your answer gives me more to go about, thanks again!

Kind regards
Mariusz

SGolbert

Hi Mariusz,

I add that RapidMiner acts as a (data) platform for machine learning processes. There is a big gain in productivity when working in a team with RM, as you get a central repository with scheduling, web apps, services, etc. (RM Server). If you are using RM Studio alone, the productivity depends on how proficient you are with either tool. The learning curve for RM is definitely better, but you pay a price (literally if you buy a license) or suffer for limitations on the hardware you can use.

Regards,

Sebastian

kypexin

Hi @Mariusz

I work in the field of credit risk and scoring, so that's exactly my case for now:

• Not all domain specific algorithms even exist in RM, for example weight of evidence and information value algorithms.
• In production, there's a legacy system based on AWS docker and specific Python code, which uses lambda function to deploy web services in production. 

That's why I use RM first for quick data cleaning and model prototyping, and then I move everything to Python, including training production models as well (which are serialised via pickle) and then creating a web service for deploying on AWS.

So in general, that's in many aspects a question of legacy production systems and a whole IT ecosystem in the organisation. When I was once creating a fraud detection engine from scratch using RM, that was a perfectly closed ecosystem which didn't need anything except RM itself.