RapidMiner - Python

Papad

Hi everyone,
I want to built a model for linear data, but I want to pass into it many setpoints in order to make right predictions.
Is it possible to achieve that through Python? Is there any other automated way to do this?
I think that it would be easier to write my script an use it into RapidMiner.
What do you think?
Thanks in advance.

SGolbert

Hi @Papad

I understand a bit more, but the problem is still quite abstract. Maybe you can provide a dataset to illustrate it.

It seems to involve time series, you can take a look at the time series extension. I'm pretty sure that if you are proficient enough in RM, you don't need Python. But if it's easier for you to use a combination of both (or Python alone for that matter), you are free to use what works best. The complexity lies on the problem itself and not on the tool that you use to solve it.

Regards,

Sebastian

David_A

Hi,

for a linear prediction you don't necessarily need python. RapidMiner already has various linear prediction models available. Just enter "linear" in the Operator search and you will get for example  "Linear Regression", "Generalized Linear Model", "Vector Linear Regression" and "Support Vector Machine (Linear)".

On the other hand, if you want to compare to a specific python model, you can still use the python scripting extension. To set there specific parameters, you can either specify them by hand, use the new python notebook integration or change the parameters with RapidMiner macros; the macros with %{macro_name} are evaluated before parsing the python script.

Best,

David

varunm1

Hello @Papad

@David_A informed you about Linear models in RapidMiner. Coming to this question.

 Is there any other automated way to do this?

Are you looking to build a model and then feed the test data (unlabelled) to make predictions periodically? If so, you can use rapidminer server to automate your process to make predictions periodically.

Thanks,
Varun

Papad

Hi @varunm1
What I don't want only a prediction, but I want to find correlations between some attributes which change linear by time. So if I see somethings different in values or in graphs I must give the reason that causes an event, so next time my model will recognize it.
If I am not clear enough ask me whatever you want.
Thanks again.

Papad

Hello @David_A ,
on what you saying, can I use macros like a specific variable that I would use in python? Or have I understood it wrong?
Thanks.

SGolbert

Hi @Papad

I think the right question if whether is possible to do that in RapidMiner. I didn't understand the use case fully, so I cannot fully answer, but in general the answer is yes and there is no need to bring Python into the party.

I would appreciate it if you explain the use case a bit better and try a (basic) solution in RapidMiner first.

Kind regards,

Sebastian

Papad

Hello @SGolbert ,
Imagine that I have a machine and some attributes of it. The basic attributes are speed and pressure. 
They involve linear ,and if there is any problem, the time that machine stops is a good clue to know.So the goal is to find correlations between all attributes and in the end find the best way to improve the speed. Although I have to train a model because anytime the machine stops, my model has to know the reason in order to predict what is going to happen after that.For example, if I have pressure under the average for an hour, it may means that a part of the machine needs to be changed because it is almost broken. I hope I gave you a clear view of what I want.
Thanks for your time.

jacobcybulski

I think you are asking the question if it is OK to create a model in Python, return it, pass it around your RapidMiner process, and then pass it to another Python call for evaluation or application.

If this is a question (regardless if this is a linear model or something more complex) then it is possible. However, we have some problems with this. First, when the Python script is executed, the Python process attached to it is terminated, so you can no longer talk to it or ask it for data. Secondly, what RapidMiner expects on output is something "tangible" that could be passed around, ideally this should be a Pandas data frame. So here you have two best options:

1. Save you model to disk (e.g. /tmp/mymodel) and return a data frame with the path to your model ('/tmp/mymodel')
2. Serialise your model into a JSON format (e.g. using JSON encoder) and return a data frame with that blob as a value)

RapidMiner will then be happy to pass your data frame around as an example set and then use it as input into another Python script which could load the model back or simply deseralise (e.g. using JASON decoder) and use it as appropriate.

I used both methods, e.g. saved deep learning models in one Python process and loading it back and using in another.

Jacob

SGolbert

Hi @Papad

I understand a bit more, but the problem is still quite abstract. Maybe you can provide a dataset to illustrate it.

It seems to involve time series, you can take a look at the time series extension. I'm pretty sure that if you are proficient enough in RM, you don't need Python. But if it's easier for you to use a combination of both (or Python alone for that matter), you are free to use what works best. The complexity lies on the problem itself and not on the tool that you use to solve it.

Regards,

Sebastian