how to submit NEW EXAMPLES to a learned model

Unknown

Hi there more advanced users of RM

I learned a model, now I want to place an query (submit a new example/observation and see the models response)
I do not know how to do it , I do not want to learn the models params every single time I want to load it the learned model and submit to it a new example.

thanks

haddock

Hi there,

It takes a bit of time to see the world from the RM point of view; of course any view is acceptable in a liberal world, but the one you are grappling with comes from Dortmund. In that world models get made from data and are applied to other data; here application means something quite specific, namely that a new prediction column is created, and filled with values for the attribute being modeled ( the label ).

What tends to happen is that the model gets applied to lots of new cases at once, as in validation, or its parameters get tweaked as it gets applied to lots of new cases, as in optimisation ploys. So applying a model to just one case is actually quite rare from the modeling point of view, but may be handy from the application point of view. In the latter case you should consider RapidAnalytics, a mighty server that lets the masses play with your delicately engineered models ... ( check out Simon's rather good videos to see this beast at work ).

But the bottom line remains that if you really want to get the most out of RM you need to work through the support material,  tutorials,samples, videos etc.. it just saves time in the long run.

Look at operators as Lego that data goes through and you won't go far wrong!

Have fun...

Unknown

haddock wrote:

Hi there, It takes a bit of time to see the world from the RM point of view; of course any view is acceptable in a liberal world, but the one you are grappling with comes from Dortmund. In that world models get made from data and are applied to other data; here application means something quite specific, namely that a new prediction column is created, and filled with values for the attribute being modeled ( the label ). What tends to happen is that the model gets applied to lots of new cases at once, as in validation, or its parameters get tweaked as it gets applied to lots of new cases, as in optimisation ploys. So applying a model to just one case is actually quite rare from the modeling point of view, but may be handy from the application point of view. In the latter case you should consider RapidAnalytics, a mighty server that lets the masses play with your delicately engineered models ... ( check out Simon's rather good videos to see this beast at work ). But the bottom line remains that if you really want to get the most out of RM you need to work through the support material,  tutorials,samples, videos etc.. it just saves time in the long run. Look at operators as Lego that data goes through and you won't go far wrong! Have fun...

Hi haddock

so are you saying if I or the masses would like to play with my delicate model such as asking it what cluster this example falls in etc... I would need to use RA and run the model as a process on the RA server? Is there no way how to accomplish that from RM?

E.g. I saw read/write parameters/weights operators and param set in RM...?
ty

haddock

so are you saying if I or the masses would like to play with my delicate model such as asking it what cluster this example falls in etc... I would need to use RA and run the model as a process on the RA server? Is there no way how to accomplish that from RM?

No.

What I am saying is..

It takes a bit of time to see the world from the RM point of view; of course any view is acceptable in a liberal world, but the one you are grappling with comes from Dortmund. In that world models get made from data and are applied to other data; here application means something quite specific, namely that a new prediction column is created, and filled with values for the attribute being modeled ( the label ). What tends to happen is that the model gets applied to lots of new cases at once, as in validation, or its parameters get tweaked as it gets applied to lots of new cases, as in optimisation ploys. So applying a model to just one case is actually quite rare from the modeling point of view, but may be handy from the application point of view. In the latter case you should consider RapidAnalytics, a mighty server that lets the masses play with your delicately engineered models ... ( check out Simon's rather good videos to see this beast at work ). But the bottom line remains that if you really want to get the most out of RM you need to work through the support material,  tutorials,samples, videos etc.. it just saves time in the long run. Look at operators as Lego that data goes through and you won't go far wrong!

Unknown

Hi haddock

could you please post a simple xml project(e.g with NN) example for RA where the querying on a learned model can be done?

It is very surprising that one cannot do inference, classification and the like on single input instance  under a learned model in a straight forward fashion.
I thing these paradigms are an integral part of ''the model theory''.
cheers
f

haddock

one cannot do inference, classification and the like on single input instance  under a learned model in a straight forward fashion.

But you can do this in RM, and I specifically say so, several times. Which of my words gave you the impression that this is not possible?

IngoRM

Hi,

(EDIT: Ok, Haddock had answered while I was typing. Anyway, find my answer below...)

I don't know really what you are talking about but I will try it in another way (although Haddock gave a perfect concise description of the basic concepts):

Of course it is possible to create predictions for fresh and unseen data (which is called "apply a model" in the RM-world) with RapidMiner as well as with RapidAnalytics. The latter might be by far more appropriate if you want to integrate those predictions into other software but still it's also possible with RM directly.

I do not know how to do it , I do not want to learn the models params every single time I want to load it the learned model and submit to it a new example.

The basic idea is:

• load the training data
• learn the model and tune its parameters
• store the trained model for later use
• load new data
• apply the model on this new data with the operator "Apply Model" (by the way, it's called 'apply model' instead of 'create prediction' since there are also (preprocessing) models which do not create a prediction but still can be applied for data transformation...)

The following resources should help you:

• The Manual:
• The samples which are delivered with RapidMiner in your first repository (especially 01_Learner/11_ModelApplier)
• The Video Tutorials (especially tutorial No 3): http://rapid-i.com/content/view/189/212/
• RapidMiner Resources: http://rapidminerresources.com/

Cheers,
Ingo

Unknown

Ingo Mierswa wrote:

Hi, (EDIT: Ok, Haddock had answered while I was typing. Anyway, find my answer below...) I don't know really what you are talking about but I will try it in another way (although Haddock gave a perfect concise description of the basic concepts): Of course it is possible to create predictions for fresh and unseen data (which is called "apply a model" in the RM-world) with RapidMiner as well as with RapidAnalytics. The latter might be by far more appropriate if you want to integrate those predictions into other software but still it's also possible with RM directly. The basic idea is: load the training data learn the model and tune its parameters store the trained model for later use load new data apply the model on this new data with the operator "Apply Model" (by the way, it's called 'apply model' instead of 'create prediction' since there are also (preprocessing) models which do not create a prediction but still can be applied for data transformation...) The following resources should help you: The Manual: The samples which are delivered with RapidMiner in your first repository (especially 01_Learner/11_ModelApplier) The Video Tutorials (especially tutorial No 3): http://rapid-i.com/content/view/189/212/ RapidMiner Resources: http://rapidminerresources.com/ Cheers, Ingo

hello Ingo

I was talking about being able to load a learned model and simply apply a new example in order to get the models response ("belongs to cluster 1", or "P(X/B/C)=x",etc.). I cannot afford to learn the model every time I want to have a new example evaluated when it takes minutes to hours to learn.

I have been just playing with "model save/load" operators when I read your response.

I'll do more homework I promise.

thx 

Unknown

It WORKS!  

The model write and model load operators is the way.

The RECIPE for all the kids:

1.Learn model-->Write Model
2.a)Read model
  b)new unlabelled data
-->apply model

is all!

IngoRM

Congrats!  ;D

And welcome to the RM-world. As you can probably see now, both Haddock and myself talked about that all the time 

By the way, instead of using the operators "Write Model" and "Read Model" you might also try to use the general purpose operators "Store" and "Retrieve" which work on the repository where you should store your processes and data anyway. I find it quite useful to also store the models there (as well as all other types of results) so I have everything in one central place.

Have fun! Cheers,
Ingo

Unknown

thx ;D

I am still getting used to the centralized repository but I guess it is a good thing even thoug data are in some proprietary binary code. Using the repository requires me to create a concise and exhaustive nomeclature thou.

I even connected MySQL to RM so I am quite happy now and discovering new things and principals every mminute!

cheers