Online learning with RapidMiner

tonio09

Hi, I'm developing an automated trading bot that receives streaming data from various sources including social media streams and market trade streams, runs machine learning algorithms on it and executes trades.

I'm using RapidMiner to prototype the machine learning system.

Is it possible to integrate RapidMiner into a Java program running on a server, do online learning with streaming data, adjust the model continuously and produce signals and predictions that can be fed back into the Java program?

MariusHelf

Hi,

RapidMiner does not yet support stream mining or online learning, but we just laid the foundation of a stream mining framework. It will need some time though until it is ready for release.
Concerning your question about the server: RapidAnalytics can do that for you. It runs on a central server, you edit your processes with RapidMiner and store them on the server in a so-called remote repository, which integrates seemlessly into RapidMiner and looks just like the standard local repository to the user.
After storing the processes on the server, you can schedule them to run on a regular basis and create e.g. a model. You can store the model either in the repository for later use, or you can feed new data to the server via a webservice, and RapidAnalytics applies the model and returns the classification result (or anything else you configured it to return).

Via the execute script operators you can of course also produce signals to external programs.

Best, Marius

adrecord

Has the online learning support been added to RapidMiner yet?

MariusHelf

Hi,

unfortunately it has not been added yet.

Best regards,
Marius

vineet_p_s

Hello
Has RapidMiner come up with a stream plugin or something similar. Use case is to read live streaming data from twitter and provide analytics.
Thanks
Vineet

sgenzer

oh yes @vineet_p_s - very nice operator that pulls in Twitter feeds.    

 

There are also many processes here on the Community where people have many use cases.  Just search "Twitter"

 

Scott

vineet_p_s

Hello Scott
The operators you mention do not capture stream data. May be I didn't explain my requirement clearly.
Basically I want to know if we can use data stream in RM. something like Apache Storm or Kafka.

Thanks
Vineet

sgenzer

hello @vineet_p_s - my apologies for not understanding your question.  No RapidMiner currently does not support streams from Apache Storm or Kafka.

 

Scott