Community & Support
Learn
Marketplace
Discussions
Categories
Discussions
- General
- Platform
- Academic
- Partner
- Regional
User Groups
Documentation
Events
Altair Exchange
- Share or Download Projects
Resources
Employee Resources
- This tab can be seen by employees only. Please do not share these resources externally.
Groups
- Join a User Group
Support

Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

Home
Discussions
Community Q&A

"RapidMiner in a Cluster"

Hi, I'm Pedro,

I would like to know how is the development to support parallelized processing in a cluster. I have more than 100GB of text to process and a cluster with 32 machines available.

Regards.

Find more posts tagged with

AI Studio

Clustering

Accepted answers

All comments

land

Hi Pedro,
one quite unsolved problem on machine learning is, that all the algorithms to build models are nearly unparallelizable. And since they are at least quadratic in runtime not applicable on all your data, even with 3000 machines.
But if you want to classify this amount of text, then you should train your model on one machine. The application of this model is highly independent and can be done even without any cluster structure by simply starting the application process on a subset of the data.

Greetings,
Sebastian

Quick Links

All Categories
Recent Discussions
Activity
Unanswered
日本語 (Japanese)
한국어(Korean)
Groups