Data Storage in RapidMiner

xstream
xstream New Altair Community Member
edited November 5 in Community Q&A
Hello,

i would like to know how data is stored in rapid miner if I import a csv file for example.
Is there an internal relational database in which the data is imported?

Do I need an external database like oracle when I work with a huge amount of data?
Tagged:

Answers

  • land
    land New Altair Community Member
    Hi,
    data is either stored in the RAM of your computer or in the database you connected to.
    Unfortunately all common relational databases do not support more than something around 1000 columns, which is to restrictive for many data mining applications. So sometimes you have to store your data in main memory.
    If you have a realy huge amount of data, you will need an external database. This is simply the case if the data does not fit into your RAM. But be careful: The usage of most datamining algorithms is prohibitive with such huge example sets, since their runtime grows with the third power of number of examples. Use learners like NaiveBayes or Perceptron for such applications.

    Greetings,
      Sebastian

Categories