incompatible with SQLite???

ycchang

Hi,

I would like to use "Stream Database' operator to feed data to my module in RapidMiner. So I first got a SQLite JDBC Drive from here
http://rapid-i.com/rapidforum/index.php/topic,2449.0.html
And successfully install it on my RapidMiner through 'Manage Database Drivers' dialog.
Also, a new database connection was added through "Manage Database Connections' dialog. When I press the "Test' button, the connection status shows 'Connection OK'.

However, after I add a "Stream Database" to my 'Process' and then run it, I got an error message:

  Process Failed
  "Database error occured: SQLite only supports TYPE_FORWARD_ONLY cursors"
  Reason: SQLite only supports TYPE_FORWARD_ONLY cursors
  Offending operator: Stream Database

Forward-only-cursor probably is the only type of cursor supported by SQLite. (Precisely speaking, SQLite does not actually have cursors. But SQLite behaves like having a forward only cursor).

So, is SQLite really incompatible with RapidMiner? If not, then how could I bypass this error?

BTW, I searched the forum and found these two relevant topics
sqlite db import: http://rapid-i.com/rapidforum/index.php/topic,2449.0.html
Using SQLite DB with RM: http://rapid-i.com/rapidforum/index.php?topic=3581.0
Unfortunately, they couldn't provide any help in my case.

Thanks.

MariusHelf

Did you try Read Database? That's probably sufficient for your use case.

Best regards,
Marius

ycchang

Hi Marius,

Thanks for the quick reply.
Yes, I tried 'Read Database' operator and it did its job pretty well in my test run. Unfortunately, there are much more than 5 million records in my data set. If I want to analyze this data without using 'Stream Database" operator, I probably have to upgrade my current hardware or apply some ML tricks (e.g. split/resample the dataset manually).

Anyway, just want to know if I can use SQLite in 'Stream Database' operator or not. If there is no quick fix for this issue, then I will seek other alternative solutions or options.

Thanks for the help again.
Yi-Chien

Marius wrote:
Did you try Read Database? That's probably sufficient for your use case. Best regards, Marius

MariusHelf

Yi-Chien,

unfortunately this seems not to be possible because the JDBC driver for SQLite not not provide the necessary features. To workaround this problem you could export your data from SQLite into another SQL database, e.g. mySQL or Postgre.

Best regards,
Marius