Problem using the Store Operator to create IO Objects with RapidMiner Server

sCapatina

Hello,

I am trying to use the Store operator to create IO objects with scheduled server processes.

I want to store these objects either in the local or server repositories, and use RapidMiner Studio to Retrieve.

No matter what repository entry I specify, it does not seem to work.

The other option is to use CSV files instead - but I am wondering if its possible with Store.

I get the following errors:
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not execute statement
com.rapidminer.operator.UserError: Cannot store data in repository at entry '//RM_Server/home/admin/Cache/Store1'. Reason: Requested repository RM_Server does not exist..
com.rapidminer.operator.UserError: Cannot store data in repository at entry '//Local Repository/5 - Cache/Store1'. Reason: Requested repository Local Repository does not exist..

Great work with version 7 btw

Thank you!

Marco_Boeck

Hi,

the problem you are experiencing is because the repository names are specific to your local Studio instance. So if you try to access //Local Repository/ABC, your Studio instance knows what to do. RM Server however has no idea what you want, it does not share your local repository names. To fix this, simply use relative path names. So if your process is stored in user/admin/process/123, then you can store in the same folder by simply entering "myData" in the Store operator. If you want to navigate upwards in the folder structure of your repository, just add "../" before the name to go one folder up. You can add as many of those as you need to navigate up an arbitrary amount of folders.

Regards,
Marco

sCapatina

Hi Marco,

Thanks for your answer. Unfortunately I've tried that and get the following error:

javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not execute statement

Best,

Sebastian

Marco_Boeck

Hi,

cannot help you unless you provide a link to the full server.log (preferably taken directly after the problem occurred).

Regards,
Marco

vipul_kumar

Hi,

 

Even I face the same error when I run my process on server. It uses store operator.Below is the content of the log file generated. Hope it helps to find some input in solving this problem.

 

May 19, 2017 10:45:04 PM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Process context is: Macros: [_ra_user : admin, _ra_jobID : 28]; Input: []; Output: []
May 19, 2017 10:45:04 PM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: No filename given for result file, using stdout for logging results!
May 19, 2017 10:45:04 PM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Process //_LOCAL/classifyDocumentTypes.rmp starts
May 19, 2017 10:45:24 PM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Storing ioobject at /apps/Classify
May 19, 2017 10:45:27 PM de.rapidanalytics.execution.AbstractProcessExecutor runProcessNow
SEVERE: Process failed: javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not execute statement
May 19, 2017 10:45:27 PM <unknown> <unknown>
SEVERE: Here:           Process[1] (Process)
           subprocess 'Main Process'
             +- Process Documents from Files[1] (Process Documents from Files)
           subprocess 'Vector Creation'
             |     +- Tokenize[252] (Tokenize)
             |     +- Filter Stopwords (English)[252] (Filter Stopwords (English))
             |     +- Stem (Snowball)[252] (Stem (Snowball))
             |     +- Transform Cases[252] (Transform Cases)
             |     +- Generate n-Grams (Terms)[252] (Generate n-Grams (Terms))
             +- k-NN[1] (k-NN)
             +- Process Documents from Files (2)[1] (Process Documents from Files)
           subprocess 'Vector Creation'
             |     +- Tokenize (2)[28] (Tokenize)
             |     +- Filter Stopwords (2)[28] (Filter Stopwords (English))
             |     +- Stem (2)[28] (Stem (Snowball))
             |     +- Transform Cases (2)[28] (Transform Cases)
             |     +- Generate n-Grams (2)[28] (Generate n-Grams (Terms))
             +- Apply Model[1] (Apply Model)
       ==>   +- Store[1] (Store)

---

@Marco_Boeck wrote:
Hi,

cannot help you unless you provide a link to the full server.log (preferably taken directly after the problem occurred).

Regards,
Marco

---

vipul_kumar

Hi,

 

 

Please find attached the server log when the process is run on rapid miner server with store operator. 
The repository name parameter for store operator is : "NewRepository/home/admin/classifyDocumentTypes".

 

 

 

Edin_Klapic

Hi vipul_kumar,

 

you may find all you need in this new Knowledge Base article.

 

Best regards,

Edin

naveen_bharadwa

Hi Edin,

 

To add more to what vipul_kumar is trying to say, the store operator works perfectly fine to store data. I'm trying to store the data from the training process, to use the trained model for a scoring process. The process throws an exception while I'm doing this.  Attached is the exception log.

Jun 23, 2017 7:09:30 AM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Process context is: Macros: [_ra_user : admin, _ra_jobID : 38]; Input: []; Output: []
Jun 23, 2017 7:09:30 AM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: No filename given for result file, using stdout for logging results!
Jun 23, 2017 7:09:30 AM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Process //_LOCAL/POC/Process/PointerTerms/assignments_train starts
Jun 23, 2017 7:09:30 AM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Storing ioobject at /POC/Process/PointerTerms/wordList
Jun 23, 2017 7:09:31 AM com.rapidminer.server.repository.enterprise.EnterpriseIOObjectEntry storeData
INFO: Storing ioobject at /POC/Process/PointerTerms/applyModel
Jun 23, 2017 7:09:32 AM <unknown> <unknown>
SEVERE: Process failed: javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not execute statement
Jun 23, 2017 7:09:32 AM <unknown> <unknown>
SEVERE: Here:           Process[1] (Process)
           subprocess 'Main Process'
             +- Read CSV[1] (Read CSV)
             +- Select Attributes[1] (Select Attributes)
             +- Nominal to Text[1] (Nominal to Text)
             +- Data to Documents[1] (Data to Documents)
             +- Process Documents[1] (Process Documents)
           subprocess 'Vector Creation'
             +- Multiply[1] (Multiply)
             +- Store (2)[1] (Store)
             +- Process Documents from Files (2)[1] (Process Documents from Files)
           subprocess 'Vector Creation'
             |     +- Tokenize (2)[495] (Tokenize)
             |     +- Filter Stopwords (2)[495] (Filter Stopwords (English))
             |     +- Stem (2)[0] (Stem (Snowball))
             |     +- Transform Cases (2)[495] (Transform Cases)
             |     +- Generate n-Grams (2)[0] (Generate n-Grams (Terms))
             +- k-NN[1] (k-NN)
       ==>   +- Store[1] (Store)

The question here is, are there certain data types that the store operator doesn't handle or is it a petty mistake that I've done.

 

Attaching the process design if it could help.

 

Edin_Klapic

Hi Naveen,

 

there is nothing wrong with your process. The exception arises because the dataset you want to store is too big for the database.

This can be seen in the following snippet from the posted server.log.

 

Caused by: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (30517356 > 16777216). You can change this value on the server by setting the max_allowed_packet' variable.

The left number display the size of the Object you want to store and the right number the actual value of the variable (in Bytes).

The mentioned variable 'max_allowed_packet' in your database installation needs to be set to a higher value.

 

 

More info to this problem can be found here.

 

Best,

Edin