how to properly connect to local mongo DB?

kayman · February 2017

Hi there, for testing purposes I am running following setup :

-> Rapidminer Studio

-> RapidMiner Server (on same device as Studio)

-> Mongo DB (localhost)

When using Studio I can access Mongo without issues using local repository and using the server repository (within Studio). However, when I want to run the process on the server directly it fails and I get a 'database misconfiguration error'

Any idea what I might be doing wrong, is there any specific thing I need to do to be able to call Mongo when using the server ?

Edin_Klapic · August 2017

Hi all,

the latest RapidMiner update v7.6 also provides an updated version of the NoSQL Extension.

After updating the extension on your local machine and on RapidMiner Server the connectivity issue no longer occurs.

Best,

Edin

MartinLiebig · February 2017

Dear Kayman,

you need to setup the connection on the server side. This connection is then passed to studio and you can use it there (also locally). There is an example for solr on the docs page: http://docs.rapidminer.com/server/administration/creating-connections/creating-other-conns.html - Mongo works pretty similar.

~Martin

kayman · February 2017

That's actually what I tried.

The problem is that when I test the connection directly on the server (connect -> test connection parameters) I get a nice green tick, but when I use a process that uses the same conenction I get this :

com.rapidminer.operator.UserError: The database configuration 'mongo_local_data' is invalid.

I've also gave access right to the whole user group to ensure I didn't miss something there.

So somewhere I am missing something. the error message is not really helping in understanding where I should do some troubleshooting.

kayman · February 2017

Anybody that was ever able to get local mongo working? I am not able to get it running.

Testing shows ok, executing shows 'the database configuration is invalid'

Running the same from desktop (while on server instance) works fine also, it only fails when executed directly on server.

Edin_Klapic · February 2017

Hi kayman,

I have a local instance running which now works fine but I remember that I ran into similar issues. Can you please help me narrowing down the problem so I can try to reproduce the connection problem:

Do you use authentication for the MongoDB connection? If yes - do you use the root user or did you create a special user?
Did you name the connection you created from within Studio exactly the same as the one you created for Server?

Best regards,

Edin

kayman · February 2017

Hi Edin,

I use no authentication, and basically I keep all out of the box settings.

So I'm using instance host = localhost, instance port is 27017.

I'm having the same setup for my server repo, and as mentioned running the flow from withing desktop (but using the server instances) works fine, running it directly on the server fails. Testing the connection directly on the server gives no error.

Hope you can use this to do some troubleshooting, it's kind of a holdback at the moment for me.

Edin_Klapic · February 2017

Hi kayman,

after some investigation I hopefully found a solution to your issue.

Login as admin on your RapidMiner Server webinterface - not within RapidMiner Studio
Delete the existing connection
Recreate the connection
Grant permissions

It seems that creating a MongoDB connection from withing Studio is a.t.m. not correctly populated to the server.

Hope this works,

Edin

kayman · February 2017

Nope :-(
Same problems.

Steps taken : Create a complete new connection on server using the webinterface, as admin.

I granted access, tested it and was confirmed ok. The test request was acknowledged by Mongo also so communication was indeed correct.

Next I refreshed the connections in desktop and newly created one appears correctly. I created a small process that stores a few examples into Mongo.

Running the process locally works fine, the data is added to Mongo as expected.

Running the same process on server gives me 'com.rapidminer.operator.UserError : The database configuration 'mongo_test' is invalid.

I've tried running the process both as admin and local user, but no difference. Both server and Desktop are 7.3

kayman · February 2017

Quick update,

I have the exact same behaviour on an additional test machine running Ubuntu. On this machine I have full admin rights and the same issue appears.

Installing connection on server (through browser), direct connection is working but using the connection through a process fails giving exactly the same error message.

Seems therefore using Mongo as localhost is not working (version 7.3)

Edin_Klapic · February 2017

Hi kayman,

I did the exact same steps as you described and everything works fine on my machine running MongoDB on localhost.

I am using RM version 7.3.001 and MongoDB version 3.4.1 (all on Windows).

Sorry I could not help

Best,

Edin

kayman · February 2017

Weird, it isn't really rocket science so this is giving me a serious headache as this is a potential showstopper for us.

Did you try running a real process storing data, or did you just test the connection itself?

The connection works for me also, it just gives a problem when I actually want to use it inside of a process.

I have it on 2 seperate machines, one running on windows and another running linux, only difference is that I am running mongo 3.2 on both instead of 3.4

I'll try if I can upgrade to 3.4 and hope that makes a difference, otherwise I'm stuck indeed.

This is my test script :

<process version="7.3.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve sample" width="90" x="45" y="34">
        <parameter key="repository_entry" value="sampledata"/>
      </operator>
      <operator activated="true" class="text:data_to_json" compatibility="7.4.001" expanded="true" height="82" name="Data To JSON (2)" width="90" x="246" y="34">
        <parameter key="ignore_arrays" value="false"/>
        <parameter key="generate_array" value="false"/>
        <parameter key="include_missing_values" value="false"/>
      </operator>
      <operator activated="true" class="nosql:mongodb_document_writer" compatibility="7.2.000" expanded="true" height="82" name="Write MongoDB (2)" width="90" x="380" y="34">
        <parameter key="mongodb_instance" value="mng_Reviews_models"/>
        <parameter key="write_concern_" value="Acknowledged"/>
        <parameter key="write_concern" value="Acknowledged"/>
        <parameter key="collection" value="sample"/>
        <parameter key="skip_invalid_documents" value="true"/>
      </operator>
      <connect from_op="Retrieve amz_de_LG_tv" from_port="output" to_op="Data To JSON (2)" to_port="example set 1"/>
      <connect from_op="Data To JSON (2)" from_port="documents" to_op="Write MongoDB (2)" to_port="documents 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

Screenshot from 2017-02-15 14-36-12.png

Edin_Klapic · February 2017

I used your process (with the Iris data set as source), stored it in the admin home folder and also renamed my MongoDB connection.

No error occurs - regardless of the way I execute the process.

One thing that pops into my mind.

What version of the NoSQL extension are you running on Server & Studio? Are those different?

Best,

Edin

kayman · February 2017

Nope, both are 7.3.0 :-(

Could also be something I did wrong with the Mongo installation, though as far as I recall that was also just a straight forward process and it seems to be working for any of my other applications using it (and with RM desktop)

Could it be related to the Mongo dbpath or so? Not that I see anything wrong with that, for my windows setup that is located on C:\data\db si that should be accessible by the server also I assume?

hcha · February 2017

I have exactly same issue with Rapidminer server, desktop and NoSQL extenstion in version 7.4. Server runs in Ubuntu 14.04. Mongo is 3.4

Edin_Klapic · February 2017

The standard installation procedure, as well as the MongoDB path, should not be the issue - I used it to.

Did you try to store and run the process not in a user home folder but instead in a generally available folder (e.g. groups)?

Else, I am running out of ideas

In the meantime I try to get some information from our development team.

Best regards,

Edin

kayman · February 2017

I can give it a try in a different folder but don't expect much tbh.

Could it be related to activation code? Mine is already a fairly old one, before noSQL was an option. Just trying to put additioanl ideas on the table...

[UPDATE] Installing the process in another folder (like groups) did not make any difference. The problem remains the same.

Edin_Klapic · February 2017

The activation code is not the problem.

@kayman and @hcha: Is it possible that you send me your server.log (located in InstallationDirectory/standalone/log/) of a day when the connection problem occurred (by PM)?

It helps our developers to further narrow down the issue.

Best regards,

Edin

yunior_betancou · April 2017

Hi @Edin_Klapic

I have exactly the same issue listed in the thread.

I am able to execute mongo reads from studio but it fails on the server. I have tried everythin listed in the thread with no avail.
Below is the full server log for the process execution. One interesting part is the warning that reads:

10:25:16,583 WARNING [com.rapidminer.server.tools.config.ServerConfigurationManager] (scheduledprocess_1491747916577) User null is not allowed to view configurationEntry: Access denied for entry Localhost of mongodb_instance

It seems the server configuration manager doesn't have a user thus it can't read the connection (my connections is called "Localhost").

Is there source code for the noSQL connector open and if so where is it hosted to take a look at what is going on?

Full log:

10:25:15,695 INFO [de.rapidanalytics.web.tools.ProcessServiceDelegate] (http-/0.0.0.0:8080-7) Scheduled process execution /home/admin/processes/Retrieve Data with parameter ID 22. Will fire first at Sun Apr 09 10:25:15 EDT 2017.
10:25:16,577 INFO [de.rapidanalytics.ejb.ProcessJobEJBImpl] (EJB default - 28) Job /home/admin/processes/Retrieve Data was assigned id 27
10:25:16,577 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Dispatching job 27 to queue DEFAULT. (SimpleProcessExecutor)
10:25:16,577 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Starting process /home/admin/processes/Retrieve Data in queue DEFAULT.
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process context is: Macros: [_ra_user : admin, _ra_jobID : 27]; Input: []; Output: []
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) No filename given for result file, using stdout for logging results!
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process //_LOCAL/home/admin/processes/Retrieve Data starts
10:25:16,583 WARNING [com.rapidminer.server.tools.config.ServerConfigurationManager] (scheduledprocess_1491747916577) User null is not allowed to view configurationEntry: Access denied for entry Localhost of mongodb_instance
10:25:16,583 SEVERE [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process failed: com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.: com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.
at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:169)
at com.rapidminer.operator.Operator.execute(Operator.java:1005) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:807) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_51]
at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.Operator.execute(Operator.java:1005) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1205) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1101) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1054) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1049) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at de.rapidanalytics.execution.AbstractProcessExecutor.runProcessNow(AbstractProcessExecutor.java:91) [rapidminer-server-ejb.jar:]
at de.rapidanalytics.execution.SimpleProcessExecutor.execute(SimpleProcessExecutor.java:31) [rapidminer-server-ejb.jar:]
at de.rapidanalytics.execution.ProcessExecutionDispatcher$ProcessRunnable.run(ProcessExecutionDispatcher.java:89) [rapidminer-server-ejb.jar:]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_51]
Caused by: com.rapidminer.tools.config.ConfigurationException: No such configured object of name Localhost of mongodb_instance
at com.rapidminer.server.tools.config.ServerConfigurationManager.logAccessViolationAndThrow(ServerConfigurationManager.java:102) [rapidminer-server-ejb.jar:]
at com.rapidminer.server.tools.config.ServerConfigurationManager.checkAccess(ServerConfigurationManager.java:97) [rapidminer-server-ejb.jar:]
at com.rapidminer.tools.config.ConfigurationManager.lookup(ConfigurationManager.java:340) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:161)
... 18 more
10:25:16,584 SEVERE [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Here: Process[1] (Process)
subprocess 'Main Process'
==> +- Read MongoDB[1] (Read MongoDB)
+- JSON To Data[0] (JSON To Data)
+- Select Attributes[0] (Select Attributes)
+- Rename[0] (Rename)
+- Generate Concatenation[0] (Generate Concatenation)
10:25:16,584 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Completed execution of process /home/admin/processes/Retrieve Data
10:25:16,586 INFO [de.rapidanalytics.ejb.ProcessJobEJBImpl] (scheduledprocess_1491747916577) Marking process 27 as completed with exception com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.
10:25:16,595 INFO [de.rapidanalytics.execution.LogRecordCollector] (scheduledprocess_1491747916577) Saving records now
10:25:16,617 INFO [de.rapidanalytics.execution.LogRecordCollector] (scheduledprocess_1491747916577) Completed saving records

Edin_Klapic · April 2017

Hi yunior_betancou,

thank you for sharing the log file. Especially the User null part is interesting.

We are investigating the problem and hope to be able to fix it soon.

Regarding your other question:

The noSQL extension is not publicly available.

Best regards,

Edin

sebas · May 2017

Same issue here. MongoDB localhost connection does not work on RapidMiner Server

Symptoms:

- Accessing the MongoDB directly from RapidMiner Studio works OK

- Setting up the connection in RapidMiner Server and testing it directly from the web interface works OK

- Running the process on RapidMiner Server generates exception : The database configuration is invalid

- User null message found in log file just before exception trace

Below the log dump

12:12:37,452 WARNING [com.rapidminer.server.tools.config.ServerConfigurationManager] (scheduledprocess_1494331957423) User null is not allowed to view configurationEntry: Access denied for entry Mongo_Processed of mongodb_instance
12:12:37,453 SEVERE [com.rapidminer.Process.21] (scheduledprocess_1494331957423) Process failed: com.rapidminer.operator.UserError: The database configuration 'Mongo_Processed' is invalid.: com.rapidminer.operator.UserError: The database configuration 'Mongo_Processed' is invalid.
        at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:169)
        at com.rapidminer.operator.Operator.execute(Operator.java:1004) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:807) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_121]
        at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.operator.Operator.execute(Operator.java:1004) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.Process.run(Process.java:1310) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.Process.run(Process.java:1173) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.Process.run(Process.java:1126) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.Process.run(Process.java:1121) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at de.rapidanalytics.execution.AbstractProcessExecutor.runProcessNow(AbstractProcessExecutor.java:91) [rapidminer-server-ejb.jar:]
        at de.rapidanalytics.execution.SimpleProcessExecutor.execute(SimpleProcessExecutor.java:31) [rapidminer-server-ejb.jar:]
        at de.rapidanalytics.execution.ProcessExecutionDispatcher$ProcessRunnable.run(ProcessExecutionDispatcher.java:89) [rapidminer-server-ejb.jar:]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_121]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_121]
Caused by: com.rapidminer.tools.config.ConfigurationException: No such configured object of name Mongo_Processed of mongodb_instance
        at com.rapidminer.server.tools.config.ServerConfigurationManager.logAccessViolationAndThrow(ServerConfigurationManager.java:102) [rapidminer-server-ejb.jar:]
        at com.rapidminer.server.tools.config.ServerConfigurationManager.checkAccess(ServerConfigurationManager.java:97) [rapidminer-server-ejb.jar:]
        at com.rapidminer.tools.config.ConfigurationManager.lookup(ConfigurationManager.java:340) [rapidminer-studio-core-7.5.0.jar:7.5.0]
        at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:161)
        ... 18 more

Edin_Klapic · May 2017

Hi all,

Thank you all for your valuable feedback incl. log files and stuff. I got the confirmation that we are working on this.

I will keep you updated as soon as I have more details I can share.

Best,

Edin

kayman · June 2017

Hi @Edin_Klapic, do you have some update on this?

Thanks in advance,

K.

Edin_Klapic · June 2017

The dev team is pretty packed up right now.

So no, unfortunately I have no update on this topic right now.

Best,

Edin

Edin_Klapic · August 2017

Hi all,

the latest RapidMiner update v7.6 also provides an updated version of the NoSQL Extension.

After updating the extension on your local machine and on RapidMiner Server the connectivity issue no longer occurs.

Best,

Edin

sebas · August 2017

I can confirm this works now

Thanks !

how to properly connect to local mongo DB?

Best Answer

Answers

Categories