[ASK] inconsistent performance on RM server

felix_laksana

Hi all,

I have some problem in RM server.

The first one is the performance when I run on rapidminer server is likely inconsistent.
Actually my process is a little bit complicated, like a lot of loops and I call a package within a package and use a read database (jdbc sql server) a lot..
I use "remember" & "recall" operator for example set, so I didn't use many read database operator because I feel when I use a read database operator it's kind of making the process a little bit slower.

I use postgres for rapidminer server (profesional) database and all of the process is on the same server.

And another one is I cannot delete the services that I created on RM server..
It says "Deletion of service <NewService> failed."

The things that I want to ask is :
1. Is there any clue what I missing here for inconsistent performance, is there any configuration to increase the performance?
2. Shall I run the standalone.bat or it will run automatically?
    Because I already tried when I restart the server and I try to log in into the RM server it works.
3. When I go to Administraction>System Information I see total memory 513MB, Max memory 8GB and Free memory 179MB..
    Is it normal? The total memory will adjust base on RM usage?
4. Is "read database" operator is taking sometimes to create a connection into the database (in my case I connect it into SQL Server database)?
5. Why I cannot delete the service that I create in RM server?

Sorry for a lot of question because I'm quiet new to rapidminer and I still need to digging up more for use it as simple and efficient as possible.
I'll post the XML if it was needed..
Here is screenshot of the package (there 9 package that call one another and wrapped in a single package "mainprocess")



Thanks in advance
Felix

Marco_Boeck

Hi,

1) That depends on various factors:
- your general system usage
- how many processes you are running at the same time
- if you're running into memory shortage
- if you're using the server for other things at the same time
- what exactly you are doing in a process. Some may need a lot of memory cleanup after certain parts which can take a bit of time

2) If you are starting the server as a Windows service, you do not need to start it via standalone.bat. You can edit the Windows service to configure how it is started, e.g. after booting. Starting the .bat manually then will actually launch a second server instance in a broken state (because all ports are already in use).

3) Yes. Java will take up to the max memory if required. Note that once you get close to that number, it will frantically start trying to free memory.

4) It depends. I can connect to a local MySQL instance and read a small table from it within less than a second. But depending on DB type, configuration and network settings, it may take longer.

5) I don't know. It may be that it is used in a reporting component? Otherwise you can check the server log file (SERVER/standalone/log/sever.log) for more information.

Regards,
Marco

felix_laksana

Hi Marco,

Thanks for your reply 

Marco Boeck wrote:

Hi, 1) That depends on various factors: - your general system usage (the system usage is normal) - how many processes you are running at the same time (only 1 process running in the server) - if you're running into memory shortage (i've checked the memory usage and it's still normal (Free more than 5GB)) - if you're using the server for other things at the same time (I'm the only one that use the server (it was a development server for RM), all things that needed for the process was stored in the same machine including SQL server database (for data)) - what exactly you are doing in a process. Some may need a lot of memory cleanup after certain parts which can take a bit of time (I try to use Free memory operator and it was slower about 1 second than before) 2) If you are starting the server as a Windows service, you do not need to start it via standalone.bat. You can edit the Windows service to configure how it is started, e.g. after booting. Starting the .bat manually then will actually launch a second server instance in a broken state (because all ports are already in use). (I already check for it in windows service and  there's RM service which is run automatically on windows startup) 3) Yes. Java will take up to the max memory if required. Note that once you get close to that number, it will frantically start trying to free memory. (okay thanks marco) 4) It depends. I can connect to a local MySQL instance and read a small table from it within less than a second. But depending on DB type, configuration and network settings, it may take longer. (I connect to SQL server database using jtds, it was less than a second but I use it a lot (more than 10 read database operator), the sql server database was stored in the same server as RM) 5) I don't know. It may be that it is used in a reporting component? Otherwise you can check the server log file (SERVER/standalone/log/sever.log) for more information. (I try to rename the service and after that the old service is still there and including the rename 1, after that I try to delete the old one and it's said failed to delete the service.) Here's some of the log : 14:50:13,583 WARNING [de.rapidanalytics.web.beans.ExportedProcessBean] (http-/0.0.0.0:8080-2) Failed to delete service NewService1: javax.ejb.EJBTransactionRolledbackException: Transaction rolled back: javax.ejb.EJBTransactionRolledbackException: Transaction rolled back Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. Caused by: javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement Caused by: org.postgresql.util.PSQLException: ERROR: ƒe[ƒuƒ‹"ra_ent_properties"‚̍XV‚Ü‚½‚͍폜‚́Aƒe[ƒuƒ‹"ra_ent_exportedprocess"‚ÌŠO•”ƒL[§–ñ"fk82149e58a6f10aad"‚Ɉᔽ‚µ‚Ü‚·   Detail: ƒL[(id)=(14179)‚Í‚Ü‚¾ƒe[ƒuƒ‹"ra_ent_exportedprocess"‚©‚çŽQÆ‚³‚ê‚Ä‚¢‚Ü‚· Regards, Marco

I'll post the process below this post.

Thanks
Felix

felix_laksana

I cannot post the XML in here because the maximum character exceeded.
I already uploaded it into the google drive.
Here's the link :
https://drive.google.com/folderview?id=0B_QIbuN0wh6VZ3BUTWJGNnpacHc&usp=sharing

Thanks
Felix

Marco_Boeck

Hi,

I forgot to ask the exact version of RapidMiner Server you are using.

Apart from that though I'm sorry but I do not have the time to go through your complex process setup. The error you are getting from PostgreSQL looks very weird indeed, I have no idea what it's about. I can delete services on our test instances no problem. This plus the performance problem when reading from your database could imply that something about your database setup is broken. I'd look into DB and JDBC driver version and if there are updates available.

As a sidenote, I'd recommend to separate the actual DB RapidMiner Server needs to run from any other database you're pulling data from.

Edit: I also just learned that PostgreSQL offers horrible performance because of how metadata is handled by the regular JDBC driver (aka consuming linearly more time depending on the number of tables in the DB).

Regards,
Marco

felix_laksana

Hi Marco,


I use RM Server V 6.0.2 ..
And for designer I use RM 6.1.0 ..
For data I use Ms SQL Server 2008 R2 express edition.

Yeah, I think there some problem with the RM server database, the data might be broken so whenever i want to delete the service it will face constraint problem..
Btw is it better to change my RM Server DB from PostgreSQL into MySQL?


Thanks
Felix

Marco_Boeck

Hi,

there is no right or wrong answer to that. It might be beneficial to try it out in your case. Also I suggest using matching versions (Studio 6.2 and Server 2.2 at the moment), otherwise you may experience inconsistencies when executing a process locally in Studio vs executing it on the Server.

Regards,
Marco

rapidprototypec

It's a little difficult to handle PostgreSQL jobs.