"Could not initialize class sun.awt.X11.XToolkit when using Grid Engine"

balamir

I'm using rapidminer 4.2  with Sun Grid Engine and I observed a strange  error. I submit several jobs to GE, each of which is a script to run rapidminer command line  to create a model with specific training set and then run rapidminer to find the accuracy with a common test set. GE provides the infrastructure to run these scripts on different machines. All my execution hosts are Kubuntu linux.  I noticed that some of my scripts failed with   "Could not initialize class sun.awt.X11.XToolkit"  error. An example execution is below:

G Oct 20, 2008 4:26:28 PM: ----------------------------------------------------
G Oct 20, 2008 4:26:28 PM: Initialization Settings
G Oct 20, 2008 4:26:28 PM: ----------------------------------------------------
G Oct 20, 2008 4:26:28 PM: Default system encoding for IO: UTF-8
G Oct 20, 2008 4:26:28 PM: Load core operators...
G Oct 20, 2008 4:26:28 PM: Load Weka operators: true
G Oct 20, 2008 4:26:28 PM: Load JDBC drivers from lib directory: true
G Oct 20, 2008 4:26:28 PM: Load JDBC drivers from classpath: false
G Oct 20, 2008 4:26:28 PM: Load plugins: true
G Oct 20, 2008 4:26:28 PM: Load plugins from '/opt/rapidminer-4.2/lib/plugins'
G Oct 20, 2008 4:26:28 PM: ----------------------------------------------------
G Oct 20, 2008 4:26:28 PM: Read rcfile '/opt/rapidminer-4.2/etc/rapidminerrc'.
G Oct 20, 2008 4:26:28 PM: Trying rcfile '/opt/rapidminer-4.2/etc/rapidminerrc.Linux'...skipped
G Oct 20, 2008 4:26:28 PM: Trying rcfile '/home/bayazit/.rapidminer/4_2_0_rapidminerrc'...skipped
G Oct 20, 2008 4:26:28 PM: Trying rcfile '/home/bayazit/.rapidminer/4_2_0_rapidminerrc.Linux'...skipped
G Oct 20, 2008 4:26:28 PM: Trying rcfile '/home/bayazit/rapidminerrc'...skipped
G Oct 20, 2008 4:26:28 PM: Trying rcfile '/home/bayazit/rapidminerrc.Linux'...skipped
G Oct 20, 2008 4:26:28 PM: Trying rapidminer.rcfile. Property not specified...skipped
G Oct 20, 2008 4:26:29 PM: Loading operators from 'operators.xml'.
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'OperatorChain': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'ModelApplier': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'ModelUpdater': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'ModelGrouper': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'ModelUngrouper': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'Process': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'Experiment': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
G Oct 20, 2008 4:26:32 PM: [Error] Cannot register 'MemoryCleanUp': java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11.XToolkit
........

I don't think rapidminer command line should do anything related to sun's X11 toolkit. I have solved my problem and I will describe it below, but I think the problem is rapidminer (or java) noticing a display variable set, tries to initialize XToolkit even though it is not going to open a window. This is fine as long as  you are running the command line in an X environment but it will start causing problems if the  environment's authorization to  access X  is removed later. Here is why I think that is the problem.


GE requires at least one  master and several execution hosts. Each execution host runs  a special GE program that communicates with GE master, receives the scripts and runs them. To start  these programs,   I ssh'ed to my execution hosts. By default, ssh provides X11 tunnel and terminal's display is set. So when I started running GE execution program on my execution host, X11 display was set, even though I used a console. All rapidminer jobs  run without problem on execution hosts until I closed my "ssh session"s to the hosts and  they started giving " sun.awt.X11.XToolkit" errors.   GE execution program is basically a daemon so it works in the background, When I closed   a ssh session, the X11 connection to that host was distrupted but the execution program still had X11 variable set, passing this information to the scripts it executed. So even though rapidminer runs in the command-line mode, it does something related to X11 and fails.

My solution was to start GE execution programs by    ssh'ing without X11 tunnel ( ssh -x option in linux). This  made sure that display was not set on the execution hosts and rapidminer worked fine.  I don't know if this is related to java VM or the way rapidminer setup, but I thought rapidminer developers should be aware of this problem.

radone

I had a similar problem running RM on a linux box (not on a grid).
My problem was I had to execute RapidMiner from "RapidMiner\scripts\rapidminer".

radone

fischer

Hi,

thank you for describing your problem and the solution. We are aware of these problems, and I agree that it is annoying. The problem is that various method calls trigger AWT initialization code in Java, even though they do not open windows. We will try to get rid of this problem, although this is hard to guarantee since we cannot control what the libraries we are using do.

Best,
Simon

Igor

Hello together,

I have the same problem with the grid system LSF using RapidMiner 4.5.
Is it solved in the latest RM version?

Cheers,
Igor

land

Hi,
you can set RapidMiner to be executed in a headless mode using the API. So it is solved.

Greetings,
  Sebastian

Igor

Yes, thanks a lot, I added "-Djava.awt.headless=true" and it works now!

Kind regards,
Igor