"Classpath to add Groovy Modules"

chaney

I'd like to use the HTTPBuilder Groovy modules, and while my groovy script works within Groovy, RapidMiner5 does not find the actual classes in the jar files.  In  RAPIDMINDER_HOME (on my machine set to c:\program files\rapid-i\rapidminer5\) I found a groovy-all.jar that seems to contain all class files that groovy needs.  Do I actually need to add my module to this .jar file (thus performing surgery on a number of other .jar files to get one encompassing "groovy-all.jar" file) or ...

may I set a classpath somewhere that "Execute Script" through RapidMinder would then read prior to execution so my classes would be found?

steffen

You can add your groovy-jar-file to the startup classpath.

Two options:
1. add the jar manually to the startup script (see <your-rapidminer-home-directory>/scripts/)
2. copy the jar into the <your-rapidminer-home-directory>/lib/ directory. Every jar found in this directory is added to classpath automatically.

hope this was helpful

steffen

chaney

Steffen-  thank you so much for responding.  Are you sure that in RapidMiner5 that copying the jar to the <your-rapidminer-home-directory>/lib/directory actually works?  I tried that before I posted and it didn't work. Putting the jars in <your-rapidminer-home-directory>/scripts/ doesn't work either.  I put a class HTTPBuilder.class into the groovy-all.jar, and that worked, but I am loathe to perform that kind of surgery. 

steffen

Hello again

1. Putting it into the /lib-directory - folder should work because it is specified that way in the startup-scripts.
2. Putting it into the /scripts - directory does not work. I meant something else. If you open the scripts located in scripts-directory with a text-editor (*.bat if you are on windows or without ending if you are on linux) you can add entries to the java-call opening rapidminer.

But in the far ends of my mind I remember that all these modifications do not work if you open rapidminer using the exe (assuming that you work on windows). So I suggest that you put the files into the lib-directory, move to the scripts - dir and open rapidminer via RapidMinerGUI.bat.

chaney

so what i have discovered is that groovy script execution within RM does NOT:

-  use the classpath of RapidMiner automagically,
- dropping a jar into the lib directory does not work,
- editing the *.bat file's classpath, using any of these - RAPIDMINER_CLASSPATH, COMPLETE_CLASSPATH, or RAPIDMINER_LIBRARIES - does not work.

You are able to write your own processes and extend them in RM, and RM picks up the references using the RM classpath,  but not using groovy.

I still haven't figured out where groovy script execution picks up the classpath.  Do any of the RM developers know?

steffen

Hello Chaney

Ok, this time I tried it out myself. I copied "commons-math-2.0.jar" (from apache project) to the lib-folder, moved to the /scripts directory (using the cmd-window), checked that RAPIDMINER_HOME is set correctly and opened rapidminer using the script RapidMinerGUI directly.

Then I  executed this process ...

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input>
      <location/>
    </input>
    <output>
      <location/>
    </output>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="619" width="947">
      <operator activated="true" class="execute_script" expanded="true" height="60" name="Execute Script" width="90" x="200" y="136">
        <parameter key="script" value="import org.apache.commons.math.stat.descriptive.DescriptiveStatistics;&#10;&#10;// Get a DescriptiveStatistics instance&#10;DescriptiveStatistics stats = new DescriptiveStatistics();&#10;&#10;// Add the data from the array&#10;for( int i = 0; i &lt;10 ; i++) {&#10;        stats.addValue(i);&#10;}&#10;&#10;// Compute some statistics&#10;double mean = stats.getMean();&#10;operator.logNote(&quot;mean:&quot; + mean);"/>
      </operator>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

and yes, it worked  ... from within eclipse. I always start rapidminer from within eclipse, so I sometimes miss problems other users have.

The problem is (as far as I see) the classloader:
When executing a groovy script, it uses the so called "Majorclassloader" which covers all jars mentioned in the manifest of rapidminer.jar and the plugins ... which does not include the jars added additionally to the lib-folder.

regards,

steffen

chaney

Steffen- Thank you for taking a second look at this.  I did start rm with the batch file.  You seem to be right, it looks like a dependency issue, I will copy all the references in the POM.xml and try it again with better results. 

I thought that the error thrown by groovy inside of rm would be different if the rm-managed groovy interpreter found the class I was calling in the jar file that was in the LIB ("unable to resolve class", with a reference to the class in the jar file (HTTPBuilder).

Plus, what also threw me off is that the groovy standalone outside of rm had no problems executing the same code, (not using grape or grab). 

I will see if I am able to figure it out, and report back.

- Carol

steffen

Hello again

I have created a modification of the groovy script operator which allows the loading of additional libraries. The plugin (including source) can be downloaded from here: http://www.steffen-springer.net/dev/rm/rapidminer-GroovyScripting-0.1.jar

Back to my example above: With the commons-math-path set correctly this script ...

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input>
      <location/>
    </input>
    <output>
      <location/>
    </output>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="309" width="806">
      <operator activated="true" class="gs:GroovyScripting" expanded="true" height="60" name="GroovyScripting" width="90" x="78" y="202">
        <parameter key="script" value="import org.apache.commons.math.stat.descriptive.DescriptiveStatistics;&#10;&#10;// Get a DescriptiveStatistics instance&#10;DescriptiveStatistics stats = new DescriptiveStatistics();&#10;&#10;// Add the data from the array&#10;for( int i = 0; i &lt;10 ; i++) {&#10;        stats.addValue(i);&#10;}&#10;&#10;// Compute some statistics&#10;double mean = stats.getMean();&#10;operator.logNote(&quot;mean:&quot; + mean);"/>
        <list key="additional_libraries">
          <parameter key="commons-math" value="c:\commons-math-2.1.jar"/>
        </list>
      </operator>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

should work  .

Some information:
- instead of specifying a library as an entry in the parameter list, the user could provide a ";"-separated list of libraries. Drawback: Less WYSIWYG. Improvement: One could add all libraries from a directory by specifying regexes.
- the solution has one general restriction: You cannot add libraries which (temporarily) overwrite classes from core-libraries. This is due to the internal top-down search order.

I would appreciate any feedback before I submit this as a feature request.


kind regards,

steffen

chaney

Steffen-  Thank you!  this is great! I'll review today and write back on the success of switching out the jars.  How I managed to work around the problem earlier:  I re-wrote what I needed using core classes, and didn't rely on any extension jars.  Thanks again - Carol

chaney

I just realized it was a plug-in, I'll report back on the installation and use of the plug-in.

land

Hi Steffen,
seems to be a good and valuable extension. We will include it as soon as possible (what still could last some days...), if you post it as a feature request with that extension linked.

Greetings,
  Sebastian