"Simple java program"

nav · June 2011

I have tried to code a simple java program that read a csv file, create a naive bayes model and save the output model. But the compiler give me this error:

com.rapidminer.operator.OperatorCreationException: No operator description object given for 'com.rapidminer.operator.io.CSVDataReader'
at com.rapidminer.tools.OperatorService.createOperator(OperatorService.java:583)
at rapidProcess.exampleProcess.main(exampleProcess.java:24)

But in the API the CSVDataReader is included in com.rapidminer.operator.io so I don't understand the error.

This is the very simple code. Thanks in advance.



package rapidProcess;

import com.rapidminer.RapidMiner;
import com.rapidminer.operator.Operator;
import com.rapidminer.operator.OperatorCreationException;
import com.rapidminer.operator.OperatorException;
import com.rapidminer.operator.io.CSVDataReader;
import com.rapidminer.operator.io.ModelWriter;
import com.rapidminer.operator.learner.bayes.NaiveBayes;
import com.rapidminer.tools.OperatorService;
import com.rapidminer.Process;

public class exampleProcess {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		RapidMiner.init();

		try {

			/* Reading Data */
			Operator trainingDataReader = OperatorService.createOperator(CSVDataReader.class);
			trainingDataReader.setParameter("csv_file", "train.csv");

			/* Classifier */
			Operator bayesClassifier = OperatorService.createOperator(NaiveBayes.class);

			/* Save model */
			Operator modelWriter = OperatorService.createOperator(ModelWriter.class);
			modelWriter.setParameter("model_file", "prova_model");

			trainingDataReader.getOutputPorts().getPortByName("output").connectTo(bayesClassifier.getInputPorts().getPortByName("training set"));
			bayesClassifier.getOutputPorts().getPortByName("model").connectTo(modelWriter.getInputPorts().getPortByName("input"));

			Process process = new Process();

			process.getRootOperator().getSubprocess(0).addOperator(bayesClassifier);

			process.run();




		} catch (OperatorCreationException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (OperatorException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} 
	}

}

colo · June 2011

Hi nav,

I experienced a similar problem some weeks ago. My own program was working fine until some update to the RapidMiner core was released. I posted this information here: http://rapid-i.com/rapidforum/index.php/topic,294.msg13414.html#msg13414
But until now the problem was not fixed, I just noticed this when testing my program again after reading your post. I would also appreciate any other solution than switching back to the old revision.

Best regards
Matthias

nav · June 2011

So for the moment the unique solution is that to switch to a previous version?

Marco_Boeck · June 2011

Hi,

the CSVDataReader has (currently) been replaced with the CSVExampleSource class. So you'll need to use this code fragment:

OperatorService.createOperator(CSVExampleSource.class);

Regards,
Marco

colo · June 2011

Hi Marco,

can you provide a similar solution as simple as that for the "Get Page" operator (see http://rapid-i.com/rapidforum/index.php/topic,294.msg13414.html#msg13414)? I thought this error was raised by some unwanted modifications (that's why I just posted this as a remark in the linked topic) but I assume you renamed some classes, as your reply indicates. But I could not find anything suitable for the problem in my case. Any ideas?

Best regards
Matthias

Marco_Boeck · June 2011

colo wrote:

Hi Marco,

can you provide a similar solution as simple as that for the "Get Page" operator (see http://rapid-i.com/rapidforum/index.php/topic,294.msg13414.html#msg13414)? I thought this error was raised by some unwanted modifications (that's why I just posted this as a remark in the linked topic) but I assume you renamed some classes, as your reply indicates. But I could not find anything suitable for the problem in my case. Any ideas?

Best regards
Matthias

Hi,

when you init RapidMiner you have to set an ExecutionMode. Depending on this ExecutionMode, certain things are not available. I suggest using this to init RapidMiner if you need to work with plugin operators:


RapidMiner.setExecutionMode(ExecutionMode.COMMAND_LINE);
RapidMiner.init();

So much for the quick&dirty fix. However other ExecutionMode settings should work as well, so maybe there are some settings which one needs to know about.. I will have to ask around for this though..

Regards,
Marco

colo · June 2011

Hi Marco,

thanks for picking this up.
I tried different execution modes without any effect (didn't notice differences between UI, COMMAND_LINE or the EMBEDDED ones for my simple test program). Before updating to revision 313 everything worked fine (as reported in the linked topic), and when switching back to 312 it still does. I took a short look at the code changes but am not familiar enough with the classes to identify the critical change. It would be great if you could figure this out someday

In order to get the plugin operators working I had to point to a directory where I placed a copy of the jar libraries. I don't know if it's possible to use the libs contained in the RM lib folder, but I didn't manage to use them by default. So I played around with the initialization properties I found in the wiki until it worked:

//System.setProperty("rapidminer.init.operators", "false"); //(file path)
System.setProperty("rapidminer.init.plugins", "true"); //(true or false)
System.setProperty("rapidminer.init.plugins.location", "C:\\Matthias\\Workspace\\Eclipse\\GUITest\\lib"); //(directory path)
System.setProperty("rapidminer.init.weka", "false"); //(true or false)
System.setProperty("rapidminer.init.jdbc.lib", "false"); //(true or false)
//System.setProperty("rapidminer.init.jdbc.lib.location", "false"); //(directory path)
System.setProperty("rapidminer.init.jdbc.classpath", "false"); //(true or false)

But don't ask me if this all makes sense

Best regards
Matthias

Marco_Boeck · July 2011

Hi Matthias,

I poked around a bit: setting the ExecutionMode to COMMAND_LINE should work. Using for example EMBEDDED_WITHOUT_UI will not work for plugin operators because otherwise RapidMiner would want to install them which would make no sense. However you can place the plugin jar files in the lib/plugin directory, then it should work.

Regards,
Marco

colo · July 2011

Hi Marco,

thanks for this update. I am currently moving away from generating operators from Java code and using predefined general processes instead. As you are recently posting in some other topics, creating a whole process via the API is not that comfortable at all.

Since I don't need dynamically built processes but just have to cover some fixed application patterns, I am fine with predefined XML declarations. Otherwise, I will play Phil's "Going Back" album while switching back to the old revision for now.

The only thing I did not test until now is how to connect two or more of those process patterns. I have to connect data collection with data analysis for example. For both tasks exist pools of 3-5 possible process patterns (processes completly designed with the GUI). After connecting them the only task remaining will be setting specific parameters for the relevant operators from Java, which should not cause so much troubles. If you have some hints on how to possibly connect multiple process files into one, I would be glad. If this is not possible with reasonable efforts, I will probably need to keep them as mutliple processes and define a data flow between them.

Best regards
Matthias

Marco_Boeck · July 2011

Hi,

1) I had to update my previous post.

2) You can create the individual processes in RapidMiner, and then create a main process to execute them one after another via the Execute Process operator. You just need to connect the input/output ports accordingly. That way, you can just execute the main process which in return executes the specified sub processes.

Regards,
Marco

colo · July 2011

Hi Marco,

it seems I won't get the process working again

I changed the ExecutionMode and tried possible variations of the system properties. Without any success. I removed all of the other operators except "Get Page" and still receive the same error message.

This is the entire code I am using:

import com.rapidminer.Process;
import com.rapidminer.RapidMiner;
import com.rapidminer.RapidMiner.ExecutionMode;
import com.rapidminer.operator.ExecutionUnit;
import com.rapidminer.operator.IOContainer;
import com.rapidminer.operator.Operator;
import com.rapidminer.operator.OperatorException;
import com.rapidminer.operator.io.web.GetWebpageOperator;
import com.rapidminer.tools.OperatorService;


public class ExecuteProcessTest {

	/**
	 * Connect the output-port <code>fromPortName</code> from Operator
	 * <code>from</code> with the input-port <code>toPortName</code> of Operator
	 * <code>to</code>.
	 */
	private static void connect(Operator from, String fromPortName,	Operator to, String toPortName) {
		from.getOutputPorts().getPortByName(fromPortName).connectTo(to.getInputPorts().getPortByName(toPortName));
	}

	/**
	 * Connect the output-port <code>fromPortName</code> from Subprocess
	 * <code>from</code> with the input-port <code>toPortName</code> of Operator
	 * <code>to</code>.
	 */

	private static void connect(ExecutionUnit from, String fromPortName, Operator to, String toPortName) {
		from.getInnerSources().getPortByName(fromPortName).connectTo(to.getInputPorts().getPortByName(toPortName));
	}
	/**
	 * Connect the output-port <code>fromPortName</code> from Operator
	 * <code>from</code> with the input-port <code>toPortName</code> of
	 * Subprocess <code>to</code>.
	 */
	private static void connect(Operator from, String fromPortName, ExecutionUnit to, String toPortName) {
		from.getOutputPorts().getPortByName(fromPortName).connectTo(to.getInnerSinks().getPortByName(toPortName));
	}

	/**
	 * Connect the output-port <code>fromPortName</code> from Subprocess
	 * <code>from</code> with the input-port <code>toPortName</code> of
	 * Subprocess <code>to</code>.
	 */
	private static void connect(ExecutionUnit from, String fromPortName, ExecutionUnit to, String toPortName) {
		from.getInnerSources().getPortByName(fromPortName).connectTo(to.getInnerSinks().getPortByName(toPortName));
	}


    private static Process createProcess(String url, String regex) {
    	//System.setProperty("rapidminer.init.operators", "false"); //(file path)
    	System.setProperty("rapidminer.init.plugins", "true"); //(true or false)
    	System.setProperty("rapidminer.init.plugins.location", "C:\\Matthias\\Workspace\\Eclipse\\GUITest\\lib"); //(directory path)
    	System.setProperty("rapidminer.init.weka", "false"); //(true or false)
    	System.setProperty("rapidminer.init.jdbc.lib", "false"); //(true or false)
    	//System.setProperty("rapidminer.init.jdbc.lib.location", "false"); //(directory path)
    	System.setProperty("rapidminer.init.jdbc.classpath", "false"); //(true or false)

		// invoke init before using the OperatorService
    	RapidMiner.setExecutionMode(ExecutionMode.COMMAND_LINE);
		RapidMiner.init();

		// create process
		Process process = new Process();
		try {
		    // create operator
		    Operator retrieveOperator = OperatorService.createOperator(GetWebpageOperator.class);
		    // set parameters
		    retrieveOperator.setParameter("url", url);
		    retrieveOperator.setParameter("random_user_agent", "true");
		    // add operator to process
		    process.getRootOperator().getSubprocess(0).addOperator(retrieveOperator);

		    // connect operators
		    connect(retrieveOperator, "output", process.getRootOperator().getSubprocess(0), "result 1");
		} catch (Exception e) { e.printStackTrace(); }
		return process;
    }

    public static IOContainer run(String url, String regex) {
    	// create process
		Process process = createProcess(url, regex);
		IOContainer processResult = null;

		try {
		    // perform process
		    processResult = process.run();
		} catch (OperatorException e) { e.printStackTrace(); }

		return processResult;
    }
    
    public static void main(String[] argv) {
		// create process
		Process process = createProcess("http://www.google.com", "<a href=\\\".*?\\\"[^>]*>(.*?)</a>");
		// print process setup
		System.out.println(process.getRootOperator().createProcessTree(0));

		try {
		    // perform process
		    IOContainer ioResult = process.run();
		    // to run the process with input created by your application use
	        // process.run(new IOContainer(new IOObject[] { ... your objects ... });
		} catch (OperatorException e) { e.printStackTrace(); }
    }
}

I don't know if something else is wrong, but it was working in the older revision without any problems...

Thank you for the simple solution of combining multiple process files by a new process using "Execute Process". This easiest way did not come to my mind... But there seems to be a major limitation for this solution. I will certainly not be able to set parameters for the operators from the embedded processes. But this is an important step before executing the comined process. Do you have any suggestions on this?

Best regards
Matthias

Edit: After thinking the whole thing over again, I assume it might become a bit untidy, but it would be a simple approach to use the suggested master process combining all process parts by using "Execute Process" operatores and set the parameters via macros. After setting the relevant parameters with some predefined macro names I will just have to deliver them with their values to the several process executions. Thanks again for providing this simple solution.

Marco_Boeck · July 2011

Hi,

do you have the web plugin jar in the appropriate folder (aka lib/plugins)? Otherwise RapidMiner will not be able to find it.

Regards,
Marco

colo · July 2011

Hi Marco,

omg - this was one thing I did not try. I tried to make the program use the RapidMiner lib files (RapidMiner_Vega\lib\plugins) or simply include the extension projects on the build path without success. The reference to RapidMiner worked fine, but the extensions had to be included via the respective jar files. Then I found the property rapidminer.init.plugins.location and specified the path to the lib files, that I copied from the above location to a local folder for my eclipse project. Then the extensions were found and everything worked. Then revision 313 was released and the program did not work anymore (without changing it in the meantime).
Now I moved the lib files from the local folder in my process from lib (as it was referenced in the example code above) to lib/plugins and suddenly everything is working again. The specified path for the plugin location using the property rapidminer.init.plugins.location seems to be ignored since 313 and the fixed location lib/plugins is used instead!? This idea would never have come to my mind. Is this some default location for Java projects that I missed all the time or was the choice of a different location removed with revision 313? However, the easier the solution the longer you have to search for it... Thanks a lot.
But perhaps you should investigate why this behaviour suddenly changed and if it was really intended...

Many thanks and best regards
Matthias

IngoRM · July 2011

Hi,

But perhaps you should investigate why this behaviour suddenly changed and if it was really intended...

yip, this indeed does not really seem to be intended but I might be mistaken...

colo · September 2011

Sorry for grabbing this old topic again, but I just experienced another thing that doesn't seem to be possible anymore.

I have to init RapidMiner using RapidMiner.init() in order to query the predefined database connection names. This always takes some seconds, so I tried to speed this up by disabling the extensions (and all of the exceptions thrown for IE plugin during initialization). I tried the following:

System.setProperty("rapidminer.init.plugins", "false");

without any success. Did the properties names change? I didn't check if setting the lib path is working again, but actually I don't think so...

Regards
Matthias

Marco_Boeck · September 2011

Hi,

it's not a System property, it's a RapidMiner parameter.
You may want to have a look at the ParameterService class:


ParameterService.getParameterValue(RapidMiner.PROPERTY_RAPIDMINER_INIT_PLUGINS);

This fragment is called by RapidMiner to know if plugins should be initialized.

Regards,
Marco

colo · September 2011

Hi Marco,

thank you, I will have a look at it.

The wiki says

You can also use the simple method RapidMiner.init() and configure the settings via this list of environment variables:

rapidminer.init.operators (file name)
rapidminer.init.plugins.location (directory name)
rapidminer.init.weka (boolean)
rapidminer.init.jdbc.lib (boolean)
rapidminer.init.jdbc.classpath (boolean)
rapidminer.init.plugins (boolean)

And things worked this way before the mentioned revision was released. If this isn't possible any longer, you should perhaps update the wiki page ( http://rapid-i.com/wiki/index.php?title=Integrating_RapidMiner_into_your_application ).

Update:
Setting the property doesn't seem to change anything. I tried

ParameterService.setParameterValue(RapidMiner.PROPERTY_RAPIDMINER_INIT_PLUGINS, String.valueOf(usePlugins));
RapidMiner.init();

with true and false, and the result is the same. All these exceptions (see below) from my beloved IE plugin shouldn't show up, if plugins are not initialized, right?

Sep 21, 2011 2:03:01 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Property rapidminer.home is not set. Guessing.
Sep 21, 2011 2:03:01 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Trying base directory of classes (build) 'C:\Matthias\Workspace\Eclipse\InformationExtractionFramework'...gotcha!
Sep 21, 2011 2:03:01 PM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
Sep 21, 2011 2:03:03 PM com.rapidminer.parameter.ParameterTypePassword decryptPassword
WARNING: Password in XML file looks like unencrypted plain text.
com.rapidminer.parameter.UndefinedParameterError: A value for the parameter 'operatorName' must be specified! 
	at com.rapidminer.parameter.Parameters.getParameter(Parameters.java:168)
	at com.rapidminer.operator.Operator.getParameter(Operator.java:1177)
	at com.rapidminer.operator.Operator.getParameterAsString(Operator.java:1195)
	at com.rapidminer.operator.preprocessing.ie.features.tools.RelationPreprocessOperatorImpl.toString(RelationPreprocessOperatorImpl.java:94)
	at com.rapidminer.operator.preprocessing.ie.features.tools.RelationPreprocessOperatorImpl.modifyMetaData(RelationPreprocessOperatorImpl.java:114)
	at com.rapidminer.operator.AbstractExampleSetProcessing$1.modifyMetaData(AbstractExampleSetProcessing.java:60)
	at com.rapidminer.operator.ports.metadata.PassThroughRule.transformMD(PassThroughRule.java:58)
	at com.rapidminer.operator.ports.metadata.MDTransformer.transformMetaData(MDTransformer.java:58)
	at com.rapidminer.operator.Operator.transformMetaData(Operator.java:2060)
	at com.rapidminer.tools.OperatorService.registerOperator(OperatorService.java:432)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:260)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:232)
	at com.rapidminer.tools.OperatorService.registerOperators(OperatorService.java:206)
	at com.rapidminer.tools.plugin.Plugin.registerOperators(Plugin.java:471)
	at com.rapidminer.tools.plugin.Plugin.registerAllPluginOperators(Plugin.java:725)
	at com.rapidminer.tools.OperatorService.init(OperatorService.java:167)
	at com.rapidminer.RapidMiner.init(RapidMiner.java:465)
	at core.rapidminer.process.ProcessSettings.initRapidMiner(ProcessSettings.java:41)
	at tests.MiniTests.main(MiniTests.java:19)

...

com.rapidminer.parameter.UndefinedParameterError: A value for the parameter 'operatorName' must be specified! 
	at com.rapidminer.parameter.Parameters.getParameter(Parameters.java:168)
	at com.rapidminer.operator.Operator.getParameter(Operator.java:1177)
	at com.rapidminer.operator.Operator.getParameterAsString(Operator.java:1195)
	at com.rapidminer.operator.preprocessing.ie.features.tools.RelationPreprocessOperatorImpl.toString(RelationPreprocessOperatorImpl.java:94)
	at com.rapidminer.operator.preprocessing.ie.features.tools.RelationPreprocessOperatorImpl.modifyMetaData(RelationPreprocessOperatorImpl.java:114)
	at com.rapidminer.operator.AbstractExampleSetProcessing$1.modifyMetaData(AbstractExampleSetProcessing.java:60)
	at com.rapidminer.operator.ports.metadata.PassThroughRule.transformMD(PassThroughRule.java:58)
	at com.rapidminer.operator.ports.metadata.MDTransformer.transformMetaData(MDTransformer.java:58)
	at com.rapidminer.operator.Operator.transformMetaData(Operator.java:2060)
	at com.rapidminer.tools.OperatorService.registerOperator(OperatorService.java:432)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:260)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:232)
	at com.rapidminer.tools.OperatorService.registerOperators(OperatorService.java:206)
	at com.rapidminer.tools.plugin.Plugin.registerOperators(Plugin.java:471)
	at com.rapidminer.tools.plugin.Plugin.registerAllPluginOperators(Plugin.java:725)
	at com.rapidminer.tools.OperatorService.init(OperatorService.java:167)
	at com.rapidminer.RapidMiner.init(RapidMiner.java:465)
	at core.rapidminer.process.ProcessSettings.initRapidMiner(ProcessSettings.java:41)
	at tests.MiniTests.main(MiniTests.java:19)
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.OperatorService parseOperators
WARNING: Cannot create operator description: rmx_informationExtraction:binary_kernel_naiveBayes_operator
java.lang.ClassNotFoundException: com.rapidminer.operator.learner.bayes.BinaryTreeKernelNaiveBayes
	at java.net.URLClassLoader$1.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(Unknown Source)
	at com.rapidminer.tools.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:102)
	at java.lang.ClassLoader.loadClass(Unknown Source)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Unknown Source)
	at com.rapidminer.operator.OperatorDescription.<init>(OperatorDescription.java:93)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:259)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:256)
	at com.rapidminer.tools.OperatorService.parseOperators(OperatorService.java:232)
	at com.rapidminer.tools.OperatorService.registerOperators(OperatorService.java:206)
	at com.rapidminer.tools.plugin.Plugin.registerOperators(Plugin.java:471)
	at com.rapidminer.tools.plugin.Plugin.registerAllPluginOperators(Plugin.java:725)
	at com.rapidminer.tools.OperatorService.init(OperatorService.java:167)
	at com.rapidminer.RapidMiner.init(RapidMiner.java:465)
	at core.rapidminer.process.ProcessSettings.initRapidMiner(ProcessSettings.java:41)
	at tests.MiniTests.main(MiniTests.java:19)
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.plugin.Plugin registerOperators
INFO: No operator descriptor specified for plugin Community. Trying plugin initializtation class com.rapidminer.community.CommunityPluginInit.
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.plugin.Plugin registerOperators
WARNING: No operator descriptor defined for: Community
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.jdbc.JDBCProperties <init>
WARNING: Missing database driver class name for 'ODBC Bridge (e.g. Access)'
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver ca.ingres.jdbc.IngresDriver not found. Probably the driver is not installed.
Sep 21, 2011 2:03:03 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver oracle.jdbc.driver.OracleDriver not found. Probably the driver is not installed.

Regards
Matthias

Marco_Boeck · September 2011

Hi,

After a quick glance at the init() method, it seems like RapidMiner will override any parameters you set prior to init() with the settings from the settings file. So you will need to either set the property "rapidminer.init.plugins=false" in the config file found in the .RapidMiner5 folder (located in the user folder), or you could try and call


ParameterService.init();
ParameterService.setParameterValue(RapidMiner.PROPERTY_RAPIDMINER_INIT_PLUGINS, String.valueOf(usePlugins));
RapidMiner.init();

This should load all settings first, then you can edit them, and then you can init RM.

Regards,
Marco

colo · September 2011

Uhm, why did the idea of looking there not come to my own mind? :-X

Thanks a lot, this works fine.

Best regards
Matthias

"Simple java program"

Answers

Categories