"SVM Runtime"

Ghostrider

How long would it take an SVM to train on a 12 input, 1000 or 2000 row dataset?  How about a Neural Network?  Would the neural network train faster or slower than SVM?  Assume that it would be running on a modern quad-core i7.  I don't have data yet, just an idea...wondering if it's feasible.  By the way, are there any plans for RapidMiner to support OpenCL or GPU computing?

haddock

No need to wait for the data - you can generate it and test for yourself, like this..

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.1.001" expanded="true" name="Process">
    <process expanded="true" height="273" width="872">
      <operator activated="true" class="generate_data" compatibility="5.1.001" expanded="true" height="60" name="Generate Data" width="90" x="6" y="75">
        <parameter key="target_function" value="complicated function"/>
        <parameter key="number_examples" value="20000"/>
        <parameter key="number_of_attributes" value="12"/>
      </operator>
      <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.1.001" expanded="true" height="76" name="SVM" width="90" x="265" y="72">
        <parameter key="svm_type" value="epsilon-SVR"/>
        <list key="class_weights"/>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="SVM" to_port="training set"/>
      <connect from_op="SVM" from_port="model" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

Over to Ingo on the GPU front ...

Ghostrider

Thanks haddock.  I forgot about the automatic data generation routine.  Honestly, I've never used one before, but can see how it's helpful for generating data quickly.  The SVM took about 1 second to train with an input of 1000 rows, 12 cols.  NN took 5 seconds.  I don't know how good the models generated are (well, for this example, probably not good since data was a complicated function).  Anyhow, I'm trying to assess quickly how well my inputs predict my target output values.  I don't care which learning algorithm is used and I don't even need a model, I just want to know as quickly as possible how my chosen set of inputs predict (though some complex combination / weighting, possibly non-linear) the output.  I just want to know roughly how well my inputs predict the target output through SVM, Neural Net, etc.  Anyone know of a good way to go about this?  I was leaning towards SVM since I heard it was a bit easier to train than NN...with Neural Nets, you have to guess the right topology (or use NEAT) in addition to training the synapse weights.  Since SVM has no local minimums, it would seem that it would converge to the optimal result faster.

Also, I think some JAR libraries that RapidMiner uses already supports OpenCL.  I know Encog which is used for nerual nets in RM does.

rakirk

With 12 attributes, try logistic regression also.

I was curious if you made any progress on the use of GPU with RM. I had previously tried this myself but didn't make any progress and then ran out of time to tinker.