Incremental Results in Optimize Parameters?

srt19170

I have an Optimize Parameters process with 80 features that runs a long time and invariably crashes with memory problems.  Is there a way to have the process incrementally report the best solution, so that I could (for example) restart using the last reported solution?  Thanks for any help!

-- Scott Turner

homburg

Hi Scott.

There are some options to solve your problem. At first you may try allocate more java heap memory by using the well known -Xmx switch. You may also try to reduce the complexity of your process. Finally (and that's what you asked for..) you may use the Log operator to enable incremental logging. Just drag a Log operator instance into your optimization operator and define which values you want to log. Very important: do not forget to enable the persistent check box. This ensures that the file will be written immediately after each optimization step. Here is an example of how such a process may look like:

<process version="5.1.009">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.1.009" expanded="true" name="Root">  
    <process expanded="true" height="584" width="962">
      <operator activated="true" class="retrieve" compatibility="5.1.009" expanded="true" height="60" name="Retrieve" width="90" x="45" y="30">
        <parameter key="repository_entry" value="../../data/Polynomial"/>
      </operator>
      <operator activated="true" class="optimize_parameters_grid" compatibility="5.1.009" expanded="true" height="94" name="ParameterOptimization" width="90" x="246" y="30">
        <list key="parameters">
          <parameter key="Training.C" value="50,100,150,200,250"/>
          <parameter key="Training.degree" value="1,2,3,4,5"/>
        </list>
        <process expanded="true" height="626" width="806">
          <operator activated="true" class="x_validation" compatibility="5.1.009" expanded="true" height="112" name="Validation" width="90" x="45" y="30">
            <parameter key="sampling_type" value="shuffled sampling"/>
            <process expanded="true" height="272" width="334">
              <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.1.009" expanded="true" height="76" name="Training" width="90" x="126" y="30">
                <parameter key="svm_type" value="epsilon-SVR"/>
                <parameter key="kernel_type" value="poly"/>
                <parameter key="degree" value="5"/>
                <parameter key="C" value="250"/>
                <parameter key="epsilon" value="0.01"/>
                <list key="class_weights"/>
              </operator>
              <connect from_port="training" to_op="Training" to_port="training set"/>
              <connect from_op="Training" from_port="model" to_port="model"/>
              <portSpacing port="source_training" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true" height="272" width="334">
              <operator activated="true" class="apply_model" compatibility="5.1.009" expanded="true" height="76" name="Test" width="90" x="45" y="30">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance_regression" compatibility="5.1.009" expanded="true" height="76" name="Evaluation" width="90" x="194" y="30">
                <parameter key="absolute_error" value="true"/>
                <parameter key="normalized_absolute_error" value="true"/>
                <parameter key="squared_error" value="true"/>
              </operator>
              <connect from_port="model" to_op="Test" to_port="model"/>
              <connect from_port="test set" to_op="Test" to_port="unlabelled data"/>
              <connect from_op="Test" from_port="labelled data" to_op="Evaluation" to_port="labelled data"/>
              <connect from_op="Evaluation" from_port="performance" to_port="averagable 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_averagable 1" spacing="0"/>
              <portSpacing port="sink_averagable 2" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="log" compatibility="5.1.009" expanded="true" height="76" name="Log" width="90" x="246" y="75">
            <parameter key="filename" value="paraopt.log"/>
            <list key="log">
              <parameter key="C" value="operator.Training.parameter.C"/>
              <parameter key="degree" value="operator.Training.parameter.degree"/>
              <parameter key="absolute" value="operator.Validation.value.performance"/>
            </list>
            <parameter key="persistent" value="true"/>
          </operator>
          <connect from_port="input 1" to_op="Validation" to_port="training"/>
          <connect from_op="Validation" from_port="averagable 1" to_op="Log" to_port="through 1"/>
          <connect from_op="Log" from_port="through 1" to_port="performance"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_performance" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Retrieve" from_port="output" to_op="ParameterOptimization" to_port="input 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

Cheers,
   Helge