EvolutionaryParameterOptimisation & Process Log

haddock

Hi Folks,

I'm using the EvolutionaryParameterOptimization operator to tune LibSVM, and want to log the results, but I hit a "null pointer" problem. You can replicate it if you replace the GridParameterOptimization operator with an EvolutionaryParameterOptimization operator in the 05_ParameterValuePlot example, like so...

<operator name="Root" class="Process" expanded="yes">
    <description text="#ylt#p#ygt# This experiment is very similar to the parameter optimization experiment in the meta sample directory. Again we use an ProcessLog operator which collects all desired values each time it is applied. In this experiment, two of the three values are parameters (the parameters C and degree of a polynomial kernel SVM) and the third value is the prediction performance which was achieved with these parameters. Please check out the parameter list of the Log operator. Instead of #yquot#value#yquot# the key #yquot#parameter#yquot# was used for two of the logging values. #ylt#/p#ygt# #ylt#p#ygt# The result is a three dimensional plot of the error against the parameters which again can be checked online during running of the experiment. This time you should set x axis to C, y axis to degree and the value from the selection list should be the calculated error. #ylt#/p#ygt#"/>
    <operator name="Input" class="ExampleSource">
        <parameter key="attributes"	value="../data/polynomial.aml"/>
    </operator>
    <operator name="EvolutionaryParameterOptimization" class="EvolutionaryParameterOptimization" expanded="yes">
        <list key="parameters">
          <parameter key="Training.degree"	value="[1.0;5.0]"/>
          <parameter key="Training.C"	value="[0.0;100.0]"/>
        </list>
        <operator name="Validation" class="XValidation" expanded="yes">
            <parameter key="sampling_type"	value="shuffled sampling"/>
            <operator name="Training" class="LibSVMLearner">
                <parameter key="C"	value="93.6637279685964"/>
                <list key="class_weights">
                </list>
                <parameter key="epsilon"	value="0.01"/>
                <parameter key="kernel_type"	value="poly"/>
                <parameter key="svm_type"	value="epsilon-SVR"/>
            </operator>
            <operator name="ApplierChain" class="OperatorChain" expanded="yes">
                <operator name="Test" class="ModelApplier">
                    <list key="application_parameters">
                    </list>
                </operator>
                <operator name="Evaluation" class="RegressionPerformance">
                    <parameter key="absolute_error"	value="true"/>
                    <parameter key="normalized_absolute_error"	value="true"/>
                    <parameter key="squared_error"	value="true"/>
                </operator>
            </operator>
        </operator>
        <operator name="Log" class="ProcessLog">
            <list key="log">
              <parameter key="C"	value="operator.Training.parameter.C"/>
              <parameter key="degree"	value="operator.Training.parameter.degree"/>
              <parameter key="absolute"	value="operator.Validation.value.performance"/>
            </list>
        </operator>
    </operator>
</operator>

Anybody got a solution to this one?

IngoRM

Hi Haddock,

this is a bug introduced into the EvolutionaryParameterOptimization in version 4.2. The optimation works correctly but unfortunately the performance values of the inner validation operators cannot be logged. We have already fixed this bug in the latest CVS release (and the fix will hence of course also be part of the next release) and also for the latest update of the Enterprise Edition.

Cheers,
Ingo