Operator IteratingPerformanceAverage

Legacy User

Hi,

I'm trying to figure out the meaning of the IteratingPerformanceAverage
operator used in sample 07_EvolutionaryParameterOptimization.

To my understanding this operator should perform the Xvalidation
(in case of the sample a genetic algorithm to find good
SVM parameters) as often as defined by its parameter "iterations".

However, when I set the value to 1, I get 44 entries (I assume this
is the number of applications to LibSVMLearner with different
parameters found by the genetic algorithm) in the logfile generated
by the ProcessLog operator. When I increase the number of "iterations"
to 2, I still have 44 entries in the logfile. Finally, with 3 iterations
I get 83 entries. I have no idea why. I would expect that increasing
the iterations from 1 to 2 would double the number of applications
of the learner, thus also double the number of entries in the logfile.

BTW, I didn't fully catch why you use the IteratingPerformanceAverage
in this sample at all. With EvolutionaryParameterOptimization you
generate a population that is improved as long as defined by
the parameter "generations_without_improval". With the
IteratingPerformanceAverage you then measure the fitness of each
individual (using the Xvalidation) multiple times (as often
as defined by the parameter "iterations")? Does this make sense?

Regards,
Tim

land

Hi Tim,
the number of entries in the log will be determined by the evolutionary parameter optimization.
The IteratingPerformanceAverage is used in this example to make the estimation of the performance more robust. Since XValidation is only an estimate of the real performance depending on the random partition of data, its more robust against randomly drawn very good or very bad partitions to perform it a number of times and average.

Greetings,
  Sebastian