Optimize Parameters Grid Quadratic Evolutionary

CharlieFirpo

Hi all!

Can anyone tell me what are the differneces between Grid, Quadratic and Evolutionary paramterer optimization operators? How they works? Any help or white paper? It's not too informative: "Evolutionary ... often more appropriate than a grid search". Why?

What are the cases and key points when one or the other operator should be used?

Thank you!

MariusHelf

In the grid search you specify a number of values for each parameter you want to optimize, and the operator tries out all possible combinations of these values. If you want to optimize only a few parameters, this should be your first choice.

The Evolutionary Parameter Optimization uses an evolutionary or genetic approach. A google search will probably spit out a number of sites explaining this concept. In RapidMiner you can choose this algorithm if you have a large number of parameters that you need to optimize.

Best regards,
Marius

CharlieFirpo

Thank you!

I'm starting to understand the grid and the evolutionary algorithms. But what about quadratic? Does it try all the combinations? In what order? What does 'region' stands for in 'if exceeds region' parameter?

MariusHelf

Optimize Parameters (Quadratic) applies a linear optimization onto the problem. The implementation is similar to Newton's Method.
As a sidenote, even though the dialog allows you to define stepsizes etc., they are ignored by the algorithm - it only uses min and max values for each parameter.

The operator could be used to optimize C and gamma for the SVM, but it tends to get caught in local optima instead of finding the global optimum.

Best regards,
Marius