ParameterOptimization of W-SMOreg kernel parameters

ravelite

Greetings!

I was recently working with SVMs for regression, and working specifically with the Weka class SMOreg. (somehow I haven't yet gotten the native SVRs to perform comparably...).

To properly select parameters for the regression (complexity, epsilon-width, and especially kernal parameters), I tried using the EvolutionaryParameterOptimizer. However, since the kernel parameters are contained in a string, they don't seem to show up in the parameter settings dialog for the Optimizer.

Is there a way to add the kernel parameters to this optimization?

cheers,

Graham

land

Hi Graham,
this is not possible, but you simply could wrapp the Evolutionary Parameteroptimization with an outer GridParameterOptimization testing all three possible values.
Non-Continoous values cannot be optimized properly by an evolutionary approach.

Greetings,
  Sebastian

ravelite

Sebastian,

Thanks for your reply.

The parameter that I would wish to add to the optimization (for example, gamma from the RBFKernel function) is indeed a continuous parameter. It just doesn't show up because the kernel options for that learner (W-SMOreg) are represented all together as a string. But the option itself is continuous (say 1e-4 to 10?) meaning basically the size of the blob over each support vector.

best,

Graham

land

Hi Graham,
thanks to  the weka guys' nearly cryptographic formulation of parameters this is not possible.
But you should get similar performance if you use the LibSVM with svm_type epsilon_SVR or nu_SVR. As far as I know the SMOreg just uses another optimization algorithm but the same model. This should result in at least an equal performance with proper parameter settings.


Greetings,
  Sebastian

Legacy User

Sebastian,

Again, thanks for your explanation.

Yet I haven't been able to get comparable results on SV Regression using the LibSVM models. I'm not sure what the problem is exactly yet.

Perhaps I have a parameter miss-set. I'll try to open a new thread comparing my results with the libraries, since theoretically you would be using the same model (and thus could expect the same result) in both libraries.

Graham