Why doesn't Auto-Model use Parameter Optimization for Deep Learning models?

pblack476

I opened up a process generated by automodel (Deep Learning with AFE) as I wanted to use it as a starting point and I saw that in the Auto FE step and the train step it does not use the Optmize Parameters operator for optimization. Why is that? I had a similar model with SVM and it did use that operator on those steps.

How does the Parameter Optimization on DL H20 models happens after all?

IngoRM

Yes, indeed.  DL is not fast exactly and keep in mind that next to the normal parameters you would also need to optimize network architecture, activation functions etc.  We will work on an "Auto Deep Learning" solution which will deal with this specifically but in "normal" Auto Model we made the compromise between speed and decent results (if not optimal).

Hope this helps,
Ingo

varunm1

Hello @pblack476

My understanding is the complexity of optimizing deep learning model hyperparamters, it also might suffer from long run times. 

Lets see @IngoRM take on this.

IngoRM

Yes, indeed.  DL is not fast exactly and keep in mind that next to the normal parameters you would also need to optimize network architecture, activation functions etc.  We will work on an "Auto Deep Learning" solution which will deal with this specifically but in "normal" Auto Model we made the compromise between speed and decent results (if not optimal).

Hope this helps,
Ingo