Optimize Parameter/Log for Decision Tree

Prentice

Hello,

I'm trying to compare different algorithms and try to find the optimal parameters. For this, I use the optimize parameter operator. However, when I try to apply it to a Decision Tree classifier it doesn't work. Or at least, the log function doesn't work properly, but I also get the same accuracy for each setting. 
But when I want to plot it in a log function it only shows one value for each parameter.

Does somebody know why this happens and how I can solve it?

Thanks
Prentice

jczogalla

Hi @Prentice!

That sounds strange indeed. I will look into this to see if this is a potential bug.

Edit:
I looked at your process and I think I see the culprit. In the Log operator, you reference the Decision Tree operator, not Decision Tree (2), which is used inside the cross validation, as well as it being referenced in the optimize parameters. Also, there should be a log called Optimize Parameters (Grid), which should already have the parameters and performance as a log. I hope this helps!

Cheers

Jan

sgenzer

hi @Prentice so I looked at your process and if I put a breakpoint right before Optimize Parameters, I see an ExampleSet with SIX examples. There is no way you are going to build a machine learning model with such a small data set. This is why your "accuracy" is always the same, etc..

Scott

Prentice

Hello @sgenzer,

Aah, so this means that my actual ExampleSet is also too small? I've got like 150 examples with 4 classes, that does explain it. It just goes each time to the same branch, hence having the same confidence.
But why does the Log operator not work when I try to use different settings for the Decision Tree? It shows the combinations in the Optimize Parameters results, but it shows the same combinations for each iteration for the results of Log. This only happens when I use the Decision Tree.

Prentice

jczogalla

Hi @Prentice!

That sounds strange indeed. I will look into this to see if this is a potential bug.

Edit:
I looked at your process and I think I see the culprit. In the Log operator, you reference the Decision Tree operator, not Decision Tree (2), which is used inside the cross validation, as well as it being referenced in the optimize parameters. Also, there should be a log called Optimize Parameters (Grid), which should already have the parameters and performance as a log. I hope this helps!

Cheers

Jan

Prentice

@jczogalla

Wow thanks, sometimes the most simplest of things get overlooked. I can't believe I spent hours trying find the cause and it's something like this haha. 
Anyway, thanks for your help!

Prentice