Gradient Boosted Trees: stopped before building all trees

phivu

Hi all,

 

I'm using Gradient Boosted Trees to build a binary classifier with ~500 features. I want to increase the number of trees during training to get better feature weights, however, if i set the number of trees > 1000, the training stops before building all trees with no reason (as in the log panel - sceenshot attached - it stopped at 43%). This is not the memory problem as i noticed the memory usage is only 27%. I didn't tick the "early stopping" checkbox.

 

Do you know the true reason behind this, or is there another way to increase the reliability of feature weights? I attached the process here, let me know if you want the data.

Thank you very much for your help!

 

Best,
phivu

MartinLiebig

Dear Phivu,

 

this looks a bit odd. Can you check your rapidminer-studio.log for more information?

 

On the DS topic: More tries does not increase the reliablity of your weights. GBTs are not bagged, so it's not like RF's. You might simply overtrain more. Than can be the reason why your tree stops. The classes are already seperated.

 

~Martin