How do you reduce variables before doing a decision tree?

Matt_Pilk

Hi!  
Just wanted some help.
1) do you need to reduce the number of variables before you execute a decision tree analysis? Currently, i have 19.  It makes the decision tree hard to read as i need to go to 12 layers to get the accuracy up.

2) If I use the select attributes for the ones i believe are important after doing some EDA, does this dilute the results or can you pass the decision tree through the original set of data?

any insights from the community would be great.

Thanks,
Matthew

BalazsBaranyRM

Hi!

Decision tree based methods are all about selecting relevant attributes. If you remove attributes before and the tree changes, these would have been relevant and your tree probably got worse. If the tree doesn't change, the removal actually found irrelevant attributes, but that would have been the case through the decision tree anyway.

It is a good idea to do an assessment of your attributes and check them for harmful things like "future" knowledge leaking into the model, or data that are hard to get, or attributes having many missing values. You could remove these manually. But you shouldn't remove attributes on the basis "I don't think these are relevant" before using any method that selects or weights attributes itself. That would be "part human, part machine learning" and it is hard to get better results from this process than from an algorithm that was written for this task.

If your decision tree is too hard to interpret AND interpretability is a more important goal than accuracy, it's better to change the pruning parameters to more strict values. That will give you a smaller, better to understand tree, without sacrificing relevant attributes before its application.

Regards,
Balázs