"Minimal gain - Decision Trees"

Madcap
New Altair Community Member
Could someone help explain to me minimal gain for decision tree pre-pruning.
How is the gain of a node quantified?
Thanks for any help
-Madcap
How is the gain of a node quantified?
Thanks for any help
-Madcap
Tagged:
0
Best Answer
-
Hello @Madcap
Gain is calculated based on entropy. Gain will give us the amount of information provided by a splitting a feature.
In case of pre-pruning, we will not build the whole tree as we will start pruning (removing split) while the tree is being built
From my understanding, the minimal gain is where we are setting a cut point (threshold). For example, if the calculated gain for a particular feature split is greater than the set minimal gain then there will be a split else it will just leave that feature as a leaf.
3
Answers
-
Hello @Madcap
Gain is calculated based on entropy. Gain will give us the amount of information provided by a splitting a feature.
In case of pre-pruning, we will not build the whole tree as we will start pruning (removing split) while the tree is being built
From my understanding, the minimal gain is where we are setting a cut point (threshold). For example, if the calculated gain for a particular feature split is greater than the set minimal gain then there will be a split else it will just leave that feature as a leaf.
3