"Minimal gain - Decision Trees"

Madcap
Madcap New Altair Community Member
edited November 2024 in Community Q&A
Could someone help explain to me minimal gain for decision tree pre-pruning. 
How is the gain of a node quantified?

Thanks for any help
-Madcap

Best Answer

  • varunm1
    varunm1 New Altair Community Member
    edited March 2019 Answer ✓
    Hello @Madcap

    Gain is calculated based on entropy. Gain will give us the amount of information provided by a splitting a feature.

    In case of pre-pruning, we will not build the whole tree as we will start pruning (removing split) while the tree is being built 

    From my understanding, the minimal gain is where we are setting a cut point (threshold). For example, if the calculated gain for a particular feature split is greater than the set minimal gain then there will be a split else it will just leave that feature as a leaf.

Answers

  • varunm1
    varunm1 New Altair Community Member
    edited March 2019 Answer ✓
    Hello @Madcap

    Gain is calculated based on entropy. Gain will give us the amount of information provided by a splitting a feature.

    In case of pre-pruning, we will not build the whole tree as we will start pruning (removing split) while the tree is being built 

    From my understanding, the minimal gain is where we are setting a cut point (threshold). For example, if the calculated gain for a particular feature split is greater than the set minimal gain then there will be a split else it will just leave that feature as a leaf.