🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.
Join and Win

forward selection and backward elimination

User: "AD2019"
New Altair Community Member
Updated by Jocelyn
I ran a multiple regression model on a dataset having 15 variables first using the "forward selection" nested operator, and then using the "backward elimination" nested operator.  I got dramatically different models.  the first had 3 independent variables, the second had 8  IVs.  why such a bid difference.  I realize the serial elimination or addition of IVs may yield local optima, but is it common to get such wildly different "optimal" models for the same dataset?  How can training yield such dramatically different trained models?
thanks in advance,
AD

Find more posts tagged with

Sort by:
1 - 2 of 21
    User: "varunm1"
    New Altair Community Member
    Accepted Answer
    Hello @AD2019

    Yes, you can get highly varying results for both types of selections. The reason is the parameter settings in these operators. If you used a forward selection, and the operator is adding each variable one by one based on the improvement and then it finds there is no improvement (Stuck in local optima) then it stops. The number of speculative rounds (helps avoid local optima) helps you bypass stopping based on one decision round. Stopping behavior and a maximal number of selections and eliminations also decides the number of attributes available after selection.



    Below is my dataset that has 408 attributes where forward selection selected 8 attributes and backward elimination selected 401 attributes with default settings.
     


    Hope this helps. Please inform if you need more information.
    User: "varunm1"
    New Altair Community Member
    Accepted Answer
    Updated by varunm1
    The backward elimination algorithm does not have this option and my suspicion is that it is getting stuck in a local optimum
    @AD2019
    Can you check again? Backward elimination also has this option.