Random forest basic question

Fred12

hi,

I a paper about general algorithm of random forest, it is said on each node, a subset of variables is chosen to test.

my Question: When its constructing the tree with the training set, will there already be made the choice of random variables to built the tree? or will it be trained with all variables, and afterwards, for testing only a small subset of variables will be chosen to test the tree on OOB data?

 

and finally, do the subsets  always have to be nonoverlapping, means distinct from eachother? or is it entirely randomly how to choose variables? so that there can also be repeated variables  (more than one of the same) to test each node?

IngoRM

Hi,

 

The random subsets are chosen during the training phase already.  The point is to "force" the different trees to cover different aspects of the data space and problem to learn.  They become somewhat weaker through that which is encounted by building the ensemble of trees.

 

Typically (also in case of RapidMiner) the subsets are completely random, i.e. overlapping can occur.

 

Best,

Ingo