Implementation of Random Forest (versus Decision Forest?)

ollestrat

Hi all,

I'm wondering how the RapidMiner RandomForest classifier is implemented. It seems to me that there are significant differences to the version of Breiman (BREIMAN, L: Random Forests Machine Learning, 45, 5–32, 2001).

Main features of Random Forests are:
- each tree grows on his individual bootstrap sample set
- at each node of the trees, a defined number of features is randomly selected and evaluated for the best split

Is the RapidMiner RandomForest classifier working like that? Are individual trees grown on bootstrap samples? And  I suppose the number of features is rather determined for the hole tree, not for each node (?). If so it would rather resemble the "Decision Forest" of Ho ( Ho, T.K. 1998: The Random Subspace Method for Constructing Decision Forests. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 8, AUGUST 1998).

The WEKA-version of the Random Forest classifier seems to follow Breiman's concept I guess, so this could be the choice anyway, however: the "Weight by Tree Importance"-operator which I would like to use does not work with the WEKA-version.

Thanks in advance.

Ollestrat

ollestrat

one additional comment: I had a look at the java code of the RandomForest Learner:

bootstrapping.setParameter(BootstrappingOperator.PARAMETER_SAMPLE_RATIO, "1.0")

it seems there is bootrapping implemented, however the sample ratio is set to 1.0 and the RandomForest operator does not offer the according parameter to adjust this ratio. Does this mean that all trees are always grown on the whole example set (thus no bootstrapping)?

Would be great to get some help  as I am completely lost on the code level.

public class RandomForestLearner extends RandomTreeLearner {

	/** The parameter name for the number of trees. */
	public static final String PARAMETER_NUMBER_OF_TREES = "number_of_trees";

	public RandomForestLearner(OperatorDescription description) {
		super(description);
	}

@Override
	public Class<? extends PredictionModel> getModelClass() {
		return RandomForestModel.class;
	}

@Override
	public Model learn(ExampleSet exampleSet) throws OperatorException {
		BootstrappingOperator bootstrapping = null;
		try {
			bootstrapping = OperatorService.createOperator(BootstrappingOperator.class);
			bootstrapping.setParameter(BootstrappingOperator.PARAMETER_USE_WEIGHTS, "false");
			bootstrapping.setParameter(BootstrappingOperator.PARAMETER_SAMPLE_RATIO, "1.0");
		} catch (OperatorCreationException e) {
			throw new OperatorException(getName() + ": cannot construct random tree learner: " + e.getMessage());
		}

		// learn base models
		List<TreeModel> baseModels = new LinkedList<TreeModel>();
		int numberOfTrees = getParameterAsInt(PARAMETER_NUMBER_OF_TREES);

		for (int i = 0; i < numberOfTrees; i++) {
			TreeModel model = (TreeModel)super.learn(bootstrapping.apply(exampleSet));
			model.setSource(getName());
			baseModels.add(model);
		}

		// create and return model
		return new RandomForestModel(exampleSet, baseModels);
	}