Unsure-aware Decision Tree

alfileres

Hi All,

I'm trying to develop a stock recommending system using RapidMiner. I have labeled the output in two possible classes: buy / sell. I'm using a decision tree as learner. I have observed that some of the leaves contain both buy / sell recommendations. However, I will be more happy if the Decision Tree would say 'I do not know' your output for this data instead of giving the majority class for this specific leave.

Propose feature:
- Be able to define a parameter to specify the required purity in a leave node (e.g. purity=0.9 could mean that the leave says 'sell' and out of the total instances that end up in this node, 90% at least are 'sell').
- This definition of purity does not depend in how many classes the problem has; so, in principle, it can be quite generic.
- I find it also interesting to extend this approach to other operators if possible.

Let me know what you think about this feature request and possible work-arounds.

Thank you,

alfileres

TobiasMalbrecht

Hi,

alfileres wrote:
Let me know what you think about this feature request and possible work-arounds.

to imply a kind of uncertainty in the model is not really possible. Nevertheless, if it comes to prediction, such uncertainty is indicated by a small confidence. The operator [tt]UncertainPredictionsTransformation[/tt] allows to mark predictions which have a confidence below a specified thresholds as uncertain which is indicated by setting the corresponding prediction to missing.

The following process should explain this:

<operator name="Root" class="Process" expanded="yes">
    <operator name="ExampleSetGenerator" class="ExampleSetGenerator">
        <parameter key="target_function"	value="polynomial classification"/>
    </operator>
    <operator name="DecisionTree" class="DecisionTree">
        <parameter key="keep_example_set"	value="true"/>
    </operator>
    <operator name="ModelApplier" class="ModelApplier">
        <list key="application_parameters">
        </list>
    </operator>
    <operator name="UncertainPredictionsTransformation" class="UncertainPredictionsTransformation">
        <parameter key="min_confidence"	value="0.9"/>
    </operator>
</operator>

Maybe this is helpful for now. Concerning the decision tree uncertainty I am not sure whether it is of great help to add such an uncertainty indicator in the models itself, as in a descriptive setting this model can easily be interpreted by the user. A purity parameter might potentially be helpful to control fitting of the tree, but at the moment we are really busy and have really few time. But maybe we will discuss that issue in the long term. For now you have to use the other parameter to control how far the tree tries to generalize or to exactly fit the data.

Regards,
Tobias