"XYValidation Model Profiling - Lift and Variable importance"

alvinc

Hi all,

I would like to profile a cross-fold validation model using lift chart and variable importance. I haven't had luck figuring how to do so and hope I may get some help on the following -

- How do I generate betas coefficients or variable influence for algorithms such as SVM, similar to the betas in Logistic Regression?
- Is it possible to output standardized betas in Logistic Regression and other algorithms?
- How do I use the lift chart node? I haven't had success placing the lift chart node after model applier node, a chart pops up with the message : "No data points available (yet)"
- Is it possible to output a decile lift table?

Thanks in advance for your guidance!

Kind regards,
Alvin

IngoRM

Hi,

How do I generate betas coefficients or variable influence for algorithms such as SVM, similar to the betas in Logistic Regression?

As far as I know thereare no betas for SVM. At least I have never noticed somebody has calculated something like that. And about variable influence for SVM: the coefficients for the hyperplane (w-values) can directly be read as measure of importance. Beside that you could have a look into the group "Preprocessing" - "Attributes" - "Weighting" which contain about 20 different variable influence calculations including those for SVM, info gain for trees, Relief etc.

Is it possible to output standardized betas in Logistic Regression and other algorithms?

They already are output of the logistic regression? And as far as I know, it is not possible to calculate those "beta"-values for other algorithms. But if you are able to explain the algorithm we could of course integrate something like that.

How do I use the lift chart node? I haven't had success placing the lift chart node after model applier node, a chart pops up with the message : "No data points available (yet)"

Try it before. Here is a setup inside of a cross validation:

<operator name="Root" class="Process" expanded="yes">
    <operator name="ExampleSetGenerator" class="ExampleSetGenerator">
        <parameter key="target_function"	value="sum classification"/>
    </operator>
    <operator name="XValidation" class="XValidation" expanded="yes">
        <operator name="MyKLRLearner" class="MyKLRLearner">
        </operator>
        <operator name="OperatorChain" class="OperatorChain" expanded="yes">
            <operator name="LiftChart" class="LiftChart">
            </operator>
            <operator name="ModelApplier" class="ModelApplier">
                <list key="application_parameters">
                </list>
                <parameter key="keep_model"	value="true"/>
            </operator>
            <operator name="Performance" class="Performance">
            </operator>
        </operator>
    </operator>
</operator>

Is it possible to output a decile lift table?

Currently not, sorry.

Cheers,
Ingo