Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

How can I insert a Lift Chart into a Validation

Hello Community,
I saw this both interesting videos about ROC and LIFT charts. In this videos the LIFT charts appears all the time after a apply model.

http://www.youtube.com/watch?v=bsiG-xCoKP8

http://www.youtube.com/watch?v=j6Hxf5UtDKU

Nevertheless, in both videos they are not using a validation. I am trying to use the validation and a Lift Chart but I am getting all the time errors? I tried to insert the lift chart after the apply model in the validation but in the end I get this errors. What can I do?

Expected AverageVector but received ExampleSet.
Expected AverageVector but received DistributionModel.
Expected AverageVector but received LiftParetoChart.

In addition I am looking for a way or setting, that in the results example set an additionally row is inserted which shows the predicted class. So I can see which value is true and which one is false.

Best Regards and thank you

Marco

Find more posts tagged with

AI Studio

Accepted answers

All comments

marcopo

No one an idea?

MariusHelf

Hi,

as always, it would be a good idea to attach the process you have so far to your post as explained in my signature.

Concerning the Lift Chart, what do you expect from a lift chart in a validation? What do you want to achieve?

Best regards,
Marius

marcopo

Hi,

actually I wanna see the original class and the class which is predicted through the model. Additionally, I wanna use the lift-chart as performance indicator.


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
    <process expanded="true" height="755" width="1015">
      <operator activated="true" class="retrieve" compatibility="5.2.008" expanded="true" height="60" name="Retrieve" width="90" x="45" y="390">
        <parameter key="repository_entry" value="../../Daten/TextMiningAusgang0MinBinominaNeu"/>
      </operator>
      <operator activated="true" class="split_validation" compatibility="5.2.008" expanded="true" height="148" name="Validation (2)" width="90" x="410" y="293">
        <parameter key="parallelize_training" value="true"/>
        <parameter key="parallelize_testing" value="true"/>
        <process expanded="true" height="759" width="483">
          <operator activated="true" class="nominal_to_numerical" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Numerical (2)" width="90" x="45" y="30">
            <list key="comparison_groups"/>
          </operator>
          <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.2.008" expanded="true" height="76" name="SVM" width="90" x="264" y="30">
            <parameter key="svm_type" value="nu-SVC"/>
            <parameter key="C" value="2.0"/>
            <list key="class_weights"/>
          </operator>
          <connect from_port="training" to_op="Nominal to Numerical (2)" to_port="example set input"/>
          <connect from_op="Nominal to Numerical (2)" from_port="example set output" to_op="SVM" to_port="training set"/>
          <connect from_op="SVM" from_port="model" to_port="model"/>
          <portSpacing port="source_training" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true" height="759" width="483">
          <operator activated="true" class="apply_model" compatibility="5.2.008" expanded="true" height="76" name="Apply Model (2)" width="90" x="45" y="30">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="create_lift_chart" compatibility="5.2.008" expanded="true" height="94" name="Create Lift Chart" width="90" x="112" y="390">
            <parameter key="target_class" value="UP"/>
          </operator>
          <operator activated="true" class="performance_binominal_classification" compatibility="5.2.008" expanded="true" height="76" name="Performance (2)" width="90" x="246" y="30">
            <parameter key="classification_error" value="true"/>
            <parameter key="precision" value="true"/>
            <parameter key="recall" value="true"/>
            <parameter key="f_measure" value="true"/>
            <parameter key="sensitivity" value="true"/>
            <parameter key="specificity" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
          <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
          <connect from_op="Apply Model (2)" from_port="model" to_op="Create Lift Chart" to_port="model"/>
          <connect from_op="Create Lift Chart" from_port="lift pareto chart" to_port="averagable 2"/>
          <connect from_op="Performance (2)" from_port="performance" to_port="averagable 1"/>
          <connect from_op="Performance (2)" from_port="example set" to_port="averagable 3"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_averagable 1" spacing="0"/>
          <portSpacing port="sink_averagable 2" spacing="0"/>
          <portSpacing port="sink_averagable 3" spacing="0"/>
          <portSpacing port="sink_averagable 4" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Retrieve" from_port="output" to_op="Validation (2)" to_port="training"/>
      <connect from_op="Validation (2)" from_port="averagable 1" to_port="result 1"/>
      <connect from_op="Validation (2)" from_port="averagable 2" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

Best Regards

Marco

MariusHelf

Hi,

you can't use the Lift Chart as output of the X-Validation. As you know, the cross validation executes it's inner processes 10 times and then builds the average of the performances of all 10 iterations. But averaging a Lift Chart is not possible.
You would have to create a model outside of the validation and then create the Lift Chart based on that model.

I hope this helps!

Best regards,
Marius