Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

Xvalidation

hi,

i am a rapid-i novice.

i want to train the model on a specific set of data and then test on another specific set. from what i can see (so far), there is a round about way of doing this by using:

Xvalidation and selelecting sampling_type = "linear sampling" and number_of_validations = 2

although this requires that both training and testing data sets have the same number of elements and are in a particular order.

is there a more general / sensible way of doing this? in particular, can i base the sampling on one of the data attributes?

many thanks, evgeny.

Find more posts tagged with

AI Studio

Accepted answers

All comments

haddock

G'Day Evgeny,

Welcome to the world of countless combinations! Sure, linear sampling in a validation wrapper would work, with the limitations you spot, but you can always go freestyle....

1. Filter your examples by attribute value to make a training set.
2. Add a learner to make a model, but do not keep the training examples.
3. Load your test set and apply the model.
4. Have a beer, and examine the results.

Actually the XVal operators just bundle this up so you can test repeatedly, but don't stop for the beer ( but you can always insert a break for that ;D ).

Good luck!

evgeny

tx for the quick response. i don't suppose you can post an example of pts 1-3? i.e. how one would do it in practice.

haddock

Just this once

<operator name="Root" class="Process" expanded="yes">
    <description text="Check comments tab!"/>
    <operator name="ExampleSetGenerator" class="ExampleSetGenerator">
        <parameter key="target_function"	value="simple non linear classification"/>
    </operator>
    <operator name="Copy exampleset" class="IOMultiplier">
        <description text="The Learner will consume the examples, so keep a copy for later."/>
        <parameter key="io_object"	value="ExampleSet"/>
    </operator>
    <operator name="1 Train on Att1 positive" class="ExampleFilter" breakpoints="after">
        <parameter key="condition_class"	value="attribute_value_filter"/>
        <parameter key="parameter_string"	value="att1&gt;0"/>
    </operator>
    <operator name="2 Make model" class="ID3Numerical">
    </operator>
    <operator name="3 Test on Att1 negative" class="ExampleFilter" breakpoints="after">
        <description text="As the learner has consumed the original exampleset, the copy madein step two is on top of the stack."/>
        <parameter key="condition_class"	value="attribute_value_filter"/>
        <parameter key="parameter_string"	value="att1&lt;0"/>
    </operator>
    <operator name="Get Beer" class="ModelApplier">
        <parameter key="keep_model"	value="true"/>
        <list key="application_parameters">
        </list>
    </operator>
</operator>

evgeny

thanks - that's very helpful.