Tomorrow and the day after tomorrow..

walden21

Hi there!

I did validation & test job for stock price forcast as below.
Could you tell me is there anything wrong in my understanding?

(1)Data : I have 780 X 5 data(as like below)
-------------------------------------------------------------------------------

Date ND_C DJ_C KSP_O PL 2006-03-30 0.13 -0.58 0.09 2.80 2006-03-31 -0.04 -0.37 1.85 1.55 2006-04-03 -0.13 0.32 1.04 1.05 2006-04-04 0.37 0.53 0.67 1.05 ... 2009-06-01 0.06 3.02 2.57 -3.35

--------------------------------------------------------------------------------
(2)Validation : I trained my PolynomialRegressin model by SlidingWindowValidation.
    and wrote this model.
Here's XML for validation

<operator name="Root" class="Process" expanded="yes">
    <operator name="ExcelExampleSource" class="ExcelExampleSource">
        <parameter key="excel_file"	value="C:\NDDJ_3cls.xls"/>
        <parameter key="sheet_number"	value="2"/>
        <parameter key="first_row_as_names"	value="true"/>
        <parameter key="create_label"	value="true"/>
        <parameter key="label_column"	value="5"/>
        <parameter key="create_id"	value="true"/>
    </operator>
    <operator name="ExampleVisualizer" class="ExampleVisualizer">
    </operator>
    <operator name="SlidingWindowValidation" class="SlidingWindowValidation" expanded="yes">
        <parameter key="training_window_width"	value="75"/>
        <parameter key="training_window_step_size"	value="1"/>
        <parameter key="test_window_width"	value="1"/>
        <operator name="OperatorChain (2)" class="OperatorChain" expanded="yes">
            <operator name="PolynomialRegression" class="PolynomialRegression">
            </operator>
            <operator name="ModelWriter" class="ModelWriter">
                <parameter key="model_file"	value="C:\DJ_NN_SW.mod"/>
            </operator>
        </operator>
        <operator name="OperatorChain" class="OperatorChain" expanded="yes">
            <operator name="ModelApplier" class="ModelApplier">
                <list key="application_parameters">
                </list>
            </operator>
            <operator name="Performance" class="Performance">
            </operator>
        </operator>
    </operator>
</operator>

(3)Test : I loaded that model and apply to "SAME" data set that was used in Condition2.
*Here's XML for test

<operator name="Root" class="Process" expanded="yes">    <operator name="ExcelExampleSource" class="ExcelExampleSource">        <parameter key="excel_file" value="C:\NDDJ_3cls.xls"/>        <parameter key="sheet_number" value="3"/>        <parameter key="first_row_as_names" value="true"/>        <parameter key="label_column" value="4"/>        <parameter key="create_id" value="true"/>    </operator>    <operator name="ModelLoader" class="ModelLoader">        <parameter key="model_file" value="C:\DJ_NN_SW.mod"/>    </operator>    <operator name="ModelApplier" class="ModelApplier">        <list key="application_parameters">        </list>    </operator> </operator>

(4)SlidingWidow parameter
1.Training_Window_Width : 75
2.Training_Window_Step_size : 1
3.Test_window_width : 1
4.Horizon : 1

Did I used "the day after tomorrow's data" to predict "tomorrow' price" in test process even after 75th data?

haddock

Hi there Walden21,

Did I used "the day after tomorrow's data" to predict "tomorrow' price" in test process even after 75th data?

No, but you did use data that comes after the test date to train the model, except on the last test example. The model that gets used in your phase 3 was trained on the last but one set of 75 examples, so if you had 1000 examples it would have been trained on examples 924-999, and validated on example 1000. If you give the examples IDs, and put in breaks before the learn and model applier operators, you can see the point.

Hope that doesn't make things more confusing...