Price forecast for the next thirty days(very urgent)

sky20

Hello everyone, have a nice day
I want to predict the price for at least the next thirty days.
I searched a lot in the forum. I found an example that almost fits my goal. But there is an error running on my data. Which I could not fix myself. Someone can help me. ?

This

<?xml version="1.0" encoding="UTF-8"?><process version="8.1.003">
  <context>
    <input/>
    <output/>
    <macros>
      <macro>
        <key>futureDays</key>
        <value>5</value>
      </macro>
      <macro>
        <key>horizon</key>
        <value>1</value>
      </macro>
      <macro>
        <key>windowSize</key>
        <value>6</value>
      </macro>
    </macros>
  </context>
  <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="false" class="store" compatibility="8.1.003" expanded="true" height="68" name="Store" width="90" x="983" y="85">
        <parameter key="repository_entry" value="//Personal/data/forecast_results"/>
      </operator>
      <operator activated="true" class="read_excel" compatibility="8.1.003" expanded="true" height="68" name="Read Excel" width="90" x="45" y="34">
        <parameter key="excel_file" value="C:\myData.xlsx"/>
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <operator activated="true" class="set_role" compatibility="5.3.013" expanded="true" height="82" name="Set Role" width="90" x="179" y="34">
        <parameter key="attribute_name" value="CLOSE"/>
        <parameter key="target_role" value="label"/>
        <list key="set_additional_roles"/>
      </operator>
      <operator activated="true" class="nominal_to_numerical" compatibility="8.1.003" expanded="true" height="103" name="Nominal to Numerical" width="90" x="179" y="136">
        <list key="comparison_groups"/>
      </operator>
      <operator activated="true" class="series:windowing" compatibility="7.4.000" expanded="true" height="82" name="Windowing" width="90" x="380" y="34">
        <parameter key="window_size" value="10"/>
        <parameter key="create_label" value="true"/>
        <parameter key="label_attribute" value="CLOSE"/>
        <parameter key="horizon" value="%{futureDays}"/>
      </operator>
      <operator activated="true" class="series:windowing" compatibility="7.4.000" expanded="true" height="82" name="Windowing (2)" width="90" x="380" y="136">
        <parameter key="window_size" value="10"/>
        <parameter key="label_attribute" value="Close"/>
      </operator>
      <operator activated="true" class="extract_macro" compatibility="8.1.003" expanded="true" height="68" name="Extract Macro" width="90" x="380" y="238">
        <parameter key="macro" value="label"/>
        <list key="additional_macros"/>
      </operator>
      <operator activated="true" class="generate_macro" compatibility="8.1.003" expanded="true" height="82" name="Generate Macro" width="90" x="380" y="340">
        <list key="function_descriptions">
          <parameter key="filter_range" value="eval(%{label})-1"/>
        </list>
      </operator>
      <operator activated="true" class="remember" compatibility="8.1.003" expanded="true" height="68" name="Remember" width="90" x="514" y="442">
        <parameter key="name" value="LastRow"/>
      </operator>
      <operator activated="true" class="series:sliding_window_validation" compatibility="7.4.000" expanded="true" height="124" name="Validation" width="90" x="514" y="187">
        <parameter key="training_window_width" value="1"/>
        <parameter key="training_window_step_size" value="1"/>
        <parameter key="test_window_width" value="1"/>
        <parameter key="horizon" value="%{futureDays}"/>
        <parameter key="cumulative_training" value="true"/>
        <process expanded="true">
          <operator activated="false" class="support_vector_machine" compatibility="8.1.003" expanded="true" height="124" name="SVM" width="90" x="179" y="34">
            <parameter key="kernel_type" value="radial"/>
            <parameter key="kernel_gamma" value="1000.0"/>
            <parameter key="C" value="1000.0"/>
          </operator>
          <operator activated="false" class="support_vector_machine_linear" compatibility="8.1.003" expanded="true" height="82" name="SVM (Linear)" width="90" x="179" y="187">
            <parameter key="C" value="100.0"/>
          </operator>
          <operator activated="true" class="linear_regression" compatibility="8.1.003" expanded="true" height="103" name="Linear Regression" width="90" x="179" y="289">
            <parameter key="feature_selection" value="none"/>
          </operator>
          <connect from_port="training" to_op="Linear Regression" to_port="training set"/>
          <connect from_op="Linear Regression" from_port="model" to_port="model"/>
          <portSpacing port="source_training" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true">
          <operator activated="true" class="apply_model" compatibility="8.1.003" expanded="true" height="82" name="Apply Model (2)" width="90" x="45" y="34">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="series:forecasting_performance" compatibility="7.4.000" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
            <parameter key="horizon" value="%{futureDays}"/>
          </operator>
          <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
          <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_averagable 1" spacing="0"/>
          <portSpacing port="sink_averagable 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="loop" compatibility="8.1.003" expanded="true" height="82" name="Loop" width="90" x="715" y="34">
        <parameter key="set_iteration_macro" value="true"/>
        <parameter key="macro_name" value="loop_forecasts"/>
        <parameter key="iterations" value="%{futureDays}"/>
        <process expanded="true">
          <operator activated="true" class="recall" compatibility="8.1.003" expanded="true" height="68" name="Recall" width="90" x="45" y="85">
            <parameter key="name" value="LastRow"/>
            <parameter key="remove_from_store" value="false"/>
          </operator>
          <operator activated="true" class="apply_model" compatibility="7.1.001" expanded="true" height="82" name="Apply Model" width="90" x="246" y="34">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="multiply" compatibility="8.1.003" expanded="true" height="82" name="Multiply" width="90" x="380" y="34"/>
          <operator activated="false" class="materialize_data" compatibility="8.1.003" expanded="true" height="82" name="Materialize Data (2)" width="90" x="45" y="187"/>
          <operator activated="true" class="generate_attributes" compatibility="8.1.003" expanded="true" height="82" name="Generate Attributes" width="90" x="514" y="34">
            <list key="function_descriptions">
              <parameter key="Date" value="date_add(Date,eval(%{loop_forecasts}),DATE_UNIT_DAY)"/>
            </list>
          </operator>
          <operator activated="true" class="set_role" compatibility="5.3.013" expanded="true" height="82" name="Set Role (2)" width="90" x="648" y="34">
            <parameter key="attribute_name" value="prediction(label)"/>
            <list key="set_additional_roles"/>
          </operator>
          <operator activated="true" class="select_attributes" compatibility="8.1.003" expanded="true" height="82" name="Select Attributes (3)" width="90" x="782" y="34">
            <parameter key="attribute_filter_type" value="single"/>
            <parameter key="attribute" value="prediction(label)"/>
          </operator>
          <operator activated="true" class="replace" compatibility="8.1.003" expanded="true" height="82" name="Replace" width="90" x="916" y="34">
            <parameter key="replace_what" value="Close"/>
            <parameter key="replace_by" value="$1-"/>
          </operator>
          <connect from_port="input 1" to_op="Apply Model" to_port="model"/>
          <connect from_op="Recall" from_port="result" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Multiply" to_port="input"/>
          <connect from_op="Multiply" from_port="output 1" to_op="Generate Attributes" to_port="example set input"/>
          <connect from_op="Generate Attributes" from_port="example set output" to_op="Set Role (2)" to_port="example set input"/>
          <connect from_op="Set Role (2)" from_port="example set output" to_op="Select Attributes (3)" to_port="example set input"/>
          <connect from_op="Select Attributes (3)" from_port="example set output" to_op="Replace" to_port="example set input"/>
          <connect from_op="Replace" from_port="example set output" to_port="output 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
          <portSpacing port="sink_output 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="append" compatibility="8.1.003" expanded="true" height="82" name="Append" width="90" x="849" y="34"/>
      <operator activated="true" class="log" compatibility="8.1.003" expanded="true" height="82" name="Log" width="90" x="782" y="289">
        <parameter key="filename" value="tmp"/>
        <list key="log">
          <parameter key="Gamma" value="operator.SVM.parameter.kernel_gamma"/>
          <parameter key="C" value="operator.SVM.parameter.C"/>
          <parameter key="Forecast Perf" value="operator.Validation.value.performance"/>
          <parameter key="Training Width" value="operator.Validation.parameter.training_window_width"/>
          <parameter key="Testing Width" value="operator.Validation.parameter.test_window_width"/>
        </list>
      </operator>
      <connect from_op="Read Excel" from_port="output" to_op="Set Role" to_port="example set input"/>
      <connect from_op="Set Role" from_port="example set output" to_op="Nominal to Numerical" to_port="example set input"/>
      <connect from_op="Nominal to Numerical" from_port="example set output" to_op="Windowing" to_port="example set input"/>
      <connect from_op="Windowing" from_port="example set output" to_op="Validation" to_port="training"/>
      <connect from_op="Windowing" from_port="original" to_op="Windowing (2)" to_port="example set input"/>
      <connect from_op="Windowing (2)" from_port="example set output" to_op="Extract Macro" to_port="example set"/>
      <connect from_op="Extract Macro" from_port="example set" to_op="Generate Macro" to_port="through 1"/>
      <connect from_op="Generate Macro" from_port="through 1" to_op="Remember" to_port="store"/>
      <connect from_op="Validation" from_port="model" to_op="Loop" to_port="input 1"/>
      <connect from_op="Validation" from_port="averagable 1" to_op="Log" to_port="through 1"/>
      <connect from_op="Loop" from_port="output 1" to_op="Append" to_port="example set 1"/>
      <connect from_op="Append" from_port="merged set" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="231"/>
    </process>
  </operator>
</process>

Or in this run
Error missing label. If I set the label !!
<?xml version="1.0" encoding="UTF-8"?><process version="8.1.003">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="8.1.003" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="read_excel" compatibility="8.1.003" expanded="true" height="68" name="Read Excel" width="90" x="45" y="34">
        <parameter key="excel_file" value="C:\myData.xlsx"/>
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <operator activated="true" class="set_role" compatibility="8.1.003" expanded="true" height="82" name="Set Role" width="90" x="112" y="136">
        <parameter key="attribute_name" value="CLOSE"/>
        <parameter key="target_role" value="label"/>
        <list key="set_additional_roles"/>
      </operator>
      <operator activated="true" class="nominal_to_numerical" compatibility="8.1.003" expanded="true" height="103" name="Nominal to Numerical" width="90" x="246" y="187">
        <list key="comparison_groups"/>
      </operator>
      <operator activated="true" class="filter_example_range" compatibility="8.1.003" expanded="true" height="82" name="Filter Example Range" width="90" x="313" y="34">
        <parameter key="first_example" value="1"/>
        <parameter key="last_example" value="16"/>
        <parameter key="invert_filter" value="true"/>
      </operator>
      <operator activated="true" class="series:windowing" compatibility="7.4.000" expanded="true" height="82" name="Windowing (2)" width="90" x="447" y="34">
        <parameter key="window_size" value="10"/>
      </operator>
      <operator activated="true" class="concurrency:cross_validation" compatibility="8.1.003" expanded="true" height="145" name="Cross Validation" width="90" x="581" y="34">
        <process expanded="true">
          <operator activated="true" class="h2o:gradient_boosted_trees" compatibility="7.6.001" expanded="true" height="103" name="Gradient Boosted Trees" width="90" x="179" y="34">
            <parameter key="number_of_trees" value="100"/>
            <list key="expert_parameters"/>
          </operator>
          <connect from_port="training set" to_op="Gradient Boosted Trees" to_port="training set"/>
          <connect from_op="Gradient Boosted Trees" from_port="model" to_port="model"/>
          <portSpacing port="source_training set" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true">
          <operator activated="true" class="apply_model" compatibility="8.1.003" expanded="true" height="82" name="Apply Model" width="90" x="45" y="34">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="performance_regression" compatibility="8.1.003" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
            <parameter key="relative_error" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
          <connect from_op="Performance" from_port="example set" to_port="test set results"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_test set results" spacing="0"/>
          <portSpacing port="sink_performance 1" spacing="0"/>
          <portSpacing port="sink_performance 2" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Read Excel" from_port="output" to_op="Set Role" to_port="example set input"/>
      <connect from_op="Set Role" from_port="example set output" to_op="Nominal to Numerical" to_port="example set input"/>
      <connect from_op="Nominal to Numerical" from_port="example set output" to_op="Filter Example Range" to_port="example set input"/>
      <connect from_op="Filter Example Range" from_port="example set output" to_op="Windowing (2)" to_port="example set input"/>
      <connect from_op="Windowing (2)" from_port="example set output" to_op="Cross Validation" to_port="example set"/>
      <connect from_op="Cross Validation" from_port="model" to_port="result 1"/>
      <connect from_op="Cross Validation" from_port="example set" to_port="result 2"/>
      <connect from_op="Cross Validation" from_port="test result set" to_port="result 3"/>
      <connect from_op="Cross Validation" from_port="performance 1" to_port="result 4"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
      <portSpacing port="sink_result 5" spacing="0"/>
    </process>
  </operator>
</process>

This is my data

 

I appreciate any help.

sky20

Hi Please help me.
Does anyone really know the objection of the submitted sample? Why does it have the error of not labeling despite labeling?

jacobcybulski

Let me try. You are using an old Series extension, so I cannot test it. Try using the built in Series extension, it has many new features. I will assume that you are using the new built-in Windowing. Let me discussed the very last case as it is simple enough for me to decipher. When you use Windowing and do not specify the index then the order of examples is used as index. In this case, your Date attribute will also be windowed and will be changed to Date-1, Date-2, etc. It will disappear and the selected horizon+1 attribute will be moved to the label, I assume you want to predict the closing stock price, so it will be Close+1. If you do use an explicit index, which I assume is a Date, then Date again vanishes and only the last date is provided. Again, Close+1 will become a label. In either case, if you did what I was imagining you'd get a label and the process will finish. I can only think that your selection of parameters in the older version of Series was different?
Jacob

sky20

Hello
thanks for your response
But I clearly did not understand what you meant.

I think you mean I should label history. And use the training example of the program. But I'm wrong with what you said. For example, I installed the updated version of the program. And I used the ready sample, but it still has an error. Can you give a clearer guide? thanks for your help

jacobcybulski

You do not need to label anything before windowing, it will create a label for you. If you labelled Close attribute it will not be found later on for windowing (unless you click the option to include special), so make Close a regular attribute. However, what you need to do is to select which attributes will be windowed (i.e. their values included for t-0, t-1, ..., etc.), then select an index variable (use Date here), and decide which of the attribute's horizon (future) you are interested in (most likely Close). Now put a breakpoint after your Windowing and run. It will Jacob

sky20

Hi dear friend
Thanks for helping me solve this problem
look
I follow the steps you said. I did
But I have two problems
First that. I want to predict the price for the next ten days.
Secondly. There is an error in execution
Can you see my submission process? And help fix the error?

thanks for your help

sky20

Hi everyone
No one to help? I really did my best. But it does not give the desired result ... Can the professors of the association help?
Thankful

jacobcybulski

See what you are getting from your Windowing operator (break after). It is likely, you have very few examples coming out. Possibly because your window size is too large, or you do not allow overlapping windows, or you do not have enough data. If you want to see 10 days ahead, you need a horizon of 10 days. I can see however that your data has dates in reverse order (so sort your data) and then you have missing dates (so you need to equalise time stamps), if you have a horizon of 10 days, you will also need to define which of them is a label. Should work.

sky20

Hello good time are you OK? I have brought more data. But again, I did not get a better result. I did not understand how to arrange the date? Do you mean this? May help. Thankful

jacobcybulski

I am not sure what you mean by "I did not get a better result", does it mean you've got an error or does it mean the forecasts were poor? Did you follow my advice? Did you equalise time-stamps (you have missing dates)? Did you see what is the output of your Windowing operator? What was the result? Jacob

sky20

Hello. Dear friend
I mean
I increased my data to 2000 and set the horizon value to 10. Because I want to predict stock prices for the next ten days. I entered all the attributes in the window. I used history as an indicator. Also close the field as a label
And I ran the program
But
I made no mistake. Only. The result was incomprehensible, ie the final price in one day was, for example, 2360, but it was predicted in the label. The price is 2960.
how is it possible?
Don't have a workable instance? I know there is a great demand. But the more time I spend. The result is getting worse ...
I do not know how for the next ten days given the amount of data. Predict stock prices.
??

jacobcybulski

I am sorry but you have not answered my questions. (1) Have you switched from the extension-based time series analysis to the built-in time series (I can see you did not and the extension is now obsolete)? (2) Have you sorted your data by index - Date (your dates are in reverse order) ? (3) Have you equalized the time stamp (using Equalize Time Stamp, which is needed as you have missing dates)? (4) Did you check what is coming out of the Windowing operator, what is it and does it make sense (start with one variable)?

Also, I suggest to do this in small steps. Rather than using all attributes for the multi-variate case - use just one, e.g. the closing price. Rather than using a 10 day horizon, how about starting with a horizon of one? You are also trying to predict 10 days ahead from a 6 day window, which is not realistic.

Just because your predictions are bad (for a single example) it does not mean your analytic process is technically incorrect! Stock price prediction is difficult and prediction 10 days ahead is very difficult. To get some reasonable prediction you'd have to incorporate other variables which are not in the stock data, such as socio-economic indicators, business news, etc. There are some good books around on financial forecasting.

Jacob

P.S. Niels Bohr said “Prediction is very difficult, especially if it's about the future!”

sky20

Hello

Thank you for your response

I will definitely review your recommendations.

I do all the solutions you suggest. And I will announce the result here. wait.