[Solved] - Using the prior value of an attribute to calculated the current value

CharlieG

I'm trying to label examples as to whether they're part of a trend that's gernerlly upward or generally downward.  I've got the first part working.  I'm calculating the difference between successive examples using the Differentiate operator and then using Generate Attribute to assign a label based on the sign of the difference.  Here's an example:

MovingAvg   DeltaMovingAvg     Trend
0.629           ?                       Up (arbitary choice)
0.600           -0.029                 Down
0.571           -0.029                 Down
0.557           -0.014                 Down
0.586           0.029                   Up
0.583           -0.003                 Down
0.583           0                       Up
0.597           0.014                   Up

So far so good.  But I really don't want the little value of -0.003 to get in the way of a generally upward trend.  So I'd like to add a rule that says - if the value of the difference is less than a threshold (say 0.02) then don't change the label on the trend from what it was in the previous example.  After applying this rule I'd get:

MovingAvg   DeltaMovingAvg     Trend
0.629           ?                       Up (arbitary choice)
0.600           -0.029                 Down
0.571           -0.029                 Down
0.557           -0.014                 Down
0.586           0.029                   Up
0.583           -0.003                 Up
0.583           0                       Up
0.597           0.014                   Up

I suspect I need a Loop Example to do this but I can't find a good example of how to use Loop Example.  Can anyone point me to a simple tutorial on Loop Example?  Or is there an easier way to do this?

MariusHelf

Hi, use the Windowing operator from the Series extension to create examples which contain the values from the previous two rows (window_size = 2), remove the unnecessary attributes, and add a Generate_Attributes operator with a rule like:
corrected_Trend: if (Trend_1 != Trend_0 && DeltaMovingAvg<0.02, Trend_1, Trend_0)

Best, Marius

CharlieG

Thanks for the suggestion Marius but if I implemented it correctly it doesn't quite solve the problem.  Going back to my example the code correctly flips the 3rd example from the bottom from Down to Up.  Then we move to the second example from the bottom.  The initial trend is Up since we have a delta of 0.  However this delta is less than my threshold of 0.02 so it goes to get the trend from the previous example.  When the windowing operation ran the previous example (3rd from bottom) said "Down".  It doesn't know that its value was just flipped to "Up".  So 2nd example from bottom gets assigned "Down" which is wrong.

Unfortunately it's looking like I may have to process example by example.

MariusHelf

Well, then you can use Loop Examples with some heavy macro arithmetic. Please have a look at the attached process.

Happy Mining!
~Marius

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.000" expanded="true" name="Process">
    <process expanded="true" height="416" width="749">
      <operator activated="true" class="generate_data" compatibility="5.3.000" expanded="true" height="60" name="Generate Data" width="90" x="45" y="30">
        <parameter key="target_function" value="random classification"/>
        <parameter key="number_of_attributes" value="1"/>
      </operator>
      <operator activated="true" class="generate_empty_attribute" compatibility="5.3.000" expanded="true" height="76" name="Generate Empty Attribute" width="90" x="179" y="30">
        <parameter key="name" value="trend_attr"/>
      </operator>
      <operator activated="true" class="set_macro" compatibility="5.3.000" expanded="true" height="76" name="Set Macro" width="90" x="313" y="30">
        <parameter key="macro" value="delta"/>
        <parameter key="value" value="2"/>
      </operator>
      <operator activated="true" class="loop_examples" compatibility="5.3.000" expanded="true" height="94" name="Loop Examples" width="90" x="447" y="30">
        <process expanded="true" height="434" width="767">
          <operator activated="true" class="branch" compatibility="5.3.000" expanded="true" height="76" name="Branch" width="90" x="45" y="30">
            <parameter key="condition_type" value="expression"/>
            <parameter key="condition_value" value="%{example} &gt;= 2"/>
            <process expanded="true" height="434" width="358">
              <operator activated="true" class="generate_macro" compatibility="5.3.000" expanded="true" height="76" name="Generate Macro" width="90" x="45" y="30">
                <list key="function_descriptions">
                  <parameter key="prev" value="%{example}-1"/>
                </list>
              </operator>
              <operator activated="true" class="extract_macro" compatibility="5.3.000" expanded="true" height="60" name="Extract Macro (2)" width="90" x="45" y="120">
                <parameter key="macro" value="v1"/>
                <parameter key="macro_type" value="data_value"/>
                <parameter key="attribute_name" value="att1"/>
                <parameter key="example_index" value="%{prev}"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="extract_macro" compatibility="5.3.000" expanded="true" height="60" name="Extract Macro" width="90" x="179" y="120">
                <parameter key="macro" value="v2"/>
                <parameter key="macro_type" value="data_value"/>
                <parameter key="attribute_name" value="att1"/>
                <parameter key="example_index" value="%{example}"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="generate_macro" compatibility="5.3.000" expanded="true" height="76" name="Generate Macro (2)" width="90" x="45" y="210">
                <list key="function_descriptions">
                  <parameter key="trend" value="if(abs(%{v2}-%{v1}) &gt; %{delta}, sgn(%{v2}-%{v1}), 0)"/>
                </list>
              </operator>
              <operator activated="true" class="set_data" compatibility="5.3.000" expanded="true" height="76" name="Set Data" width="90" x="179" y="210">
                <parameter key="example_index" value="%{example}"/>
                <parameter key="attribute_name" value="trend_attr"/>
                <parameter key="value" value="%{trend}"/>
                <list key="additional_values"/>
              </operator>
              <connect from_port="condition" to_op="Generate Macro" to_port="through 1"/>
              <connect from_op="Generate Macro" from_port="through 1" to_op="Extract Macro (2)" to_port="example set"/>
              <connect from_op="Extract Macro (2)" from_port="example set" to_op="Extract Macro" to_port="example set"/>
              <connect from_op="Extract Macro" from_port="example set" to_op="Generate Macro (2)" to_port="through 1"/>
              <connect from_op="Generate Macro (2)" from_port="through 1" to_op="Set Data" to_port="example set input"/>
              <connect from_op="Set Data" from_port="example set output" to_port="input 1"/>
              <portSpacing port="source_condition" spacing="0"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="sink_input 1" spacing="0"/>
              <portSpacing port="sink_input 2" spacing="0"/>
            </process>
            <process expanded="true" height="434" width="358">
              <connect from_port="condition" to_port="input 1"/>
              <portSpacing port="source_condition" spacing="0"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="sink_input 1" spacing="0"/>
              <portSpacing port="sink_input 2" spacing="0"/>
            </process>
          </operator>
          <connect from_port="example set" to_op="Branch" to_port="condition"/>
          <connect from_op="Branch" from_port="input 1" to_port="example set"/>
          <portSpacing port="source_example set" spacing="0"/>
          <portSpacing port="sink_example set" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
          <portSpacing port="sink_output 2" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Generate Empty Attribute" to_port="example set input"/>
      <connect from_op="Generate Empty Attribute" from_port="example set output" to_op="Set Macro" to_port="through 1"/>
      <connect from_op="Set Macro" from_port="through 1" to_op="Loop Examples" to_port="example set"/>
      <connect from_op="Loop Examples" from_port="example set" to_port="result 1"/>
      <connect from_op="Loop Examples" from_port="output 1" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

CharlieG

Thanks Marius - that did the trick.  A Loop Examples example even I could follow.