ARIMA operator returning attribute does not exist in RM studio

Samir

Hi all,

I have the Temp attribute in the data set but ARIMA can't recognize it and returns attribute does not exist. When I save the data as excel file and then retrieve the same data as excel file instead of fetching from data base, ARIMA does not return that error. I also attached the excel file.

Could it be a UI problem and how to work around it?

OS : Ubuntu 18.04

RM studio : 9.5

data set  : 2 regular attribute / 721 examples

XML :

<?xml version="1.0" encoding="UTF-8"?><process version="9.5.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="9.5.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="subprocess" compatibility="9.5.001" expanded="true" height="82" name="Subprocess" width="90" x="112" y="34">
        <process expanded="true">
          <operator activated="true" class="web:get_webpage" compatibility="9.0.000" expanded="true" height="68" name="Get Page" width="90" x="45" y="34">
            <parameter key="url" value="???"/>
            <parameter key="random_user_agent" value="false"/>
            <parameter key="connection_timeout" value="10000"/>
            <parameter key="read_timeout" value="10000"/>
            <parameter key="follow_redirects" value="true"/>
            <parameter key="accept_cookies" value="none"/>
            <parameter key="cookie_scope" value="global"/>
            <parameter key="request_method" value="GET"/>
            <list key="query_parameters">
              <parameter key="db" value="NIMBLE"/>
              <parameter key="q" value="SELECT%20mean(%22temp%22)%20FROM%20%22autogen%22.%22env%22%20WHERE%20(%22bizLocation%22%20%3D~%20%2F%5Eurn%3Aepc%3Aid%3Asgln%3AbizLocation%5C.PodComp%5C.2%24%2F)%20AND%20time%20%3E%3D%20now()%20-%205d%20GROUP%20BY%20time(10m)%3B"/>
              <parameter key="u" value="grafana"/>
              <parameter key="p" value="grafana"/>
              <parameter key="pretty" value="true"/>
            </list>
            <list key="request_properties"/>
            <parameter key="override_encoding" value="false"/>
            <parameter key="encoding" value="SYSTEM"/>
          </operator>
          <operator activated="true" class="web:json_to_xml" compatibility="9.0.000" expanded="true" height="68" name="JSON to XML" width="90" x="179" y="34">
            <parameter key="enclosing_tag_name" value="json"/>
          </operator>
          <operator activated="true" class="text:remove_document_parts" compatibility="8.2.000" expanded="true" height="68" name="Remove Document Parts" width="90" x="313" y="34">
            <parameter key="deletion_regex" value="&lt;*json&gt;|&lt;*results&gt;|&lt;*series&gt;|&lt;columns&gt;time&lt;/columns&gt;&lt;columns&gt;temp&lt;/columns&gt;|&lt;statement_id&gt;0&lt;/statement_id&gt;|&lt;name&gt;env&lt;/name&gt;|&lt;/values&gt;|&lt;/array&gt;|&lt;columns&gt;time&lt;/columns&gt;&lt;columns&gt;humid&lt;/columns&gt;|&lt;statement_id&gt;1&lt;/statement_id&gt;|&lt;/"/>
          </operator>
          <operator activated="true" class="operator_toolbox:split_document_into_collection" compatibility="2.3.000" expanded="true" height="82" name="Split Document into Collection" width="90" x="447" y="34">
            <parameter key="split_string" value="&lt;values&gt;"/>
          </operator>
          <operator activated="true" class="split" compatibility="9.5.001" expanded="true" height="82" name="Split" width="90" x="45" y="187">
            <parameter key="attribute_filter_type" value="single"/>
            <parameter key="attribute" value="Token"/>
            <parameter key="attributes" value="Token"/>
            <parameter key="use_except_expression" value="false"/>
            <parameter key="value_type" value="nominal"/>
            <parameter key="use_value_type_exception" value="false"/>
            <parameter key="except_value_type" value="file_path"/>
            <parameter key="block_type" value="single_value"/>
            <parameter key="use_block_type_exception" value="false"/>
            <parameter key="except_block_type" value="single_value"/>
            <parameter key="invert_selection" value="false"/>
            <parameter key="include_special_attributes" value="false"/>
            <parameter key="split_pattern" value="(&lt;array&gt;)|(&lt;array&gt;).*"/>
            <parameter key="split_mode" value="ordered_split"/>
          </operator>
          <operator activated="true" class="nominal_to_date" compatibility="9.5.001" expanded="true" height="82" name="Nominal to Date" width="90" x="179" y="187">
            <parameter key="attribute_name" value="Token_2"/>
            <parameter key="date_type" value="date_time"/>
            <parameter key="date_format" value="yyyy-MM-dd'T'HH:mm:ss'Z'"/>
            <parameter key="time_zone" value="Europe/Berlin"/>
            <parameter key="locale" value="English (United States)"/>
            <parameter key="keep_old_attribute" value="false"/>
          </operator>
          <operator activated="true" class="filter_examples" compatibility="9.5.001" expanded="true" height="103" name="Filter Examples" width="90" x="313" y="187">
            <parameter key="parameter_expression" value=""/>
            <parameter key="condition_class" value="missing_attributes"/>
            <parameter key="invert_filter" value="true"/>
            <list key="filters_list">
              <parameter key="filters_entry_key" value="class.eq.fail"/>
            </list>
            <parameter key="filters_logic_and" value="true"/>
            <parameter key="filters_check_metadata" value="true"/>
          </operator>
          <operator activated="true" class="rename_by_replacing" compatibility="9.5.001" expanded="true" height="82" name="Rename by Replacing" width="90" x="447" y="187">
            <parameter key="attribute_filter_type" value="single"/>
            <parameter key="attribute" value="Token_3"/>
            <parameter key="attributes" value=""/>
            <parameter key="use_except_expression" value="false"/>
            <parameter key="value_type" value="attribute_value"/>
            <parameter key="use_value_type_exception" value="false"/>
            <parameter key="except_value_type" value="time"/>
            <parameter key="block_type" value="attribute_block"/>
            <parameter key="use_block_type_exception" value="false"/>
            <parameter key="except_block_type" value="value_matrix_row_start"/>
            <parameter key="invert_selection" value="false"/>
            <parameter key="include_special_attributes" value="false"/>
            <parameter key="replace_what" value="Token_3"/>
            <parameter key="replace_by" value="temp"/>
          </operator>
          <operator activated="true" class="rename_by_replacing" compatibility="9.5.001" expanded="true" height="82" name="Rename by Replacing (2)" width="90" x="581" y="187">
            <parameter key="attribute_filter_type" value="value_type"/>
            <parameter key="attribute" value=""/>
            <parameter key="attributes" value=""/>
            <parameter key="use_except_expression" value="false"/>
            <parameter key="value_type" value="date_time"/>
            <parameter key="use_value_type_exception" value="false"/>
            <parameter key="except_value_type" value="time"/>
            <parameter key="block_type" value="attribute_block"/>
            <parameter key="use_block_type_exception" value="false"/>
            <parameter key="except_block_type" value="value_matrix_row_start"/>
            <parameter key="invert_selection" value="false"/>
            <parameter key="include_special_attributes" value="false"/>
            <parameter key="replace_what" value="Token_2"/>
            <parameter key="replace_by" value="datetime"/>
          </operator>
          <operator activated="true" class="select_attributes" compatibility="9.5.001" expanded="true" height="82" name="Select Attributes" width="90" x="715" y="187">
            <parameter key="attribute_filter_type" value="subset"/>
            <parameter key="attribute" value=""/>
            <parameter key="attributes" value="datetime|temp"/>
            <parameter key="use_except_expression" value="false"/>
            <parameter key="value_type" value="attribute_value"/>
            <parameter key="use_value_type_exception" value="false"/>
            <parameter key="except_value_type" value="time"/>
            <parameter key="block_type" value="attribute_block"/>
            <parameter key="use_block_type_exception" value="false"/>
            <parameter key="except_block_type" value="value_matrix_row_start"/>
            <parameter key="invert_selection" value="false"/>
            <parameter key="include_special_attributes" value="false"/>
          </operator>
          <operator activated="true" class="parse_numbers" compatibility="9.5.001" expanded="true" height="82" name="Parse Numbers" width="90" x="715" y="34">
            <parameter key="attribute_filter_type" value="subset"/>
            <parameter key="attribute" value="Token_3"/>
            <parameter key="attributes" value="temp"/>
            <parameter key="use_except_expression" value="false"/>
            <parameter key="value_type" value="nominal"/>
            <parameter key="use_value_type_exception" value="false"/>
            <parameter key="except_value_type" value="file_path"/>
            <parameter key="block_type" value="single_value"/>
            <parameter key="use_block_type_exception" value="false"/>
            <parameter key="except_block_type" value="single_value"/>
            <parameter key="invert_selection" value="false"/>
            <parameter key="include_special_attributes" value="false"/>
            <parameter key="decimal_character" value="."/>
            <parameter key="grouped_digits" value="false"/>
            <parameter key="grouping_character" value=","/>
            <parameter key="infinity_representation" value=""/>
            <parameter key="unparsable_value_handling" value="fail"/>
          </operator>
          <connect from_op="Get Page" from_port="output" to_op="JSON to XML" to_port="document"/>
          <connect from_op="JSON to XML" from_port="document" to_op="Remove Document Parts" to_port="document"/>
          <connect from_op="Remove Document Parts" from_port="document" to_op="Split Document into Collection" to_port="document"/>
          <connect from_op="Split Document into Collection" from_port="example set" to_op="Split" to_port="example set input"/>
          <connect from_op="Split" from_port="example set output" to_op="Nominal to Date" to_port="example set input"/>
          <connect from_op="Nominal to Date" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
          <connect from_op="Filter Examples" from_port="example set output" to_op="Rename by Replacing" to_port="example set input"/>
          <connect from_op="Rename by Replacing" from_port="example set output" to_op="Rename by Replacing (2)" to_port="example set input"/>
          <connect from_op="Rename by Replacing (2)" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
          <connect from_op="Select Attributes" from_port="example set output" to_op="Parse Numbers" to_port="example set input"/>
          <connect from_op="Parse Numbers" from_port="example set output" to_port="out 1"/>
          <portSpacing port="source_in 1" spacing="0"/>
          <portSpacing port="sink_out 1" spacing="0"/>
          <portSpacing port="sink_out 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="9.5.001" expanded="true" height="145" name="Optimize Parameters (Grid)" origin="GENERATED_TUTORIAL" width="90" x="380" y="34">
        <list key="parameters">
          <parameter key="ARIMA Trainer.p:_order_of_the_autoregressive_model" value="[1;5;4;linear]"/>
          <parameter key="ARIMA Trainer.q:_order_of_the_moving-average_model" value="[0.0;5;5;linear]"/>
        </list>
        <parameter key="error_handling" value="fail on error"/>
        <parameter key="log_performance" value="true"/>
        <parameter key="log_all_criteria" value="false"/>
        <parameter key="synchronize" value="false"/>
        <parameter key="enable_parallel_execution" value="true"/>
        <process expanded="true">
          <operator activated="true" class="time_series:arima_trainer" compatibility="9.5.000" expanded="true" height="103" name="ARIMA Trainer" origin="GENERATED_TUTORIAL" width="90" x="380" y="34">
            <parameter key="time_series_attribute" value=""/>
            <parameter key="has_indices" value="true"/>
            <parameter key="indices_attribute" value="datetime"/>
            <parameter key="p:_order_of_the_autoregressive_model" value="5"/>
            <parameter key="d:_degree_of_differencing" value="0"/>
            <parameter key="q:_order_of_the_moving-average_model" value="5"/>
            <parameter key="estimate_constant" value="true"/>
            <parameter key="main_criterion" value="aic"/>
          </operator>
          <connect from_port="input 1" to_op="ARIMA Trainer" to_port="example set"/>
          <connect from_op="ARIMA Trainer" from_port="forecast model" to_port="output 1"/>
          <connect from_op="ARIMA Trainer" from_port="performance" to_port="performance"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_performance" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
          <portSpacing port="sink_output 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="time_series:apply_forecast" compatibility="9.5.000" expanded="true" height="82" name="Apply Forecast" origin="GENERATED_TUTORIAL" width="90" x="648" y="187">
        <parameter key="forecast_horizon" value="144"/>
        <parameter key="add_original_time_series" value="true"/>
        <parameter key="add_combined_time_series" value="true"/>
        <description align="center" color="transparent" colored="false" width="126">The best fitting model is used to forecast the next 144 values of the Time Series</description>
      </operator>
      <connect from_op="Subprocess" from_port="out 1" to_op="Optimize Parameters (Grid)" to_port="input 1"/>
      <connect from_op="Optimize Parameters (Grid)" from_port="performance" to_port="result 3"/>
      <connect from_op="Optimize Parameters (Grid)" from_port="parameter set" to_port="result 4"/>
      <connect from_op="Optimize Parameters (Grid)" from_port="output 1" to_op="Apply Forecast" to_port="forecast model"/>
      <connect from_op="Apply Forecast" from_port="example set" to_port="result 1"/>
      <connect from_op="Apply Forecast" from_port="original" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
      <portSpacing port="sink_result 5" spacing="0"/>
    </process>
  </operator>
</process>

MartinLiebig

Hi @Samir ,

the reason is your get page in the beginning. RapidMiner cannot know what attributes are available, and thus we are not able to propagate the metadata accordingly. The ARIMA operator thus does not know whats coming in and the drop downs are not populated. You can still run the process after typing in the right attribute. The warning can be neglected.

Best,

Martin

MartinLiebig

Hi,

can you please check if the Temp attribute is of type numerical? It needs to be to be recognised. To parse it you likely want to use the Parse Numbers operator.

Best,
Martin

Samir

Hi @mschmitz

Yes. I already used Parse Numbers operator to change Temp from Nominal to Numeric.

I did something and the process runs error free but ARIMA operator still has the yellow sign saying that Temp attribute is missing.

I didn't enter the Time series attribute and tried to run the process and I got a pop up saying required parameter missing. When I push FIX NOW a dialog pops to set the parameter and when I enter the attribute there, I no longer get the initial error.

Also when I use the drop down menu to set a parameter, often there's nothing there although all operators are connected but again sometimes without any change I see the attributes in the drop down menu.

Best,

Saeed

MartinLiebig

Hi @Samir ,

the reason is your get page in the beginning. RapidMiner cannot know what attributes are available, and thus we are not able to propagate the metadata accordingly. The ARIMA operator thus does not know whats coming in and the drop downs are not populated. You can still run the process after typing in the right attribute. The warning can be neglected.

Best,

Martin