How to use Polynomial Regression in rapidminer correctly

Hello, everyone. This is my first forum post asking questions about polynomial regression in rapidminer.
The original data is:x:4194.06 3466.45 2070.08 874.98 corresponding to y:91540.07 109460.36 120338.64 102182.19
As shown in the first flow, the first result expression is obtained by using the polynomial regression operator.
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="9.6.000" expanded="true" name="Process">
<parameter key="logverbosity" value="init"/>
<parameter key="random_seed" value="2001"/>
<parameter key="send_mail" value="never"/>
<parameter key="notification_email" value=""/>
<parameter key="process_duration_for_mail" value="30"/>
<parameter key="encoding" value="SYSTEM"/>
<process expanded="true">
<operator activated="true" class="read_excel" compatibility="9.6.000" expanded="true" height="68" name="Read Excel" width="90" x="45" y="85">
<parameter key="excel_file" value="C:\Users\1\Desktop\question data.xlsx"/>
<parameter key="sheet_selection" value="sheet number"/>
<parameter key="sheet_number" value="1"/>
<parameter key="imported_cell_range" value="A1"/>
<parameter key="encoding" value="SYSTEM"/>
<parameter key="first_row_as_names" value="true"/>
<list key="annotations"/>
<parameter key="date_format" value=""/>
<parameter key="time_zone" value="SYSTEM"/>
<parameter key="locale" value="English (United States)"/>
<parameter key="read_all_values_as_polynominal" value="false"/>
<list key="data_set_meta_data_information">
<parameter key="0" value="x.true.real.attribute"/>
<parameter key="1" value="y.true.real.attribute"/>
</list>
<parameter key="read_not_matching_values_as_missings" value="false"/>
<parameter key="datamanagement" value="double_array"/>
<parameter key="data_management" value="auto"/>
</operator>
<operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="179" y="85">
<parameter key="attribute_name" value="y"/>
<parameter key="target_role" value="label"/>
<list key="set_additional_roles">
<parameter key="x" value="regular"/>
</list>
</operator>
<operator activated="true" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression" width="90" x="313" y="85">
<parameter key="max_iterations" value="5000"/>
<parameter key="replication_factor" value="2"/>
<parameter key="max_degree" value="2"/>
<parameter key="min_coefficient" value="-100.0"/>
<parameter key="max_coefficient" value="100.0"/>
<parameter key="use_local_random_seed" value="false"/>
<parameter key="local_random_seed" value="1992"/>
</operator>
<connect from_op="Read Excel" from_port="output" to_op="Set Role" to_port="example set input"/>
<connect from_op="Set Role" from_port="example set output" to_op="Polynomial Regression" to_port="training set"/>
<connect from_op="Polynomial Regression" from_port="model" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
The second flow, based on the original data, creates a new list of attributes as x^2=z, and uses the linear regression operator to make the second result expression.
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="9.6.000" expanded="true" name="Process">
<parameter key="logverbosity" value="init"/>
<parameter key="random_seed" value="2001"/>
<parameter key="send_mail" value="never"/>
<parameter key="notification_email" value=""/>
<parameter key="process_duration_for_mail" value="30"/>
<parameter key="encoding" value="SYSTEM"/>
<process expanded="true">
<operator activated="true" class="read_excel" compatibility="9.6.000" expanded="true" height="68" name="Read Excel" width="90" x="45" y="85">
<parameter key="excel_file" value="C:\Users\1\Desktop\question data.xlsx"/>
<parameter key="sheet_selection" value="sheet number"/>
<parameter key="sheet_number" value="1"/>
<parameter key="imported_cell_range" value="A1"/>
<parameter key="encoding" value="SYSTEM"/>
<parameter key="first_row_as_names" value="true"/>
<list key="annotations"/>
<parameter key="date_format" value=""/>
<parameter key="time_zone" value="SYSTEM"/>
<parameter key="locale" value="English (United States)"/>
<parameter key="read_all_values_as_polynominal" value="false"/>
<list key="data_set_meta_data_information">
<parameter key="0" value="x.true.real.attribute"/>
<parameter key="1" value="y.true.real.attribute"/>
</list>
<parameter key="read_not_matching_values_as_missings" value="false"/>
<parameter key="datamanagement" value="double_array"/>
<parameter key="data_management" value="auto"/>
</operator>
<operator activated="true" class="generate_attributes" compatibility="9.6.000" expanded="true" height="82" name="Generate Attributes" width="90" x="179" y="85">
<list key="function_descriptions">
<parameter key="z" value="x*x"/>
</list>
<parameter key="keep_all" value="true"/>
</operator>
<operator activated="false" class="rename" compatibility="9.6.000" expanded="true" height="82" name="Rename" width="90" x="246" y="238">
<parameter key="old_name" value="x"/>
<parameter key="new_name" value="x^2"/>
<list key="rename_additional_attributes"/>
</operator>
<operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="313" y="85">
<parameter key="attribute_name" value="y"/>
<parameter key="target_role" value="label"/>
<list key="set_additional_roles">
<parameter key="x" value="regular"/>
</list>
</operator>
<operator activated="true" class="linear_regression" compatibility="9.6.000" expanded="true" height="103" name="Linear Regression" width="90" x="514" y="85">
<parameter key="feature_selection" value="none"/>
<parameter key="alpha" value="0.05"/>
<parameter key="max_iterations" value="10"/>
<parameter key="forward_alpha" value="0.05"/>
<parameter key="backward_alpha" value="0.05"/>
<parameter key="eliminate_colinear_features" value="false"/>
<parameter key="min_tolerance" value="0.05"/>
<parameter key="use_bias" value="true"/>
<parameter key="ridge" value="1.0E-8"/>
</operator>
<operator activated="false" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression" width="90" x="581" y="238">
<parameter key="max_iterations" value="5000"/>
<parameter key="replication_factor" value="2"/>
<parameter key="max_degree" value="2"/>
<parameter key="min_coefficient" value="-100.0"/>
<parameter key="max_coefficient" value="100.0"/>
<parameter key="use_local_random_seed" value="false"/>
<parameter key="local_random_seed" value="1992"/>
</operator>
<connect from_op="Read Excel" from_port="output" to_op="Generate Attributes" to_port="example set input"/>
<connect from_op="Generate Attributes" from_port="example set output" to_op="Set Role" to_port="example set input"/>
<connect from_op="Set Role" from_port="example set output" to_op="Linear Regression" to_port="training set"/>
<connect from_op="Linear Regression" from_port="model" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
I want to ask why the results of the two processes are not the same, the original data presents a quadratic nonlinear relationship, and why the quadratic expression cannot be made by polynomial regression.
Thanks you very much!


Answers
-
Hi @rookie,
Thanks for sharing the data and process.
If we have got four (4) example and train a polynomial regression, we may fail for the model. So I filled up the gap with interpolation to add more data here. Also this Polynomial regression will not perform well without the normalization...
Process attached here for your reference.<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="false" class="read_excel" compatibility="9.6.000" expanded="true" height="68" name="Read Excel" width="90" x="45" y="136"> <parameter key="excel_file" value="C:\Users\1\Desktop\question data.xlsx"/> <parameter key="sheet_selection" value="sheet number"/> <parameter key="sheet_number" value="1"/> <parameter key="imported_cell_range" value="A1"/> <parameter key="encoding" value="SYSTEM"/> <parameter key="first_row_as_names" value="true"/> <list key="annotations"/> <parameter key="date_format" value=""/> <parameter key="time_zone" value="SYSTEM"/> <parameter key="locale" value="English (United States)"/> <parameter key="read_all_values_as_polynominal" value="false"/> <list key="data_set_meta_data_information"> <parameter key="0" value="x.true.real.attribute"/> <parameter key="1" value="y.true.real.attribute"/> </list> <parameter key="read_not_matching_values_as_missings" value="false"/> <parameter key="datamanagement" value="double_array"/> <parameter key="data_management" value="auto"/> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="utility:create_exampleset" compatibility="9.6.000" expanded="true" height="68" name="Create ExampleSet" width="90" x="45" y="34"> <parameter key="generator_type" value="comma separated text"/> <parameter key="number_of_examples" value="100"/> <parameter key="use_stepsize" value="false"/> <list key="function_descriptions"/> <parameter key="add_id_attribute" value="false"/> <list key="numeric_series_configuration"/> <list key="date_series_configuration"/> <list key="date_series_configuration (interval)"/> <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/> <parameter key="time_zone" value="SYSTEM"/> <parameter key="input_csv_text" value="x,y 4194.06, 91540.07 3466.45, 109460.36 2070.08, 120338.64 874.98, 102182.19 "/> <parameter key="column_separator" value=","/> <parameter key="parse_all_as_nominal" value="false"/> <parameter key="decimal_point_character" value="."/> <parameter key="trim_attribute_names" value="true"/> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="parse_numbers" compatibility="9.6.000" expanded="true" height="82" name="Parse Numbers" width="90" x="179" y="34"> <parameter key="attribute_filter_type" value="all"/> <parameter key="attribute" value=""/> <parameter key="attributes" value=""/> <parameter key="use_except_expression" value="false"/> <parameter key="value_type" value="nominal"/> <parameter key="use_value_type_exception" value="false"/> <parameter key="except_value_type" value="file_path"/> <parameter key="block_type" value="single_value"/> <parameter key="use_block_type_exception" value="false"/> <parameter key="except_block_type" value="single_value"/> <parameter key="invert_selection" value="false"/> <parameter key="include_special_attributes" value="false"/> <parameter key="decimal_character" value="."/> <parameter key="grouped_digits" value="false"/> <parameter key="grouping_character" value=","/> <parameter key="infinity_representation" value=""/> <parameter key="unparsable_value_handling" value="fail"/> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="313" y="34"> <parameter key="attribute_name" value="y"/> <parameter key="target_role" value="label"/> <list key="set_additional_roles"> <parameter key="x" value="regular"/> </list> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="normalize" compatibility="9.6.000" expanded="true" height="103" name="Normalize" width="90" x="447" y="34"> <parameter key="return_preprocessing_model" value="false"/> <parameter key="create_view" value="false"/> <parameter key="attribute_filter_type" value="all"/> <parameter key="attribute" value=""/> <parameter key="attributes" value=""/> <parameter key="use_except_expression" value="false"/> <parameter key="value_type" value="numeric"/> <parameter key="use_value_type_exception" value="false"/> <parameter key="except_value_type" value="real"/> <parameter key="block_type" value="value_series"/> <parameter key="use_block_type_exception" value="false"/> <parameter key="except_block_type" value="value_series_end"/> <parameter key="invert_selection" value="false"/> <parameter key="include_special_attributes" value="true"/> <parameter key="method" value="Z-transformation"/> <parameter key="min" value="0.0"/> <parameter key="max" value="1.0"/> <parameter key="allow_negative_values" value="false"/> <description align="center" color="transparent" colored="false" width="126">rescale the range</description> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="subprocess" compatibility="9.6.000" expanded="true" height="82" name="Subprocess" width="90" x="581" y="34"> <process expanded="true"> <operator activated="true" class="sort" compatibility="9.6.000" expanded="true" height="82" name="Sort" width="90" x="112" y="34"> <parameter key="attribute_name" value="x"/> <parameter key="sorting_direction" value="increasing"/> </operator> <operator activated="true" class="generate_id" compatibility="9.6.000" expanded="true" height="82" name="Generate ID" width="90" x="246" y="34"> <parameter key="create_nominal_ids" value="false"/> <parameter key="offset" value="0"/> </operator> <operator activated="true" class="generate_attributes" compatibility="9.6.000" expanded="true" height="82" name="Generate Attributes" width="90" x="380" y="34"> <list key="function_descriptions"> <parameter key="INDEX" value="50*(id-1)+1"/> </list> <parameter key="keep_all" value="true"/> </operator> <operator activated="true" class="time_series:equalize_numerical_indices" compatibility="9.6.000" expanded="true" height="82" name="Equalize Numerical Indices" width="90" x="514" y="34"> <parameter key="indices_attribute" value="INDEX"/> <parameter key="equalize_method" value="number of examples and range(start,stop)"/> <parameter key="number_of_examples" value="custom"/> <parameter key="custom_number_of_examples" value="151"/> <parameter key="start_value" value="same as original data"/> <parameter key="custom_start_value" value="0.0"/> <parameter key="stop_value" value="same as original data"/> <parameter key="custom_stop_value" value="100.0"/> <parameter key="step_size_(numerical)" value="1.0"/> <parameter key="replace_type_numerical" value="linear interpolation"/> <parameter key="replace_type_nominal" value="previous value"/> <parameter key="replace_type_date_time" value="previous value"/> <parameter key="replace_value_numerical" value="0.0"/> <parameter key="replace_value_nominal" value="unknown"/> <parameter key="skip_other_missings" value="true"/> <parameter key="replace_infinity" value="true"/> <parameter key="replace_empty_strings" value="true"/> <parameter key="ensure_finite_values" value="true"/> </operator> <operator activated="true" class="select_attributes" compatibility="9.6.000" expanded="true" height="82" name="Select Attributes" width="90" x="715" y="34"> <parameter key="attribute_filter_type" value="subset"/> <parameter key="attribute" value=""/> <parameter key="attributes" value="x|y"/> <parameter key="use_except_expression" value="false"/> <parameter key="value_type" value="attribute_value"/> <parameter key="use_value_type_exception" value="false"/> <parameter key="except_value_type" value="time"/> <parameter key="block_type" value="attribute_block"/> <parameter key="use_block_type_exception" value="false"/> <parameter key="except_block_type" value="value_matrix_row_start"/> <parameter key="invert_selection" value="false"/> <parameter key="include_special_attributes" value="false"/> </operator> <connect from_port="in 1" to_op="Sort" to_port="example set input"/> <connect from_op="Sort" from_port="example set output" to_op="Generate ID" to_port="example set input"/> <connect from_op="Generate ID" from_port="example set output" to_op="Generate Attributes" to_port="example set input"/> <connect from_op="Generate Attributes" from_port="example set output" to_op="Equalize Numerical Indices" to_port="example set"/> <connect from_op="Equalize Numerical Indices" from_port="equalized example set" to_op="Select Attributes" to_port="example set input"/> <connect from_op="Select Attributes" from_port="example set output" to_port="out 1"/> <portSpacing port="source_in 1" spacing="0"/> <portSpacing port="source_in 2" spacing="0"/> <portSpacing port="sink_out 1" spacing="0"/> <portSpacing port="sink_out 2" spacing="0"/> </process> <description align="center" color="transparent" colored="false" width="126">add more points to the training set</description> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="multiply" compatibility="9.6.000" expanded="true" height="103" name="Multiply" width="90" x="715" y="34"/> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression (2)" width="90" x="849" y="238"> <parameter key="max_iterations" value="280"/> <parameter key="replication_factor" value="2"/> <parameter key="max_degree" value="2"/> <parameter key="min_coefficient" value="-10.0"/> <parameter key="max_coefficient" value="10.0"/> <parameter key="use_local_random_seed" value="false"/> <parameter key="local_random_seed" value="23"/> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="generate_attributes" compatibility="9.6.000" expanded="true" height="82" name="Generate Attributes (2)" width="90" x="849" y="34"> <list key="function_descriptions"> <parameter key="z" value="x^2"/> </list> <parameter key="keep_all" value="true"/> <description align="center" color="transparent" colored="false" width="126">get quadratic term x^2</description> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="false" class="concurrency:optimize_parameters_grid" compatibility="9.6.000" expanded="true" height="145" name="Optimize Parameters (Grid)" width="90" x="648" y="289"> <list key="parameters"> <parameter key="Polynomial Regression.max_iterations" value="[1.0;300;30;linear]"/> <parameter key="Polynomial Regression.max_degree" value="[1.0;2;3;linear]"/> <parameter key="Polynomial Regression.replication_factor" value="[1.0;2;3;linear]"/> </list> <parameter key="error_handling" value="fail on error"/> <parameter key="log_performance" value="true"/> <parameter key="log_all_criteria" value="false"/> <parameter key="synchronize" value="false"/> <parameter key="enable_parallel_execution" value="true"/> <process expanded="true"> <operator activated="true" class="concurrency:cross_validation" compatibility="9.6.000" expanded="true" height="145" name="Cross Validation" width="90" x="246" y="34"> <parameter key="split_on_batch_attribute" value="false"/> <parameter key="leave_one_out" value="false"/> <parameter key="number_of_folds" value="5"/> <parameter key="sampling_type" value="automatic"/> <parameter key="use_local_random_seed" value="false"/> <parameter key="local_random_seed" value="1992"/> <parameter key="enable_parallel_execution" value="true"/> <process expanded="true"> <operator activated="true" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression" width="90" x="112" y="34"> <parameter key="max_iterations" value="50"/> <parameter key="replication_factor" value="5"/> <parameter key="max_degree" value="5"/> <parameter key="min_coefficient" value="-10.0"/> <parameter key="max_coefficient" value="10.0"/> <parameter key="use_local_random_seed" value="false"/> <parameter key="local_random_seed" value="23"/> </operator> <connect from_port="training set" to_op="Polynomial Regression" to_port="training set"/> <connect from_op="Polynomial Regression" from_port="model" to_port="model"/> <portSpacing port="source_training set" spacing="0"/> <portSpacing port="sink_model" spacing="0"/> <portSpacing port="sink_through 1" spacing="0"/> </process> <process expanded="true"> <operator activated="true" class="apply_model" compatibility="9.6.000" expanded="true" height="82" name="Apply Model" width="90" x="45" y="34"> <list key="application_parameters"/> <parameter key="create_view" value="false"/> </operator> <operator activated="true" class="performance_regression" compatibility="9.6.000" expanded="true" height="82" name="Performance" width="90" x="179" y="34"> <parameter key="main_criterion" value="first"/> <parameter key="root_mean_squared_error" value="true"/> <parameter key="absolute_error" value="false"/> <parameter key="relative_error" value="false"/> <parameter key="relative_error_lenient" value="false"/> <parameter key="relative_error_strict" value="false"/> <parameter key="normalized_absolute_error" value="false"/> <parameter key="root_relative_squared_error" value="false"/> <parameter key="squared_error" value="false"/> <parameter key="correlation" value="false"/> <parameter key="squared_correlation" value="false"/> <parameter key="prediction_average" value="false"/> <parameter key="spearman_rho" value="false"/> <parameter key="kendall_tau" value="false"/> <parameter key="skip_undefined_labels" value="true"/> <parameter key="use_example_weights" value="true"/> </operator> <connect from_port="model" to_op="Apply Model" to_port="model"/> <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/> <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/> <connect from_op="Performance" from_port="performance" to_port="performance 1"/> <portSpacing port="source_model" spacing="0"/> <portSpacing port="source_test set" spacing="0"/> <portSpacing port="source_through 1" spacing="0"/> <portSpacing port="sink_test set results" spacing="0"/> <portSpacing port="sink_performance 1" spacing="0"/> <portSpacing port="sink_performance 2" spacing="0"/> </process> </operator> <connect from_port="input 1" to_op="Cross Validation" to_port="example set"/> <connect from_op="Cross Validation" from_port="model" to_port="model"/> <connect from_op="Cross Validation" from_port="example set" to_port="output 1"/> <connect from_op="Cross Validation" from_port="performance 1" to_port="performance"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="source_input 2" spacing="0"/> <portSpacing port="sink_performance" spacing="0"/> <portSpacing port="sink_model" spacing="0"/> <portSpacing port="sink_output 1" spacing="0"/> <portSpacing port="sink_output 2" spacing="0"/> </process> </operator> </process> <?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <operator activated="true" class="h2o:generalized_linear_model" compatibility="9.3.001" expanded="true" height="124" name="Generalized Linear Model" width="90" x="983" y="34"> <parameter key="family" value="AUTO"/> <parameter key="link" value="family_default"/> <parameter key="solver" value="AUTO"/> <parameter key="reproducible" value="false"/> <parameter key="maximum_number_of_threads" value="4"/> <parameter key="use_regularization" value="false"/> <parameter key="lambda_search" value="false"/> <parameter key="number_of_lambdas" value="0"/> <parameter key="lambda_min_ratio" value="0.0"/> <parameter key="early_stopping" value="true"/> <parameter key="stopping_rounds" value="3"/> <parameter key="stopping_tolerance" value="0.001"/> <parameter key="standardize" value="false"/> <parameter key="non-negative_coefficients" value="false"/> <parameter key="add_intercept" value="true"/> <parameter key="compute_p-values" value="false"/> <parameter key="remove_collinear_columns" value="false"/> <parameter key="missing_values_handling" value="MeanImputation"/> <parameter key="max_iterations" value="0"/> <parameter key="specify_beta_constraints" value="false"/> <list key="beta_constraints"/> <parameter key="max_runtime_seconds" value="0"/> <list key="expert_parameters"/> </operator> </process>
These two models are close but I can not guarantee the polynomial will output similar coefficient without normalization
I would strongly suggest to use GLM with new attribute manually created or attributes from Auto Feature Engineer.
Happy Rapid-Mining and Stay Healthy!
YY
5 -
First of all, thank you for your answer
. According to your description, I am as the data is too little, and not standardized, to lead to the results out? But these four samples are real data , need the four data to construct a yuan quadratic polynomial, Because nonlinear equations can be converted to linear equations , so I use z instead of x2, I have the linear regression equation. But why do with polynomial regression is not to come out, how do you explain that please?Polynomial regression is there any limit to this operator ?
0 -
hi @yyhuang,
Sorry in advance, I don't know how to use the function of this forum.That's why it took so long to reply
First of all, thank you for your answer. According to your description, I am as the data is too little, and not standardized, to lead to the results out? But these four samples are real data , need the four data to construct a yuan quadratic polynomial, Because nonlinear equations can be converted to linear equations , so I use z instead of x2, I have the linear regression equation. But why do with polynomial regression is not to come out, how do you explain that please?Polynomial regression is there any limit to this operator ?
0