"Insufficient results with M5P regression tree"

New Altair Community Member
Updated by Jocelyn
Hello,
if anyone is interested please try the following:
produce a file containing two columns
x = 0, 0.1, 0.2, ..., 12.6;
y = sin(x)
Then apply M5P (with or without normalization).
The result is quite disappointing. Does anyone know how to get an acceptable result?
I expected to get something like a picewise linear approximation of the sin function,
but got something far away from this.
Thank You.
if anyone is interested please try the following:
produce a file containing two columns
x = 0, 0.1, 0.2, ..., 12.6;
y = sin(x)
Then apply M5P (with or without normalization).
The result is quite disappointing. Does anyone know how to get an acceptable result?
I expected to get something like a picewise linear approximation of the sin function,
but got something far away from this.
Thank You.
Find more posts tagged with
Sort by:
1 - 6 of
61
Hi,
sorry, here is the XML
<operator name="Root" class="Process" expanded="yes">
<operator name="ExampleSource" class="ExampleSource">
<parameter key="attributes" value="C:\Programme\Rapid-I\RapidMiner-4.4\sinus"/>
</operator>
<operator name="Normalization" class="Normalization">
</operator>
<operator name="W-M5P" class="W-M5P">
<parameter key="keep_example_set" value="true"/>
<parameter key="U" value="true"/>
<parameter key="M" value="10.0"/>
</operator>
<operator name="ModelApplier" class="ModelApplier">
<list key="application_parameters">
</list>
</operator>
</operator>
What I get is a piecewise constant result, i.e. the leafes of the tree are: y = const
Only the last leaf gives a linear model: y = 3.2196 * x - 4.5545
If I had such a "really" linear model at all leafes of the tree, it would be ok, i.e. as
I would expect it.
There are no settings which can improve it, even if the tree could result in y = a*x+b
in each leaf, which should give a better prediction. So why does'nt M5P behave like
this?
If I select the smoothed tree the results are even worse.
I hope I could make my "problem" more clear to you.
P.S.:
Maybe if you google for "stepwise regression tree HUANG" or go directly to
http://www.landcover.org/pdf/ijrs24_p75.pdf
and there at page 77 (i.e. page 3 in the 16 pages document) you see what I
mean. If this SRT algorithm would become a part of RapidMiner I would
appreciate it
, even if I don't understand why M5P doesn't behave comparable.
sorry, here is the XML
<operator name="Root" class="Process" expanded="yes">
<operator name="ExampleSource" class="ExampleSource">
<parameter key="attributes" value="C:\Programme\Rapid-I\RapidMiner-4.4\sinus"/>
</operator>
<operator name="Normalization" class="Normalization">
</operator>
<operator name="W-M5P" class="W-M5P">
<parameter key="keep_example_set" value="true"/>
<parameter key="U" value="true"/>
<parameter key="M" value="10.0"/>
</operator>
<operator name="ModelApplier" class="ModelApplier">
<list key="application_parameters">
</list>
</operator>
</operator>
What I get is a piecewise constant result, i.e. the leafes of the tree are: y = const
Only the last leaf gives a linear model: y = 3.2196 * x - 4.5545
If I had such a "really" linear model at all leafes of the tree, it would be ok, i.e. as
I would expect it.
There are no settings which can improve it, even if the tree could result in y = a*x+b
in each leaf, which should give a better prediction. So why does'nt M5P behave like
this?
If I select the smoothed tree the results are even worse.
I hope I could make my "problem" more clear to you.
P.S.:
Maybe if you google for "stepwise regression tree HUANG" or go directly to
http://www.landcover.org/pdf/ijrs24_p75.pdf
and there at page 77 (i.e. page 3 in the 16 pages document) you see what I
mean. If this SRT algorithm would become a part of RapidMiner I would
appreciate it

I think you're getting into trouble because of the value of M (mininum # values per leaf), and the cyclical nature of the data. If I take your process flow, change M from 10 to 5, I get linear models for nodes 1, 5, 6, 7, 10, 11, 14, 15, 16, and 20, and constant values elsewhere.
Ok, I see the difference.
Nevertheless, I cannot understand, why the fraction of constant leafs, i.e. y = const, increases if I change M from 5 to 6.
I get 10 constant leafes more at positions where y = a*x+b would be better. Isn't the result with a constant regression
worse than a non constant regression in the leafs?
It's clear to me that thealgorithm is from Weka and not RapidMiner, so You cannot know in detail what happens.
Nevertheless, I only want to understand, why, by increasing M, the number of constant leafs increases even
if it worses the result.
By the way, if you are an expert
, would it be possible to post a workflow for optimizing the parameters automatically.
Up to now I didn't get the right feeling for applying meta methods like grid search or x-validation in the right way.
Thank's in advance. (At least I need an answer on my question, the workflow would be nice)
Nevertheless, I cannot understand, why the fraction of constant leafs, i.e. y = const, increases if I change M from 5 to 6.
I get 10 constant leafes more at positions where y = a*x+b would be better. Isn't the result with a constant regression
worse than a non constant regression in the leafs?
It's clear to me that thealgorithm is from Weka and not RapidMiner, so You cannot know in detail what happens.
Nevertheless, I only want to understand, why, by increasing M, the number of constant leafs increases even
if it worses the result.
By the way, if you are an expert

Up to now I didn't get the right feeling for applying meta methods like grid search or x-validation in the right way.
Thank's in advance. (At least I need an answer on my question, the workflow would be nice)
I'm far from an expert with RM. :-) And I've never used M5P before now, so what little I know it just came from a little experimentation and Googling yesterday. The paper describing the method seems to be available at http://www.cs.waikato.ac.nz/pubs/wp/1996/uow-cs-wp-1996-23.pdf, and it may answer that question. My guess is that there's some kind of rule that if the slope of the regression model is too close to zero, it gets rounded off to zero. Maybe it's an interaction between the number of observations in each node and the regressed slope. Beyond that, you'd probably have more luck getting an answer from a Weka list or forum
As for the parameter optimization, take a look at 07_Meta/01_ParameterOptimization.xml in the RM samples directory. The GridParameterOptimization node is where you'd specify what parameters you want to tinker with.
As for the parameter optimization, take a look at 07_Meta/01_ParameterOptimization.xml in the RM samples directory. The GridParameterOptimization node is where you'd specify what parameters you want to tinker with.
I just tried the following process, and the only changes from the default settings are to click the check box for parameters N, U, and R : And the plot of x vs. prediction(y) looks, to my eyes, much more sin-like. But I don't know if using an unpruned, unsmoothed learner makes sense for your problem.
Keith