"Interpretation of attribute weights"

Question

I've been using EvolutionaryWeighting to optimize attribute weights with the NearestNeighbors learner, and I'm hoping to get some clarity on how to interpret the values of attribute weights.

(For my processes, I have a numerical label, all the numerical attributes have been normalized, and there are one or two polynominal attributes.)

1) I've tried running it both with bounded_mutation=TRUE (which constrains weights to between 0 and 1) and bounded_mutation=FALSE, where weights can vary freely.  Is bounded_mutation=TRUE essentially just a rescaling of the attribute weights to fit in the [0,1] interval (so they are functionally equivalent)?  Or is there something other side effect of this setting that I should be aware of?

2) WIth bounded_mutation=FALSE, the weights can become negative.  Does the sign of the weight value have any particular meaning (e.g. does that mean the attribute is negatively correlated with the label)?  Or is it just the absolute value of the weight that indicates its importance?

3) With bounded_mutation=TRUE, it appears to be the case, at least after several generations, that there is an attribute with weight=1, and one with weight=0.  Does the weight=0 imply that the attribute is not used at all in computing the neighbors?

Thanks,
Keith

land · Answer

Hi guys,
bounded really means bounded :)  During the evolutionary optimization process each individum is restricted to values between 0 and 1. This might cause different results than an unbounded mutation, since values can be inverted (5 becomes -5 if weight is - 1). This might improve performance for some learners, since a separating boundary gets wider.
Normalization destroys this effect, since only the absolut values are used.

Greetings,
  Sebastian

Jason_2 · Answer

Hi,

Any thoughts on this? I am experiencing the same issue

Thanks,
Jason

keith · Answer

Following up my own question... Some further investigation has shown that it's normalize_weights, not bounded_mutation that does the conversion to the 0-1 scale. Sorry for that misunderstanding. However, I am still a little confused, then, on the difference between normalize_weights and bounded_mutation, and how they interact. Consider the following process: If I run this process twice, once with normalize_weights=TRUE, once with normalize_weights=FALSE, I get two sets of weights that clearly correspond to each other (the normalize_weights=TRUE values are norm_wgt = (abs(Wgt)--Max(abs(wgt)) / (Max(abs(wgt)) - Min(abs(wgt))). But if I run it with bounded_mutations=TRUE (and normalize_weights=FALSE), the weights aren't even in the same rank order as the previous runs: Attrib A B C att1 0.156 0.000 0.555 att2 7.666 0.998 0.759 att3 -7.684 1.000 0.104 att4 -0.929 0.103 0.224 att5 -3.942 0.503 0.246 A: Normalize_weights=FALSE B: Normalize_weights=TRUE C: Bounded_mutations=TRUE (Norm_wgts=False) Something's obviously different is happening with bounded_mutations, but I don't understand enough of what's being done behind the scenes to really interpret these results. Can someone help? Thanks, Keith