Polynomial regression with mixed terms

michaelhecht

I would appreciate if the polynomial regression would be able to apply also mixed terms, i.e.
if there are attributes like X1,X2,X3 and a label Y and I specify a second order polynomial
that I could get

Y = a * X1 + b * X2 + c * X3 + d * X1*X2 + e * X2*X3 + f * X1*X3 + g * X1^2 + h * X2^2 + i * X3^2

maybe with a kind of optimization of used terms according to an implicite cross validation,
since the application of all possible mixed terms could "explode" the size of the polynomial,
if the size of attribute is large. On the other hand maybe one could specify the maximum number
of mixed terms, e.g. to 3 which means that intermixing is only allowed for 3 attributes. With higher
order polynomials one would get terms like: a * X1 * X4^3 * X5^2.

land

Hi,
as you said, the invention of all mixed terms would let the problem explode. So you would have to do some sort of feature selection internally, validation is needed of course, too.
I personally believe, that this is the wrong way, because you loose so much of controll. It's a simple approach but you could make everything you want to have done internally just inside your process. Use a feature generation providing multiplications, perhabs a genetic algorithm, just as you like, together with a validation over a linear regression and you have everything you want...

Greetings,
  Sebastian