"Linear regression beats ANN"

Question

Hi,
i have a dataset consisting of 1000 samples and 19 attributes. The data is housing data (living area, presence of heating, bath, neighborhood characteristics, etc). The target value is house price. The dataset has 8 binary attributes.
If i apply linear regression to this dataset the results are far superior to an ANN, although from my understanding the data is to complex for linear regression.
Also decission trees and SVM are inferior to linear regression.
Have you some advice, how i can validate the results and check why linear regression is that good?

Thank you very much.

fikio · Answer

chaosbringer  wrote:
Hi,
i have a dataset consisting of 1000 samples and 19 attributes. The data is housing data (living area, presence of heating, bath, neighborhood characteristics, etc). The target value is house price. The dataset has 8 binary attributes.
If i apply linear regression to this dataset the results are far superior to an ANN, although from my understanding the data is to complex for linear regression.
Also decission trees and SVM are inferior to linear regression.
Have you some advice, how i can validate the results and check why linear regression is that good?

Thank you very much.

I come from a statistical background, to clarify, your dataset has 1000 observations and 19 variables, 8 of which are binary? Why do you believe that the data is too complex for linear regression, have you looked at variables univariately with a scatterplot or done some modeling to determine that there are nonlinear relationships?

I am used to evaluating models with AUCs, what numbers are you getting? Is MSQE the mean squared error?

chaosbringer · Answer

Hi,
tank you, that helped. Fiddling with the parameters improved the situation significantly.
However, another problem raises:
T-Test says, that the means are the same (p=1,0).
If i test-wise modify the parameters of the neural net to produce a realy bad result, the t-test still return 1.
How can it be, that the t-test returns 1, even though the RMSEs are very different (0.5 vs 0.34)?

Thank you

wessel · Answer

Make a convergence plot.
E.g. measure the RMSE at every iteration.
Maybe you need to train your network for many more iterations.
With 19 inputs, your network gets very big, very fast, so you have lots of weights to optimize.
An alternative problem could be premature convergence, e.g. getting stuck in local optima.

Best regards,

Wessel