"Testing data differs to Training - Neural networks better than Decision Trees?"

ruser
ruser New Altair Community Member
edited November 5 in Community Q&A
As I'm aware, Decision tree is the worst choice for the learner, if the testing/validation data set differs considerably from the Training data set. But I hear that the Neural networks is relatively more robust even in such cases.

What is that special property of the 'Neural Networks' which makes it more accurate in such cases? Please explain.
Tagged:

Answers

  • land
    land New Altair Community Member
    Hi,
    I will not go into the deep statistical properties of both learner. If you want that, buy a book or book one of our courses.
    Only so much: The usage of neural net works seems to be sometimes a religion, so it will solve everything for the follower of the neural net. Take their claims with caution.
    I personally don't see there any reason, why the NN should cope with the different data better than the Decision Tree IN GENERAL. This depends on parmeter settings, data distribution and so on...

    One small hint: It would make me chattier, if you ask your questions in a polite way. You don't pay us and so a short command like "Please explain." does not motivate me very much...

    Greetings,
      Sebastian
  • BAMBAMBAM
    BAMBAMBAM New Altair Community Member
    I second what Sebastian said.  If you'd like to understand more about the difference between performance on training versus testing data, look at:

    http://en.wikipedia.org/wiki/Overfitting

    and possibly:
    http://en.wikipedia.org/wiki/Overfitting

    Also there is a concept in search and optimization called "No Free Lunch" or "TANSTAAFL" that is somewhat more related to your question:
    http://en.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization.

    There is a link from that article to a publication that should at least address your question more directly:
    ^ Wolpert, David (1996), "“The Lack of A Priori Distinctions between Learning Algorithms," Neural Computation, pp. 1341-1390.

    but I'm unfamiliar with it and so pass the torch back to ruser...

    hope this helps,
    John
  • ruser
    ruser New Altair Community Member
    Hi Sebastian,
    I understand your concern. I'll correct that from next time onwards.
    I know it requires lot of patience and dedication to take time and answer others' questions. I appreciate that.
    I owe lot of thanks to you guys.

    regards

Categories