[Delayed] Neural net shuffles elements between clusters

T-Unit
T-Unit New Altair Community Member
edited November 5 in Community Q&A
Hello,

i face the following problem:

I did some clustering and now i have about 1.700 data sets that belong to serveral clusters (cluster_0, cluster_1, ..., cluster_18).
I have additional 44 data sets that should by classified. To classify them a neural net should learn the 1.700 data sets above (cluser is the label-attribute). The neural net works good, so far but there is a major problem: similar elements are grouped into the same cluster, but the cluster itself seems to be the false one. To verify if this is a general problem, i told the neural net to learn from the great data set (1.700 examples) and classify the same 1.700 elements when the net was constructed.

Example:
The training data may be as followed (capital letters represent elements):
cluster_0: A, B, C, D
cluster_10: E, F, G, H
cluster_15: I, J, K, L

When the generated model of the neural net is applied on the same data that were used to train the net the results are for example:
cluster_0:E, F, G, H
cluster_10:  I, J, K, L
cluster_15: A, B, C, D

... the elements are grouped together (fine!) but not into the right group (not fine!).

Anyone knows how to solve this problem?

Greetings,
Thomas

PS: I would like to post the process i use, but my message would exeed the maximum of 20000 cahrakters. Is the whole process needed or should only some parts of the process do it?

Answers

  • MariusHelf
    MariusHelf New Altair Community Member
    Hi Thomas,

    that is actually a bug, and I admit that this is an exceptionally nasty one. I already have an idea of the cause, so no need to post your process setup. We will include a fix in one of the next releases.
    Unfortunately, I can't think of any workaround currently, so the only thing I can suggest until the bug is fixed is to try another classification algorithm.

    Best regards,
    Marius
  • MariusHelf
    MariusHelf New Altair Community Member
    Unfortunately, I can't reproduce the error. Which version of RapidMiner are you using? Does this still happen in the latest version, RapidMiner 5.3? If so, I would need your process setup. Please cut your process as much as is possible while still reproducing the error before posting.

    Best regards,
    Marius
  • T-Unit
    T-Unit New Altair Community Member
    Hello Marius,

    thanks for your reply.

    Because i'm running out of time with my project i tried a lot of process-modifications to yesterdays model. In fact i have now round about 30 new processes and i guess, that i unfortunately have overwritten the relevant process, but i will have a closer look to my files when there is more time to do so.

    Also i will try to cut the process, because yesterdays realized "bug" was found when i uses RM 5.3.

    I set the threads status to "delayed" until i had a closer look to my files.

    Thanks anyway for reading and replying my questions

    Greetings,
    Thomas

    PS: Because it was very urgent i tried the "decisiontree" instead of "neural net" for the classification of the unseen data and - as far as i see - it works fine.
  • rapidnewbe
    rapidnewbe New Altair Community Member
    Seems like I have the same problem. I use the current version 3.5.

    Link: http://rapid-i.com/rapidforum/index.php/topic,6231.msg21807.html#msg21807

    I posted my process so probably you can reproduce this bug with this information.

    By now it helps a lot to know that this is a bug. If you find a workarround, please post it. I am still interested in using the neural net classification.