SOM Reduction - Preprocessing Save doesn't work

Legacy User
Legacy User New Altair Community Member
edited November 5 in Community Q&A
When I attempt to create and save a preprocessing model with 205 features ( a mix of nominal and numeric features ) with about 10,000 training rows, after processing for about a hour, the modelwrite writes out about a 12 meg model, then hangs and freezes. Any attempts to use this model results in errors. Is this a size or complexity limitation?
Tagged:

Answers

  • TobiasMalbrecht
    TobiasMalbrecht New Altair Community Member
    Hi,

    which preprocessing model did you create and save? Could you please post your process XML?! Given your description, I have no idea what could be the problem here.

    Regards,
    Tobias
  • land
    land New Altair Community Member
    Hi,
    since the SOMReduction should save the generated SOM, there should be no complexity problem with the number of examples. (Ignoring time needed to compute the SOM)
    How long did it freeze, before it terminated the process? It might be, that saving might last a few minutes.

    A quoting of the error messages would be usefull, too.


    Greetings,
      Sebastian
  • Legacy User
    Legacy User New Altair Community Member
    Here's the error message when trying to load the model created from the SOM preprocessing

    Exception: com.rapidminer.operator.UserError
    Message: Could not read file 'F:\ModelDevelopment\preprocess_som.mod': Cannot read from XML stream, wrong format: Could not call java.security.CodeSource.readObject()
    ---- Debugging information ----
    message            : Could not call java.security.CodeSource.readObject()
    cause-exception    : java.lang.RuntimeException
    cause-message      : null
    class              : com.rapidminer.operator.ContainerModel
    required-type      : java.security.CodeSource
    line number        : 2224478
    -------------------------------.
    Stack trace:

      com.rapidminer.operator.io.ModelLoader.apply(ModelLoader.java:97)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.Process.run(Process.java:571)
      com.rapidminer.Process.run(Process.java:541)
      com.rapidminer.Process.run(Process.java:531)
      com.rapidminer.gui.ProcessThread.run(ProcessThread.java:61)

    Cause
    Exception: java.io.StreamCorruptedException
    Message: invalid stream header: 1F8B0800
    Stack trace:

      java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
      java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
      com.rapidminer.operator.io.ModelLoader.apply(ModelLoader.java:76)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
      com.rapidminer.operator.Operator.apply(Operator.java:656)
      com.rapidminer.Process.run(Process.java:571)
      com.rapidminer.Process.run(Process.java:541)
      com.rapidminer.Process.run(Process.java:531)
      com.rapidminer.gui.ProcessThread.run(ProcessThread.java:61)
  • land
    land New Altair Community Member
    Hi,
    I just found your bug. Funny one :) Since the SOM was a thread itself, the IOObjectWriter tried to save nearly everything. Complete rapidMiner, libraries, everything, since the thread knew its threadpool, which probably knew all threads including the gui...And when the IOObjectWriter started to save itself, it hang up. What was probably a good idea after all :)

    In the newest repository version it now works.

    Greetings,
      Sebastian
  • Legacy User
    Legacy User New Altair Community Member
    Great. How or where do I access the repository version? I assume it's CVS somewhere, and is the repository built into binaries or do I have to compile ... ?
  • TobiasMalbrecht
    TobiasMalbrecht New Altair Community Member
    Hi,

    http://rapid-i.com/content/view/25/48/lang,de/

    explains the anonymous CVS access to our developer version via Eclipse. The developer CVS repository naturally only contains the source code. The compilation or running of RapidMiner, respectively, is pretty easy using Eclipse. Just follow the instructions which are explained on the aforementioned website.

    Regards,
    Tobias
  • Legacy User
    Legacy User New Altair Community Member
    Checked it out with eclipse. Compile worked great, including the fix on the SOM preprocessing saving/loading issue. Thanks.

Categories