Save results of operators which have a long runtime in one process

User: "MoWei"
New Altair Community Member
Updated by Jocelyn

Hello everyone,

I have a large data set with about 10000 examples and about 40 attributes. There are only numeric attributes (real and integer). I used the „Weight by SVM“ operator to weight the attribute and afterwards I took the „Select by Weight“ operator to continue with the top ten attributes. Now I want extend the process to predict the label attribute. So I have to try different operators like Decision Tree an so on. The problem is every time I start to run the process the „Weight by SVM“ need about 20 minutes so that I have to wait a lot of time if I run the process every time from the beginning.

Now the question: What is the best way to save the results of the „Weight by SVM“ operator? At the moment I just want to change operators after I used the „Weight by SVM“ and „Select by Weight“ operator so that the selected attributes for prediction are always the same. 

My solution at the moment: I select the attributes and store the reduced data set in one process and in an other process I retrieve the reduced data set and try to predict the label in there.

Is it somehow possible to put all the operations in one process without waiting a lot of time while the „Weight by SVM“ is running? Cache or something like that?

Thanks you very much.

Best regards

Moritz

Find more posts tagged with

Sort by:
1 - 2 of 21
    User: "varunm1"
    New Altair Community Member
    Accepted Answer
    Hello @MoWei

    I got your point, you want to see the process as it is but somehow the process should automatically skip the enables SVM weights operator if its already run. 

    I don't think this is possible if you want it in the same process. From my understanding, enabling and disabling operator which you are currently using is the way to skip an operator in the current process.

    I think you are already aware of creating multiple processes (one with data preprocessing and other with running models). This is similar to the to your second image but with two processes, as it stores the output of one process and retrieves in other.

    @yyhuang is there any way of skipping an operator while it's enabled?
    User: "YYH"
    Altair Employee
    Accepted Answer
    If the operator is enabled but in a "select subprocess" branch, it is possible to be skipped.

    You can manually select the branch ID, doing nothing or enable the "weight by"...

    HTH.