"Problems with generating Association Rules"
riddler16
New Altair Community Member
Hi all i´m new here and i have a question about the FP-Algorithms, Association Rules and espacially about buffering 
My whole RM process is very simple. I read about 2000 transactions from a csv file. Then there is a preprocessing step with a nominal2binominal operator. The output goes to the fp-growth and after fp-growth to the AssociationRulesGenerator.
Everything works fine...till i choose a low support. (about 0.2).
I get the error message that i have to less memory and that i should update it or try the cached database operator. I have already tried it with more then 10GB Ram and still the same
My question is, if there is an chance to get these rules? Maybe by caching the results or writing into a file or something like this?
The Problem is not the fp-growth algorithm. At least i think so, because the status bar on the bottom of RM tells me after some seconds, that the fp-growth calculation is over and now the generation of the association rules beginns
Here is the xml-file:
My whole RM process is very simple. I read about 2000 transactions from a csv file. Then there is a preprocessing step with a nominal2binominal operator. The output goes to the fp-growth and after fp-growth to the AssociationRulesGenerator.
Everything works fine...till i choose a low support. (about 0.2).
I get the error message that i have to less memory and that i should update it or try the cached database operator. I have already tried it with more then 10GB Ram and still the same
My question is, if there is an chance to get these rules? Maybe by caching the results or writing into a file or something like this?
The Problem is not the fp-growth algorithm. At least i think so, because the status bar on the bottom of RM tells me after some seconds, that the fp-growth calculation is over and now the generation of the association rules beginns
Here is the xml-file:
Thanks a lot for any ideas.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.0.0" expanded="true" name="Root">
<parameter key="logverbosity" value="warning"/>
<process expanded="true" height="584" width="918">
<operator activated="true" class="read_csv" compatibility="5.0.10" expanded="true" height="60" name="Read CSV" width="90" x="29" y="23">
<parameter key="file_name" value="\output.csv"/>
<parameter key="encoding" value="windows-1252"/>
<parameter key="skip_comments" value="false"/>
<parameter key="use_first_row_as_attribute_names" value="false"/>
<list key="data_set_meta_data_information">
<parameter key="0" value="attribute_0.true.1.regular"/>
...
<parameter key="30" value="attribute_30.false.1.regular"/>
</list>
<parameter key="attribute_names_already_defined" value="true"/>
</operator>
<operator activated="true" class="subprocess" compatibility="5.0.0" expanded="true" height="76" name="Preprocessing" width="90" x="180" y="30">
<process expanded="true" height="605" width="950">
<operator activated="true" class="nominal_to_binominal" compatibility="5.0.0" expanded="true" height="94" name="Nominal2Binominal" width="90" x="497" y="30">
<parameter key="transform_binominal" value="true"/>
</operator>
<connect from_port="in 1" to_op="Nominal2Binominal" to_port="example set input"/>
<connect from_op="Nominal2Binominal" from_port="example set output" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="fp_growth" compatibility="5.0.0" expanded="true" height="76" name="FPGrowth" width="90" x="313" y="30">
<parameter key="find_min_number_of_itemsets" value="false"/>
<parameter key="min_support" value="0.1"/>
</operator>
<operator activated="true" class="create_association_rules" compatibility="5.0.0" expanded="true" height="60" name="AssociationRuleGenerator" width="90" x="447" y="75">
<parameter key="min_confidence" value="0.7"/>
</operator>
<connect from_op="Read CSV" from_port="output" to_op="Preprocessing" to_port="in 1"/>
<connect from_op="Preprocessing" from_port="out 1" to_op="FPGrowth" to_port="example set"/>
<connect from_op="FPGrowth" from_port="example set" to_port="result 2"/>
<connect from_op="FPGrowth" from_port="frequent sets" to_op="AssociationRuleGenerator" to_port="item sets"/>
<connect from_op="AssociationRuleGenerator" from_port="rules" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="36"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
</process>
</operator>
</process>
Tagged:
0
Answers
-
Hey,
I just recognised that the problem is not the association rules generator. It is the WriteIntoFile-Operator. I just run the same configuration but without any output and it seems to work.
So now i'm searching for any other possiblities to manage this problem because of course i need the output
The problem is that i don´t know if the file size is getting to big for the OS or the filesystem have to less memory to save this file.
Maybe somebody here has a suggestion how to deal with it?
0