Inconsistent results
I'm starting to use RapidMiner after using Weka and there's one thing that is driving me crazy.
I'm using a simple dataset:
http://cs.uns.edu.ar/~cic/dm2007/downloads/datasets/titanic.arff
Four nominal columns: passenger class (4 classes), age (adult or child), sex (man or woman), survived? (yes or no).
I store that data in a repository and create a very simple process: retrieve the data and apply W-Apriori operator with default parameters. No rules found message.
I don't understand how is working. In Weka GUI with the same parameters, rules are not only found but also they are right. If I insert an operator (with a multiplier) before W-Apriori and write the input to a arff file, in order to ensure what input is getting W-Apriori operator, and run Weka with that file, the same rules are obtained. I've tested with other datasets, and they aren't working either.
If I switch the algorithm to W-FPGrowth, results are consistent. Same results in Weka and in RapidMiner. I also did try to look inside of Weka plugin. I did a binary comparison between my Weka.jar and classes in the plugin. They matched completely. So I'm using exactly the same Weka version.
Does anybody know what I'm doing wrong?
I know that RapidMiner have other own operators for association rules, but all my team is working with Weka, and my first step to convince them to switch to RapidMiner is showing that all we do in Weka is directly possible in RapidMiner.
Thanks in advance!
I'm using a simple dataset:
http://cs.uns.edu.ar/~cic/dm2007/downloads/datasets/titanic.arff
Four nominal columns: passenger class (4 classes), age (adult or child), sex (man or woman), survived? (yes or no).
I store that data in a repository and create a very simple process: retrieve the data and apply W-Apriori operator with default parameters. No rules found message.
I don't understand how is working. In Weka GUI with the same parameters, rules are not only found but also they are right. If I insert an operator (with a multiplier) before W-Apriori and write the input to a arff file, in order to ensure what input is getting W-Apriori operator, and run Weka with that file, the same rules are obtained. I've tested with other datasets, and they aren't working either.
If I switch the algorithm to W-FPGrowth, results are consistent. Same results in Weka and in RapidMiner. I also did try to look inside of Weka plugin. I did a binary comparison between my Weka.jar and classes in the plugin. They matched completely. So I'm using exactly the same Weka version.
Does anybody know what I'm doing wrong?
I know that RapidMiner have other own operators for association rules, but all my team is working with Weka, and my first step to convince them to switch to RapidMiner is showing that all we do in Weka is directly possible in RapidMiner.
Thanks in advance!
Find more posts tagged with
Sort by:
1 - 7 of
71
New Altair Community Member
OPThanks for your quick reply, Marius.
Sorry for forgetting those important details:
OS X 10.8
Apple Java 1.6.0_51
RapidMiner 5.3.008
Weka Extension 5.3.1 (which includes Weka 3.6.9)
Weka Standalone 3.6.9
I've also tried with other Weka Extension versions with no luck.
Please, tell me if you need more details. I'm very interested in migrating my work group to RapidMiner. I find RapidMiner very useful!
Thanks again.
Sorry for forgetting those important details:
OS X 10.8
Apple Java 1.6.0_51
RapidMiner 5.3.008
Weka Extension 5.3.1 (which includes Weka 3.6.9)
Weka Standalone 3.6.9
I've also tried with other Weka Extension versions with no luck.
Please, tell me if you need more details. I'm very interested in migrating my work group to RapidMiner. I find RapidMiner very useful!
Thanks again.
New Altair Community Member
Actually I can't reproduce a "no rules found" message. Where is it shown?
What I do observe is that the results of the W-Apriori are not shown due to a bug that will probably be fixed in the release of RapidMiner.
Best regards,
Marius
What I do observe is that the results of the W-Apriori are not shown due to a bug that will probably be fixed in the release of RapidMiner.
Best regards,
Marius
New Altair Community Member
OPI was aware about the bug you mentioned. Because of that, I was writing the output to a file. No explicit message "No rules found" appears, but no rules are shown. Sorry for the misunderstanding.
This is the content of the RapidMiner output:
This is the content of the RapidMiner output:
When I run Weka GUI, this is the output:
Apriori
=======
Minimum support: 0.1 (220 instances)
Minimum metric <confidence>: 0.8
Number of cycles performed: 18
Generated sets of large itemsets:
Size of set of large itemsets L(1): 6
Size of set of large itemsets L(2): 1
Best rules found:
Apriori
=======
Minimum support: 0.35 (770 instances)
Minimum metric <confidence>: 0.9
Number of cycles performed: 13
Generated sets of large itemsets:
Size of set of large itemsets L(1): 4
Size of set of large itemsets L(2): 5
Size of set of large itemsets L(3): 2
Best rules found:
1. Clase=0 885 ==> Edad=1 885 conf:(1)
2. Clase=0 Sexo=1 862 ==> Edad=1 862 conf:(1)
3. Sexo=1 Sobrevivio?=0 1364 ==> Edad=1 1329 conf:(0.97)
4. Clase=0 885 ==> Sexo=1 862 conf:(0.97)
5. Clase=0 Edad=1 885 ==> Sexo=1 862 conf:(0.97)
6. Clase=0 885 ==> Edad=1 Sexo=1 862 conf:(0.97)
7. Sobrevivio?=0 1490 ==> Edad=1 1438 conf:(0.97)
8. Sexo=1 1731 ==> Edad=1 1667 conf:(0.96)
9. Edad=1 Sobrevivio?=0 1438 ==> Sexo=1 1329 conf:(0.92)
10. Sobrevivio?=0 1490 ==> Sexo=1 1364 conf:(0.92)
New Altair Community Member
Hm, I would not exclude that there actually may be something wrong, but the first thing I see is that you are using different settings in RapidMiner and in Weka, namely the parameters for min confidence and min support differ. Did you change them on purpose? Do the other parameters actually match?
Best regards,
Marius
Best regards,
Marius
New Altair Community Member
OPI left all parameters with default values in both Weka and RapidMiner. They have exactly the same values. You can see here:
I think what you refer is part of the algorithm output.
Nevertheless, if I set the output min confidence (0.35) and support (0.9), I get the following message in RapidMiner:
I think what you refer is part of the algorithm output.
Nevertheless, if I set the output min confidence (0.35) and support (0.9), I get the following message in RapidMiner:
No large itemsets and rules found!
Best regards,
Marius