Connecting Process Documents WordList with FP-Growth FrequentItemSets

Hello,

I was doing text mining and have a result from Process Documents with the term frequencys and a result from FP-Growth with the itemset frequencys. So I have a word, let's say cookie, which occures in my document 150 times. I get this Information from the WordList (Process Documents). For this example I have this frequently-occurring itemsets:

Size Support Item 1 Item 2

1 0.150 cookie

2 0.051 cookie milk

2 0.017 cookie marshmallow

2 0.023 cookie chocolate

2 0.012 cookie crackers

2 0.014 cookie strawberries

2 0.011 cookie raspberries

And now I want to combine the two tables to know the percentage of how often cookies and milk occure together, when cookie is written in the text. And which is the absolute frequency from the occurence of cookie and milk. At the moment the Support only shows it in relation to the whole dataset...

Thanks for helping

Find more posts tagged with

AI Studio

FP-Growth

Text Mining + NLP

Accepted answers

MartinLiebig

Hi @imke,

isnt this how confidence is defined in association rules? If yes, just use Create Association Rules on the FP-Growth result.

For details on the AR-Metrics see: https://community.rapidminer.com/t5/RapidMiner-Studio-Knowledge-Base/How-To-Interpret-the-Results-of-Create-Association-Rules/ta-p/32107

BR,

Martin

MartinLiebig

Hi,

it should be: Number of Documents which contains token A / All Documents in the corpus.

Best,

Martin

All comments

MartinLiebig

Hi @imke,

isnt this how confidence is defined in association rules? If yes, just use Create Association Rules on the FP-Growth result.

For details on the AR-Metrics see: https://community.rapidminer.com/t5/RapidMiner-Studio-Knowledge-Base/How-To-Interpret-the-Results-of-Create-Association-Rules/ta-p/32107

BR,

Martin

imke

Hi Martin,

actually I have written that explination a few times but I missunderstood that. Thank's for your quick answer now it is clear. But I have an other question now:

The Help Explination says:

"The support supp(X) of an itemset X is defined as the proportion of transactions in the data set which contain the itemset."

Does that mean: 'Document Occurences (WordList)' devided by the 'number of textes' ?

And which is the difference between "Total Occurences" and "Document Occurences" in the WordList from Process Documents?

Thank you again!

MartinLiebig

Hi,

it should be #documentsHavingTokenA/AllDocuments

Best,

Martin

imke

Hi @mschmitz

what do you ean with: #documentsHavingTokenA/AllDocuments?

Is there a hyperlink missing?

Thank's

Imke

MartinLiebig

Hi,

it should be: Number of Documents which contains token A / All Documents in the corpus.

Best,

Martin

imke

Hi @mschmitz,

thank you!

I have some more questions to the FP-Growth and the Confidence:

When I look at my results from FP-Growth I have more itemsets then with the Create Association Rules operator, although my min Confidence is at 0.001. Why is it like that?

And why does the Create Association Rules changes the order of the itemsets?

I had milk -> cookie with FP-Growth and now I have cookie -> milk. Does the Create Association Rules operator choose the order with the higher confidence?

Thank’s a lot!

Imke