I'm trying to learn market basket analysis

tonyboy9

I read documentation for the Create Association Rules operator. The tutorial process uses the Iris sample set. 



This is the tutorial process for Create Association Rules.



I attempted to duplicate this, pre-processing under Iris. To my amazement it ran in Studio.



This is the Association Rules Results for Iris:



This is my MarketBasketAnalysis data set:



I ran another process substituting in place my MarketBasketAnalysis data set. Again with a minor correction, to my amazement, it ran.



These are the Association Rules Results for my MarketBasketAnalysis



Finally, questions on the Results for Iris and MarketBasketAnalysis. How am I to interpret any of this? I like the confidence numbers .929 and .949. I re-examined parameters for FP-Growth and Create Association Rules. In the Results for MarketBasketAnalysis, what does this say about customer behavior when it comes to shopping habits? Once more, thank you for your time.
 

jacobcybulski

The most important aspect of association analysis is to set the requirements as to what it is exactly you are trying to achieve. Association analysis, of which Market Basket Analysis is one type, is trying to determine what kind of items go with what kind of items together. This could be products you buy together in one transaction (shopping basket), or product types being ordered by one customer over time (with or without consideration of order), or what web pages are being clicked on by customers of a specific profile, or what movies / actors / directors are being watched by Netflix subscribers, or songs / albums/ artists / genres listened to by Spotify subscribers. In all these cases, there are three questions: (1) what is a "basket", (2) what are the "products you put into the basket", and (3) what exactly you want to get out of it. Having answered the first two questions you can then proceed to your "basket" representation and association analysis (as per the tutorial). The answer to the last question will help you figure out if you are creating a classification, clustering or some predictive model (e.g. if you watched the following movies we can suggest the movie you have not watched as yet). So in your case, I do not think you have formulated these three questions as yet, so the results do not make much sense (your premises seem a bit ad hoc).

I'll give you an extreme example of this. I was interested in the analysis of Wikileaks Afgan Files and tried to figure out what are the types of things of interest they may be talking about (in a sense I was after term clustering). So my basket = Wikileaks report, my product = word in the report, my objective = (association) clustering. So I created a process, which parsed all Wikileaks files, represented them as vectors of Term Occurrences (acceptable for FP Growth), applied FP Growth and Created Association Rules, which permitted me to visualise my associations in a graph (kind of clusters), which could then be explored by tuning the rules support and confidence, to reveal more or less details.

I hope it helps -- Jacob.

jacobcybulski

The most important aspect of association analysis is to set the requirements as to what it is exactly you are trying to achieve. Association analysis, of which Market Basket Analysis is one type, is trying to determine what kind of items go with what kind of items together. This could be products you buy together in one transaction (shopping basket), or product types being ordered by one customer over time (with or without consideration of order), or what web pages are being clicked on by customers of a specific profile, or what movies / actors / directors are being watched by Netflix subscribers, or songs / albums/ artists / genres listened to by Spotify subscribers. In all these cases, there are three questions: (1) what is a "basket", (2) what are the "products you put into the basket", and (3) what exactly you want to get out of it. Having answered the first two questions you can then proceed to your "basket" representation and association analysis (as per the tutorial). The answer to the last question will help you figure out if you are creating a classification, clustering or some predictive model (e.g. if you watched the following movies we can suggest the movie you have not watched as yet). So in your case, I do not think you have formulated these three questions as yet, so the results do not make much sense (your premises seem a bit ad hoc).

I'll give you an extreme example of this. I was interested in the analysis of Wikileaks Afgan Files and tried to figure out what are the types of things of interest they may be talking about (in a sense I was after term clustering). So my basket = Wikileaks report, my product = word in the report, my objective = (association) clustering. So I created a process, which parsed all Wikileaks files, represented them as vectors of Term Occurrences (acceptable for FP Growth), applied FP Growth and Created Association Rules, which permitted me to visualise my associations in a graph (kind of clusters), which could then be explored by tuning the rules support and confidence, to reveal more or less details.

I hope it helps -- Jacob.

tonyboy9

Alright, Jacob, the three questions, I get it. Thanks for putting the time in.