evaluating clustering algorithms?

Mm861552
Mm861552 New Altair Community Member
edited November 5 in Community Q&A
We are working on text clustering for the data science project we find a few algorithms that can work with text like
-K-means
-K-medoids 
These two are centroid clustering and we use Davies Bouldin evaluation metrics to evaluate them  
-Agglomerative clustering
-Top-down clustering
These two are hierarchical clustering but we don't know how to evaluate them we need to compare between these four, so we need to find unified evaluation method

Answers

  • Telcontar120
    Telcontar120 New Altair Community Member
    Since clustering is typically an unsupervised ML problem, it is complicated to find metrics that can be used to compare across algorithms that do very different things (such as the ones you have listed).  What exactly are you trying to compare between the results?  You can use trivial metrics like "number of clusters" or "number of records per cluster" but that doesn't necessarily tell you very much.  If your dataset has a performance variable (a "label" in RapidMiner terms) then you can try to use the helpful "Map Clusters on Labels" operator and then look at other metrics that are more usual for supervised ML problems, but you need to make sure you have the right number of clusters compared to labels for this to work.