evaluating clustering algorithms?
We are working on text clustering for the data science project we find a few algorithms that can work with text like
-K-means
-K-medoids
These two are centroid clustering and we use Davies Bouldin evaluation metrics to evaluate them
-Agglomerative clustering
-Top-down clustering
These two are hierarchical clustering but we don't know how to evaluate them we need to compare between these four, so we need to find unified evaluation method
Find more posts tagged with
Sort by:
1 - 1 of
11
Since clustering is typically an unsupervised ML problem, it is complicated to find metrics that can be used to compare across algorithms that do very different things (such as the ones you have listed). What exactly are you trying to compare between the results? You can use trivial metrics like "number of clusters" or "number of records per cluster" but that doesn't necessarily tell you very much. If your dataset has a performance variable (a "label" in RapidMiner terms) then you can try to use the helpful "Map Clusters on Labels" operator and then look at other metrics that are more usual for supervised ML problems, but you need to make sure you have the right number of clusters compared to labels for this to work.