Method-independent indices for cluster validation and estimating the number of clusters

Given a data set and a clustering algorithm running on it with different input parameter values, we obtain different partitionings of the data set into not necessarily meaningful clusters. As a consequence, in most applications the resulting clustering scheme requires some sort of evaluation of its validity. Evaluating the results of a clustering algorithm is known as “cluster validation.” The present chapter focuses on relative cluster validity criteria that are used to compare different clusterings and find the one that “best” fits the considered data. These criteria are implemented by validity indices that can be evaluated from the data set and the given clustering alone without having access to a “true” clustering. This chapter aims at presenting an overview of the available relative cluster validity indices and at highlighting the differences between them and their implicit assumptions. Furthermore, we mention some software packages, we stress the requirements that are under-addressed by the recent approaches, and address new research directions.