Shrinking and Regularizing Finite Mixture Models

The presence of groups of observations with different characteristics is often suspected in data. However, the group memberships are either not available or not observable. Such a situation requires the application of a statistical method in the data analysis which allows to explicitly account for the presence of these latent groups and which aims at determining the group sizes as well as the group characteristics. The standard model-based tool in statistical analysis for this problem is the finite mixture model. Finite mixture models have been used for more than 100 years and represent a flexible and generally applicable statistical tool with many extensions and variations already proposed. However, some problems remain still unresolved such as the correct selection of variables to include in the analysis which drive the group structure and the choice of a suitable model which avoids overfitting the heterogeneity in order to ensure easy interpretability and precise estimation of parameters. In this research project we will aim at improving the application of finite mixture models by providing tools based on shrinkage and regularization which allow selecting a suitable model where relevant variables and irrelevant variables are automatically distinguished and the parameters are chosen in a way to avoid overfitting heterogeneity. Theoretical results will be complemented by applications and software implementations as add-on package for the open-source software R, an environment for statistical computing and graphics (http://www.R-project.org). The availability of improved statistical methods in combination with software implementations allows for a better analysis and increased understanding of data in empirical quantitative research. Due to the wide applicability of finite mixture models, for example in astronomy, biology, economic, marketing, medicine and psychology, results of this research project are assumed to have an impact also on other areas of research, by allowing for improved insights into latent group structures which are present in the data.

Cluster analysis is a statistical method which allows to identify structure in data by grouping observations together. Cluster analysis is applied in many different areas where data are analyzed. The model-based approach for cluster analysis embeds the clustering problem within a statistical inference framework and uses mixture models as the underlying data generating process. The model-based approach is appealing because statistical inference methods can be used to resolve crucial questions appearing in cluster analysis applications. In addition, extensions are readily available by considering different statistical models for the components of the mixture model. Pursuing a Bayesian approach for inference facilitates the inclusion of prior information on the cluster shapes or their number. This reduces the ambiguity in the clustering problem and helps to find sensible clustering solutions. In this project we advanced Bayesian mixture modeling focusing on the model-based clustering context. We pursued a holistic approach covering aspects of both model and prior specification as well as model estimation and general considerations for applications. We aimed at bridging finite and infinite mixture models and were able to highlight important aspects which allow to identify similarities but also crucial differences between the two model classes. We investigated general aspects such as the inclusion of a prior on the number of components to account for model uncertainty and obtain a fully Bayesian model specification which resulted in the generalized mixture of finite mixtures (MFM) model. We shed light on how explicit prior specifications impact on implicitly induced priors which are often of more interest in clustering applications. We advanced estimation of the Bayesian mixture model by proposing the telescoping sampler for the generalized MFM model and an efficient Markov chain Monte Carlo sampling schemes for the case of a mixture-of-experts model. Overall, the project results help to successfully apply Bayesian mixture modeling techniques and enhance their use for model-based clustering and thus broaden and improve the statistical toolbox available for data analysis. Aiming at a suitable dissemination of the project results, we did not only publish several research articles in peer-reviewed journals targeted at experts in the field, but also ensured to provide more accessible contributions, in particular by co-editing and contributing chapters to the ``CRC Handbook of Mixture Analysis'' and contributing an entry to ``Wiley StatsRef: Statistics Reference Online''. R packages implementing some of the computational methods developed during this project are available open-source from the Comprehensive R Archive Network.