Triple Gamma Shrinkage for Gaussian Process Regression

Statistical models lie at the heart of many decision-making processes in the modern world. They are used to predict the weather, to model the spread of diseases, and to forecast economic developments, to name just a few examples. An important trade-off in the development of such models is the balance between flexibility and interpretability. Highly flexible models can capture complex relationships in the data, but their black-box nature often makes it difficult to understand how they arrive at their predictions. This drawback may become problematic in high-stakes applications, such as medical diagnoses or financial forecasting. On the flipside, highly interpretable models often make too many assumptions about the relationships between variables, limiting their ability to capture complex patterns in the data. Further complicating this trade-off is the increasing availability of high-dimensional data, where the number of explanatory variables often outstrips the number of observations. In such scenarios, being able to effectively determine the variables that drive the outcome becomes crucial for accurate predictions and interpretability. The proposed research aims to address these issues by fusing flexible Gaussian process regression (GPR) models with cutting edge variable selection tools borrowed from the field of Bayesian statistics. GPR models are a flexible tool for regression analysis, as they make few assumptions about the functional form of the relationship between variables, but still allow for interpretability. However, as the number of variables grows, the flexibility of GPR models can lead to large predictive uncertainty, which in turn decreases interpretability. The proposed research will investigate the use of so called "shrinkage priors", tools developed for Bayesian variable selection, to automatically determine the variables that drive the outcome in GPR models. The main idea behind shrinkage priors is to strongly pull small parameter values towards zero, while still allowing larger parameter values to filter through. This allows for the automatic selection of variables that are relevant for the outcome (which is synonymous with being far from zero), while still retaining the flexibility of the GPR model. Such models would have applications in a diverse array of fields, such as economics, biology, and engineering. The proposed research will not only contribute to the scientific literature on GPR models and Bayesian variable selection, but will also provide a valuable tool for researchers in a wide range of fields. The algorithms developed will be made available to the wider scientific community in the form of an R package, allowing for easy application to other data sets and further development of the algorithms.