Shrinkage estimators for prediction-out-of-sample

Modern statistical theory features powerful and highly efficient shrinkage estimators. In regression, performance analyses of such estimators are mainly focused on parameter estimation and on in-sample prediction, where the goal is estimation of the regression function at those points that were observed in the training sample. Comparatively little is known about the performance of shrinkage estimators for out-of-sample prediction, where the goal consists of estimating the regression function at new and hitherto un-observed points. Recently, Huber and Leeb (2013) showed that the James-Stein estimator can fail to dominate the maximum-likelihood estimator for out- of-sample prediction. The goal of the proposed research project is to analyze this and related phenomena, to design new shrinkage estimators with good predictive performance out-of-sample, and to develop inference methods like prediction intervals based on these new estimators.

The project has broken new ground in the area of predictive inference with shrinkage estimators. In particular, it was found that these methods can perform particularly well in situations where the system of interest is very complex and where, at the same time, the available training dataset is comparatively small. Such situations are very common in certain Big Data applications. For such situations, new prediction methods and new methods for predictive inference were developed.