Predicting Rare Events More Accurately (PREMA)

Logistic regression is one of the most commonly used statistical methods to estimate prognostic models that relate a binary outcome (with levels event and non-event) to a number of binary, categorical or continuous explanatory variables. A low prevalence of events, encountered frequently in clinical or epidemiological studies, but also in other fields of empirical research, causes underestimation and instability of estimates of the event probability in subjects who are likely to experience the rare event. This happens because the analysis is disproportionally influenced by the subjects without events. This effect is even more pronounced when the number of explanatory variables approaches or exceeds the number of outcome events. Recently, penalized likelihood regression (PLR) methods have become popular for analyses with high- dimensional explanatory variable spaces. PLR methods shrink the estimates of regression coefficients towards zero in order to decrease their mean squared error. While this also decreases the overall mean squared error of predicted event probabilities, in the rare events situation (RES) poor predictions for the subjects which are at high risk for an event are still encountered. The main objective of this project is to further develop PLR with regard to the high-dimensional RES by elaborating and evaluating novel approaches to estimation, tuning and validation. At the estimation stage, there are several possibilities for an enhancement of PLR. Besides modifications of weighting methods, various types of penalties, e.g., the combination of different classical penalties (Firth and LASSO or Firth and ridge) and generalizations of the Firth-type penalty, will be taken into account. Moreover, the tuning criteria commonly used in PLR to control the amount of penalization of the model, such as optimization of cross-validated deviance or overall misclassification error, are not expected to behave well in the RES. Here, we will deduce tuning criteria that put more weight to the observations with events in order to obtain more accurate event predictions for subjects with higher underlying event probabilities, at the cost of less accurate estimation for subjects not susceptible to events. In a similar way, measures of predictive accuracy used for model validation will be adapted to the RES. The performance of the proposed methods will be evaluated on real-life data sets and in comprehensive simulation studies. Implemented in statistical software packages, the results of this project will be of practical value, whenever predictions for strongly imbalanced binary outcomes have to be derived from high-dimensional data.

In this project, novel statistical methods for the prognosis of rare events based on characteristics available at the time of prognosis were developed. These new methods were investigated with regard to their statistical properties and compared with existing methods to gain new insight in what could be the most efficient statistical analysis to obtain precise predictions. The main contribution of this project lies in the development of modifications of the logistic regression analysis method. Two new extensions were developed, abbreviated by FLIC and FLAC, which open new routes to precise prediction of the occurrence of rare events such as side effects of drugs. In order to make these new methodology accessible to the international research community, computer programs were developed which are freely available.