Evaluation of Adaptive Design Procedures

Interim analyses in medical studies, e.g., in clinical trials, are performed for ethical and economical reasons. On the one hand studies should not be extended (and decisions postponed), if a clear tendency favouring a particular treatment evolves so that all patients can benefit from medical progress as soon as possible. On the other hand no additional patients should be treated with a new therapeutic concept (for which in such a situation only limited knowledge about the risks is available) if the ongoing trial gives no indication for a potential patient benefit. Moreover, sequential trials with early decisions may reduce costs for the development of improved therapies. Group sequential tests with a low number of planned interim analyses are most widely used in medical studies. For these designs strictly the number of interim analyses, the group sizes and the decision boundaries have to be laid down a priori in the planning phase. "Optimal" trials in general depend on other facts, e.g., on the value of the relevant treatment effect, the variability, the statistical model to be applied, the doses, and many others. If all this would be sufficiently known a priori to run such a trial presumably would be unethical, as there is nothing left to learn. Adaptive sequential designs can use all the information collected up to the interim analyses from inside or outside the trial to perform mid-trial design modifications which appear to be reasonable or necessary to account for deficiencies in the planning phase. To control the overall type I error probability these adaptive designs essentially adhere to a common invariance principle: Separate test statistics are calculated from the samples at the different stages and aggregated in a predefined way for test decisions. Hence any design modification which under the null hypothesis preserves the distributional properties of the separate stage-wise test statistics do not inflate the level alpha [e.g., Bauer, Brannath & Posch, 2001]. Due to the wide flexibility of these methods (which extends to the number of interim analyses), it is difficult to assess their merits for applied problems. The goal of this research project is to invest-igate adaptive multi-stage designs with regard to power, average sample size and estimation issues. This work requires exact calculations under simple distributional assumptions (normal) and simulations (e.g. for survival studies) to compare the methods with classical non-adaptive approaches (e.g. single stage and group sequential designs). Special emphasis is put on how to overcome misspecification in the planning phase by suitable mid-trial design modification.

Adaptive study designs allow for performing mid-trial design modifications based on data from inside or outside an ongoing trial without inflating the probability of producing false positive test decisions. This is a change of the paradigm, since in conventional experiments the design specifications have to be strictly fixed a priori. In the beginning sample size modification has attracted most interest. We investigated advantages and disadvantages of sample size reassessment in an interim analysis. It has been shown that re-evaluation of the chances to reach the goal in the final analysis produces poor estimates of the true chances in particular if the interim analysis is performed early. Moreover, using the effect size observed up to the interim analysis as an estimate of the true unknown effect size produces a high variability in the reassessed sample size. If the observed effect is higher than the true one the re-estimated sample size needed to get a significant result at the end of the trial may be considerably smaller than for the conventional test: The calculation in the interim analysis starts from the optimistic trend observed, and, at the same time, it uses the optimistic trend in place of the true effect to quantify the probability of a success. It was also possible to derive results about the form of a sample size reassessment rule which produces a minimum average sample size. A major achievement was also to show how the expected costs of a trial can be reduced if later planned interim analyses can be skipped in case of poor chances to achieve an early positive decision, or un- planned interim analyses can be inserted in case of good chances to get early positive decisions. A burning issue in clinical trials is changing goals. E.g., a trial starts with the intention to show non-inferiority of a new treatment as compared to an active control treatment. However, in the interim analysis there may be evidence that the new treatment is superior to the control. We demonstrated which and how different types of sequential decision boundaries can be used in a trial with simultaneous sequential tests for non-inferiority and superiority. During the project the issue of gene association studies has become a new challenge to medical research. We investigated the behaviour of two-stage designs with a very large number of hypotheses and rather small sample sizes per hypothesis. Assuming constrained costs we derived optimal two-stage designs which control the false discovery rate, i.e., the expected fraction of false positive among all markers identified as effective. It turns out that for correctly identifying a high average number of effective markers about three quarters of the sample should be used at the first stage to screen about 10% of the hypotheses to be investigated also at the second stage. Beyond further methodological research according to the intention of the project a focus was also put on producing innovative protocols in specific clinical research questions, covering issues of treatment selection, multiple testing and estimation.