Optimal test procedures for multiple hypotheses controlling the familywise expected loss

Optimal weighted Bonferroni tests and their graphical extensions

Regulatory guidelines mandate the strong control of the familywise error rate in confirmatory clinical trials with primary and secondary objectives. Bonferroni tests are one of the popular choices for multiple comparison procedures and are building blocks of more advanced procedures. It is usually of interest to find the optimal weighted Bonferroni split for multiple hypotheses. We consider two popular quantities as the optimization objectives, which are the disjunctive power and the conjunctive power. The former is the probability to reject at least one false hypothesis and the latter is the probability to reject all false hypotheses. We investigate the behavior of each of them as a function of different Bonferroni splits, given assumptions about the alternative hypotheses and correlations between test statistics. Under independent tests, unique optimal Bonferroni weights exist; under dependence, optimal Bonferroni weights may not be unique based on a fine grid search. In general, we propose an optimization algorithm based on constrained nonlinear optimization and multiple starting points. The proposed algorithm efficiently identifies optimal Bonferroni weights to maximize the disjunctive or conjunctive power. In addition, we apply the proposed algorithm to graphical approaches, which include many Bonferroni-based multiple comparison procedures. Utilizing the closed testing principle, we adopt a two-step approach to find optimal graphs using the disjunctive power. We also identify a class of closed test procedures that optimize the conjunctive power. We apply the proposed algorithm to a case study to illustrate the utility of optimal graphical approaches that reflect study objectives.

Discussion on "Optimal test procedures for multiple hypotheses controlling the familywise expected loss" by Willi Maurer, Frank Bretz, and Xiaolei Xun

This comment builds on the familywise expected loss (FWEL) framework suggested by Maurer, Bretz, and Xun in 2022. By representing the populationwise error rate (PWER) as FWEL, it is illustrated how the FWEL framework can be extended to clinical trials with multiple and overlapping populations and the PWER can be generalized to more general losses. The comment also addresses the question of how to deal with midtrial changes in the posttrial risks and related losses that are caused by data-driven decisions. Focusing on multiarm trials with the possibility of dropping treatments midtrial, we suggest to switch from control of the unconditional expected loss to control of the conditional expected loss that is related to the actual risks and is conditional on the sample event that causes the change in the risks. The problem and here suggested solution is also motivated with a sequence of independent trials for a hitherto incurable disease which ends when an efficient treatment is found. No multiplicity adjustment is applied in this case and we show how this can be justified by the consideration of the changing out-trial risks and with control of conditional type I error rates and losses.