Is the classical Wald test always suitable under response-adaptive randomization?

Inter-reader agreement of breast magnetic resonance imaging and contrast-enhanced mammography in breast cancer diagnosis: a multi-reader retrospective study

OBJECTIVE

Breast magnetic resonance imaging (MRI) and contrast-enhanced mammography (CEM) are nowadays used in breast imaging but studies about their inter-reader agreement are lacking. Therefore, we compared the inter-reader agreement of CEM and MRI in breast cancer diagnosis in the same patients.

METHODS

Breast MRI and CEM exams performed in a single center (09/2020-09/2021) for an IRB-approved study were retrospectively and independently evaluated by four radiologists of two different centers with different levels of experience who were blinded to the clinical and other imaging data. The reference standard was the histological diagnosis or at least 1-year negative imaging follow-up. Inter-reader agreement was examined using Cohen's and Fleiss' kappa (κ) statistics and compared with the Wald test.

RESULTS

Of the 750 patients, 395 met inclusion criteria (44.5 ± 14 years old), with 752 breasts available for CEM and MRI. Overall agreement was moderate (κ = 0.60) for MRI and substantial (κ = 0.74) for CEM. For expert readers, the agreement was substantial (κ = 0.77) for MRI and almost perfect (κ = 0.82) for CEM; for non-expert readers was fair (κ = 0.39); and for MRI and moderate (κ = 0.57) for CEM. Pairwise agreement between expert readers and non-expert readers was moderate (κ = 0.50) for breast MRI and substantial (κ = 0.74) for CEM and it showed a statistically superior agreement of the expert over the non-expert readers only for MRI (p = 0.011) and not for CEM (p = 0.062).

CONCLUSIONS

The agreement of CEM was superior to that of MRI (p = 0.012), including for both expert (p = 0.031) and non-expert readers (p = 0.005).

Response-adaptive randomization in clinical trials: from myths to practical considerations

Response-Adaptive Randomization (RAR) is part of a wider class of data-dependent sampling algorithms, for which clinical trials are typically used as a motivating application. In that context, patient allocation to treatments is determined by randomization probabilities that change based on the accrued response data in order to achieve experimental goals. RAR has received abundant theoretical attention from the biostatistical literature since the 1930's and has been the subject of numerous debates. In the last decade, it has received renewed consideration from the applied and methodological communities, driven by well-known practical examples and its widespread use in machine learning. Papers on the subject present different views on its usefulness, and these are not easy to reconcile. This work aims to address this gap by providing a unified, broad and fresh review of methodological and practical issues to consider when debating the use of RAR in clinical trials.

A new inferential approach for response-adaptive clinical trials: the variance-stabilized bootstrap

This paper discusses disadvantages and limitations of the available inferential approaches in sequential clinical trials for treatment comparisons managed via response-adaptive randomization. Then, we propose an inferential methodology for response-adaptive designs which, by exploiting a variance stabilizing transformation into a bootstrap framework, is able to overcome the above-mentioned drawbacks, regardless of the chosen allocation procedure as well as the desired target. We derive the theoretical properties of the suggested proposal, showing its superiority with respect to likelihood, randomization and design-based inferential approaches. Several illustrative examples and simulation studies are provided in order to confirm the relevance of our results.

Inter-Reader Agreement of Diffusion-Weighted Magnetic Resonance Imaging for Breast Cancer Detection: A Multi-Reader Retrospective Study

PURPOSE

In order to evaluate the use of un-enhanced magnetic resonance imaging (MRI) for detecting breast cancer, we evaluated the accuracy and the agreement of diffusion-weighted imaging (DWI) through the inter-reader reproducibility between expert and non-expert readers.

MATERIAL AND METHODS

Consecutive breast MRI performed in a single centre were retrospectively evaluated by four radiologists with different levels of experience. The per-breast standard of reference was the histological diagnosis from needle biopsy or surgical excision, or at least one-year negative follow-up on imaging. The agreement across readers (by inter-reader reproducibility) was examined for each breast examined using Cohen's and Fleiss' kappa (κ) statistics. The Wald test was used to test the difference in inter-reader agreement between expert and non-expert readers.

RESULTS

Of 1131 examinations, according to our inclusion and exclusion criteria, 382 women were included (49.5 ± 12 years old), 40 of them with unilateral mastectomy, totaling 724 breasts. Overall inter-reader reproducibility was substantial (κ = 0.74) for expert readers and poor (κ = 0.37) for non- expert readers. Pairwise agreement between expert readers and non-expert readers was moderate (κ = 0.60) and showed a statistically superior agreement of the expert readers over the non-expert readers (p = 0.003).

CONCLUSIONS

DWI showed substantial inter-reader reproducibility among expert-level readers. Pairwise comparison showed superior agreement of the expert readers over the non-expert readers, with the expert readers having higher inter-reader reproducibility than the non-expert readers. These findings open new perspectives for prospective studies investigating the actual role of DWI as a stand-alone method for un-enhanced breast MRI.

A simple solution to the inadequacy of asymptotic likelihood-based inference for response-adaptive clinical trials

The present paper discusses drawbacks and limitations of likelihood-based inference in sequential clinical trials for treatment comparisons managed via Response-Adaptive Randomization. Taking into account the most common statistical models for the primary outcome—namely binary, Poisson, exponential and normal data—we derive the conditions under which (i) the classical confidence intervals degenerate and (ii) the Wald test becomes inconsistent and strongly affected by the nuisance parameters, also displaying a non monotonic power. To overcome these drawbacks, we provide a very simple solution that could preserve the fundamental properties of likelihood-based inference. Several illustrative examples and simulation studies are presented in order to confirm the relevance of our results and provide some practical recommendations.

Compound optimal allocations for survival clinical trials

The aim of the present paper is to provide optimal allocations for comparative clinical trials with survival outcomes. The suggested targets are derived adopting a compound optimization strategy based on a subjective weighting of the relative importance of inferential demands and ethical concerns. The ensuing compound optimal targets are continuous functions of the treatment effects, so we provide the conditions under which they can be approached by standard response-adaptive randomization procedures, also guaranteeing the applicability of the classical asymptotic inference. The operating characteristics of the suggested methodology are verified both theoretically and by simulation, including the robustness to model misspecification. With respect to the other available proposals, our strategy always assigns more patients to the best treatment without compromising inference, taking into account estimation efficiency and power as well. We illustrate our procedure by redesigning two real oncological trials.

Optimal designs for testing hypothesis in multiarm clinical trials

The present paper deals with the problem of designing randomized multiarm clinical trials for treatment comparisons in order to achieve a suitable trade-off among inferential precision and ethical concerns. Although the large majority of the literature is focused on the estimation of the treatment effects, in particular for the case of two treatments with binary outcomes, the present paper takes into account the inferential goal of maximizing the power of statistical tests to detect correct conclusions about the treatment effects for normally response trials. After discussing the allocation optimizing the power of the classical multivariate test of homogeneity, we suggest a multipurpose design methodology, based on constrained optimization, which maximizes the power of the test under a suitable ethical constraint reflecting the effectiveness of the treatments. The ensuing optimal allocation depends in general on the unknown model parameters but, contrary to the unconstrained optimal solution or to some targets proposed in the literature, it is a non-degenerate continuous function of the treatment contrasts, and therefore it can be approached by standard response-adaptive randomization procedures. The properties of this constrained optimal allocation are described both theoretically and through suitable examples, showing good performances both in terms of ethical gain and statistical efficiency, taking into account estimation precision as well.