102
|
Janes H, Brown MD, Pepe MS. Designing a study to evaluate the benefit of a biomarker for selecting patient treatment. Stat Med 2015; 34:3503-15. [PMID: 26112650 PMCID: PMC4626364 DOI: 10.1002/sim.6564] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2014] [Revised: 03/20/2015] [Accepted: 05/26/2015] [Indexed: 12/31/2022]
Abstract
Biomarkers that predict the efficacy of treatment can potentially improve clinical outcomes and decrease medical costs by allowing treatment to be provided only to those most likely to benefit. We consider the design of a randomized clinical trial in which one objective is to evaluate a treatment selection marker. The marker may be measured prospectively or retrospectively using samples collected at baseline. We describe and contrast criteria around which the trial can be designed. An existing approach focuses on determining if there is a statistical interaction between the marker and treatment. We propose three alternative approaches based on estimating clinically relevant measures of improvement in outcomes with use of the marker. Importantly, our approaches accommodate the common scenario in which the marker-based rule for recommending treatment is developed with data from the trial. Sample sizes are calculated for powering a trial to assess these criteria in the context of adjuvant chemotherapy for the treatment of estrogen-receptor-positive, node-positive breast cancer. In this example, we find that larger sample sizes are generally required for assessing clinical impact than for simply evaluating if there is a statistical interaction between marker and treatment. We also find that retrospectively selecting a case-control subset of subjects for marker evaluation can lead to large efficiency gains, especially if cases and controls are matched on treatment assignment.
Collapse
Affiliation(s)
- Holly Janes
- Fred Hutchinson Cancer Research Center, Seattle, Washington, USA
- University of Washington, Seattle, Washington, USA
| | - Marshall D Brown
- Fred Hutchinson Cancer Research Center, Seattle, Washington, USA
| | - Margaret S Pepe
- Fred Hutchinson Cancer Research Center, Seattle, Washington, USA
- University of Washington, Seattle, Washington, USA
| |
Collapse
|
103
|
McShane LM, Cavenagh MM, Lively TG, Eberhard DA, Bigbee WL, Williams PM, Mesirov JP, Polley MYC, Kim KY, Tricoli JV, Taylor JMG, Shuman DJ, Simon RM, Doroshow JH, Conley BA. Criteria for the use of omics-based predictors in clinical trials: explanation and elaboration. BMC Med 2013; 11:220. [PMID: 24228635 PMCID: PMC3852338 DOI: 10.1186/1741-7015-11-220] [Citation(s) in RCA: 91] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2013] [Accepted: 08/06/2013] [Indexed: 12/18/2022] Open
Abstract
High-throughput 'omics' technologies that generate molecular profiles for biospecimens have been extensively used in preclinical studies to reveal molecular subtypes and elucidate the biological mechanisms of disease, and in retrospective studies on clinical specimens to develop mathematical models to predict clinical endpoints. Nevertheless, the translation of these technologies into clinical tests that are useful for guiding management decisions for patients has been relatively slow. It can be difficult to determine when the body of evidence for an omics-based test is sufficiently comprehensive and reliable to support claims that it is ready for clinical use, or even that it is ready for definitive evaluation in a clinical trial in which it may be used to direct patient therapy. Reasons for this difficulty include the exploratory and retrospective nature of many of these studies, the complexity of these assays and their application to clinical specimens, and the many potential pitfalls inherent in the development of mathematical predictor models from the very high-dimensional data generated by these omics technologies. Here we present a checklist of criteria to consider when evaluating the body of evidence supporting the clinical use of a predictor to guide patient therapy. Included are issues pertaining to specimen and assay requirements, the soundness of the process for developing predictor models, expectations regarding clinical study design and conduct, and attention to regulatory, ethical, and legal issues. The proposed checklist should serve as a useful guide to investigators preparing proposals for studies involving the use of omics-based tests. The US National Cancer Institute plans to refer to these guidelines for review of proposals for studies involving omics tests, and it is hoped that other sponsors will adopt the checklist as well.
Collapse
Affiliation(s)
- Lisa M McShane
- Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 5W130, MSC 9735, 9609 Medical Center Drive, Bethesda, MD 20892-9735, USA
| | - Margaret M Cavenagh
- Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 4W432, MSC 9730, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - Tracy G Lively
- Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 4W420, MSC 9730, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - David A Eberhard
- Department of Pathology and Lineberger Comprehensive Cancer Center, Brinkhous-Bullitt Bldg., Campus Box 7525, University of North Carolina, Chapel Hill, NC 27599, USA
| | - William L Bigbee
- Department of Pathology and University of Pittsburgh Cancer Institute, Hillman Cancer Center, UPCI Research Pavilion, Suite 2.32b, 5117 Centre Avenue, Pittsburgh, PA 15213, USA
| | - P Mickey Williams
- Frederick National Laboratory for Cancer Research, National Cancer Institute, National Institutes of Health, Bldg. 320, Room 2, 1050 Boyles Street, Frederick, MD 21702, USA
| | - Jill P Mesirov
- Computational Biology and Bioinformatics, Broad Institute of Massachusetts Institute of Technology and Harvard University, 7 Cambridge Center, Cambridge, MA 02142, USA
| | - Mei-Yin C Polley
- Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 5W638, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - Kelly Y Kim
- Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 4W430, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - James V Tricoli
- Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 3W526, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - Jeremy MG Taylor
- Department of Biostatistics, University of Michigan, 1415 Washington Heights, Ann Arbor, MI 48109, USA
| | - Deborah J Shuman
- Office of the Director, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 3A44, 31 Center Drive, Bethesda, MD 20892, USA
| | - Richard M Simon
- Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 5W110, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| | - James H Doroshow
- Office of the Director, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 3A44, 31 Center Drive, Bethesda, MD 20892, USA
| | - Barbara A Conley
- Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Room 4W426, 9609 Medical Center Drive, Bethesda, MD 20892, USA
| |
Collapse
|
104
|
McShane LM, Polley MYC. Development of omics-based clinical tests for prognosis and therapy selection: the challenge of achieving statistical robustness and clinical utility. Clin Trials 2013; 10:653-65. [PMID: 24000377 DOI: 10.1177/1740774513499458] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
BACKGROUND Many articles have been published in biomedical journals reporting on the development of prognostic and therapy-guiding biomarkers or predictors developed from high-dimensional data generated by omics technologies. Few of these tests have advanced to routine clinical use. PURPOSE We discuss statistical issues in the development and evaluation of prognostic and therapy-guiding biomarkers and omics-based tests. METHODS Concepts relevant to the development and evaluation of prognostic and therapy-guiding clinical tests are illustrated through discussion and examples. Some differences between statistical approaches for test evaluation and therapy evaluation are explained. The additional complexities introduced in the evaluation of omics-based tests are highlighted. RESULTS Distinctions are made between clinical validity of a test and clinical utility. To establish clinical utility for prognostic tests, it is explained why absolute risk should be evaluated in addition to relative risk measures. The critical role of an appropriate control group is emphasized for evaluation of therapy-guiding tests. Common pitfalls in the development and evaluation of tests generated from high-dimensional omics data such as model overfitting and inappropriate methods for test performance evaluation are explained, and proper approaches are suggested. LIMITATIONS The cited references do not comprise an exhaustive list of useful references on this topic, and a systematic review of the literature was not performed. Instead, a few key points were highlighted and illustrated with examples drawn from the oncology literature. CONCLUSIONS Approaches for the development and statistical evaluation of clinical tests useful for predicting prognosis and selecting therapy differ from standard approaches for therapy evaluation. Proper evaluation requires an understanding of the clinical setting and what information is likely to influence clinical decisions. Specialized expertise relevant to building mathematical predictor models from high-dimensional data is helpful to avoid common pitfalls in the development and evaluation of omics-based tests.
Collapse
Affiliation(s)
- Lisa M McShane
- Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute, Bethesda, MD, USA
| | | |
Collapse
|
105
|
Kovalchik SA, Varadhan R, Weiss CO. Assessing heterogeneity of treatment effect in a clinical trial with the proportional interactions model. Stat Med 2013; 32:4906-23. [PMID: 23788362 DOI: 10.1002/sim.5881] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2012] [Accepted: 05/24/2013] [Indexed: 01/10/2023]
Abstract
Understanding how individuals vary in their response to treatment is an important task of clinical research. For standard regression models, a proportional interactions model first described by Follmann and Proschan (1999) offers a powerful approach for identifying effect modification in a randomized clinical trial when multiple variables influence treatment response. In this paper, we present a framework for using the proportional interactions model in the context of a parallel-arm clinical trial with multiple prespecified candidate effect modifiers. To protect against model misspecification, we propose a selection strategy that considers all possible proportional interactions models. We develop a modified Bonferroni correction for multiple testing that accounts for the positive correlation among candidate models. We describe methods for constructing a confidence interval for the proportionality parameter. In simulation studies, we show that our modified Bonferroni adjustment controls familywise error and has greater power to detect proportional interactions compared with multiplcity-corrected subgroup analyses. We demonstrate our methodology by using the Studies of Left Ventricular Dysfunction Treatment trial, a placebo-controlled randomized clinical trial of the efficacy of enalapril to reduce the risk of death or hospitalization in chronic heart failure patients. An R package called anoint is available for implementing the proportional interactions methodology.
Collapse
Affiliation(s)
- Stephanie A Kovalchik
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, Rockville, MD, U.S.A
| | | | | |
Collapse
|
106
|
McShane LM. Statistical challenges in the development and evaluation of marker-based clinical tests. BMC Med 2012; 10:52. [PMID: 22642713 PMCID: PMC3379945 DOI: 10.1186/1741-7015-10-52] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/08/2012] [Accepted: 05/29/2012] [Indexed: 11/10/2022] Open
Abstract
Exciting new technologies for assessing markers in human specimens are now available to evaluate unprecedented types and numbers of variations in DNA, RNA, proteins, or biological structures such as chromosomes. These markers, whether viewed individually, or collectively as a 'signature', have the potential to be useful for disease risk assessment, screening, early detection, prognosis, therapy selection, and monitoring for therapy effectiveness or disease recurrence. Successful translation from basic research findings to clinically useful test requires basic, translational, and regulatory sciences and a collaborative effort among individuals with varied types of expertise including laboratory scientists, technology developers, clinicians, statisticians, and bioinformaticians. The focus of this commentary is the many statistical challenges in translational marker research, specifically in the development and validation of marker-based tests that have clinical utility for therapeutic decision-making.
Collapse
Affiliation(s)
- Lisa M McShane
- Biometric Research Branch and Cancer Diagnosis Program, Division of Cancer Treatment and Diagnosis, National Cancer Institute, 6130 Executive Boulevard, EPN 8126, Bethesda, MD 20892-7434, USA.
| |
Collapse
|