Generating and evaluating evidence of the clinical utility of molecular diagnostic tests in oncology

Methods for determining clinical utility

Measuring the clinical utility of a diagnostic test involves evaluating its impact on patient outcomes, clinical decision-making, and healthcare resource utilization. Determining clinical utility requires accessing patient medical history and outcomes data. These studies involve enrolling patients undergoing diagnostic tests and tracking their clinical outcomes. Researchers can determine the test's clinical utility by comparing the outcomes of patients who receive the diagnostic test to those who do not. These outcomes include benefits and harm. The highest level of evidence to support clinical utility determinations may be obtained from clinical trials. However, clinical laboratories are often not involved in clinical trials, and laboratory specialists may not be experienced in conducting such trials. Many established laboratory tests have never had clinical utility determined. Prospective studies assessing a diagnostic test's impact on clinical outcomes may require long-term patient monitoring, which is problematic. This paper presents methods that may be used to assess clinical utility.

Assessing Bias and Limitations of Clinical Validation Studies of Molecular Diagnostic Tests for Indeterminate Thyroid Nodules: Systematic Review and Meta-Analysis

Background: Molecular tests for thyroid nodules with indeterminate fine needle aspiration results are increasingly used in clinical practice; however, true diagnostic summaries of these tests are unknown. A systematic review and meta-analysis were completed to (1) evaluate the accuracy of commercially available molecular tests for malignancy in indeterminate thyroid nodules and (2) quantify biases and limitations in studies that validate those tests. Summary: PubMed, EMBASE, and Web of Science were systematically searched through July 2021. English language articles that reported original clinical validation attempts of molecular tests for indeterminate thyroid nodules were included if they reported counts of true-negative, true-positive, false-negative, and false-positive results. We performed screening and full-text review, followed by assessment of eight common biases and limitations, extraction of diagnostic and histopathological information, and meta-analysis of clinical validity using a bivariate linear mixed-effects model. Forty-nine studies were included. Meta-analysis of Afirma Gene expression classifiers (GEC; n = 38 studies) revealed a sensitivity of 0.92 (confidence interval: 0.90-0.94), specificity of 0.26 (0.20-0.32), negative likelihood ratio (LR-) of 0.32 (0.23-0.44), positive LR+ of 1.24 (1.15-1.35), and area under the curve (AUC) of 0.83 (0.74-0.89). Afirma Genomic Sequencing Classifier (GSC; n = 10) had a sensitivity of 0.94 (0.89-0.96), specificity of 0.38 (0.27-0.50), LR- of 0.18 (0.10-0.30), LR+ of 1.52 (1.28-1.87), and AUC of 0.91 (0.62-0.92). ThyroSeq v1 and v2 (n = 10) had a sensitivity of 0.86 (0.82-0.90), specificity of 0.74 (0.59-0.85), LR- of 0.19 (0.13-0.26), LR+ of 3.52 (2.08-5.92), and AUC of 0.86 (0.81-0.90). ThyroSeq v3 (n = 6) had a sensitivity of 0.92 (0.86-0.95), specificity of 0.41 (0.18-0.69), LR- of 0.24 (0.09-0.62), LR+ of 1.67 (1.09-2.98), and AUC of 0.90 (0.63-0.92). Fourteen percent of studies conducted a blinded histopathologic review of excised thyroid nodules, and 8% made the decision to go to surgery blind to molecular test results. Conclusions: Meta-analyses reveal a high diagnostic accuracy of molecular tests for thyroid nodule assessment of malignancy risk; however, these studies are subject to several limitations. Limitations and their potential clinical impacts must be addressed and, when feasible, adjusted for using valid statistical methodologies.

Precision medicine in follicular lymphoma: Focus on predictive biomarkers

Current care for patients with follicular lymphoma (FL) offers most of them long-term survival. Improving it further will require careful patient selection. This review focuses on predictive biomarkers (ie, those whose outcome correlations depend on the treatment strategy) in FL, because awareness of what patient subsets benefit most or least from each therapy will help in this task. The first part of this review aims to summarize what biomarkers are predictive in FL, the magnitude of the effect and the quality of the evidence. We find predictive biomarkers in the setting of (a) indication of active treatment, (b) front-line induction (use of anthracyline-based regimens, CHOP vs bendamustine, addition of rituximab), (c) post-(front-line)induction (rituximab maintenance, radioimmunotherapy), and (d) relapse (hematopoietic stem cell transplant) and targeted agents. The second part of this review discusses the challenges of precision medicine in FL, including (a) cost, (b) clinical relevance considerations, and (c) difficulties over the broad implementation of biomarkers. We then provide our view on what biomarkers may become used in the next few years. We conclude by underscoring the importance of assessing the potential predictiveness of available biomarkers to improve patient care but also that there is a long road ahead before reaching their broad implementation due to remaining scientific, technological, and economic hurdles.

Use of Real-World Evidence in US Payer Coverage Decision-Making for Next-Generation Sequencing-Based Tests: Challenges, Opportunities, and Potential Solutions

OBJECTIVES

Given the potential of real-world evidence (RWE) to inform understanding of the risk-benefit profile of next-generation sequencing (NGS)-based testing, we undertook a study to describe the current landscape of whether and how payers use RWE as part of their coverage decision making and potential solutions for overcoming barriers.

METHODS

We performed a scoping literature review of existing RWE evidentiary frameworks for evaluating new technologies and identified barriers to clinical integration and evidence gaps for NGS. We synthesized findings as potential solutions for improving the relevance and utility of RWE for payer decision-making.

RESULTS

Payers require evidence of clinical utility to inform coverage decisions, yet we found a relatively small number of published RWE studies, and these are predominately focused on oncology, pharmacogenomics, and perinatal/pediatric testing. We identified 3 categories of innovation that may help address the current undersupply of RWE studies for NGS: (1) increasing use of RWE to inform outcomes-based contracting for new technologies, (2) precision medicine initiatives that integrate clinical and genomic data and enable data sharing, and (3) Food and Drug Administration reforms to encourage the use of RWE. Potential solutions include development of data and evidence review standards, payer engagement in RWE study design, use of incentives and partnerships to lower the barriers to RWE generation, education of payers and providers concerning the use of RWE and NGS, and frameworks for conducting outcomes-based contracting for NGS.

CONCLUSIONS

We provide numerous suggestions to overcome the data, methodologic, infrastructure, and policy challenges constraining greater integration of RWE in assessments of NGS.

Clinical Utility of the Nuclear-localized AR-V7 Biomarker in Circulating Tumor Cells in Improving Physician Treatment Choice in Castration-resistant Prostate Cancer

BACKGROUND

Proof of the clinical utility of a biomarker is when its use informs a management decision and improves patient outcomes relative to when it is not used.

OBJECTIVE

To model the clinical benefit of the nuclear-localized androgen receptor splice variant 7 (AR-V7) test for men with progressing metastatic castration-resistant prostate cancer (mCRPC) at the second line of therapy or greater to inform the choice of an androgen receptor signaling inhibitor (ARSI) or a taxane.

DESIGN, SETTING, AND PARTICIPANTS

The study population was a cross-sectional cohort of 193 unique patients with progressing mCRPC from whom 255 samples were drawn at the time of the second line or later treatment decision who then received an ARSI or taxane, with up to 3 yr of additional follow-up Circulating tumor cells (CTCs) were identified from blood samples and tested for AR-V7. Physicians were blinded to AR-V7 status and the testing laboratory was blinded to outcomes.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSES

We measured physician propensity for choosing an ARSI or taxane based on patient prognosis. We also measured overall survival (OS) adjusted for physician propensity by drug class; OS data were analyzed both without and with knowledge of nuclear-localized AR-V7 status.

RESULTS AND LIMITATIONS

Treating physicians had a propensity for choosing a taxane over an ARSI for patients with more advanced disease or who received an ARSI as the immediate prior therapy. After adjusting for physician propensity, discernible OS differences were not observed between taxane- and ARSI-treated patients (median 15.6 vs 14.4 mo; p =0.11). Patients with detectable nuclear-localized AR-V7 in CTCs had superior survival with taxanes over ARSIs (median 9.8 vs 5.7 mo; p = 0.041). AR-V7-negative patients had superior survival on ARSIs over taxanes (p = 0.033) but overlapping curves limit the interpretation. Mutivariable models showed a robust interaction between AR-V7 status and drug, and a lower risk of death on taxanes for AR-V7-positive men.

CONCLUSIONS

Use of the nuclear-localized AR-V7 CTC test to inform treatment choice can improve patient outcomes relative to decisions based solely on standard-of-care measures.

PATIENT SUMMARY

Men with metastatic prostate cancer who test positive for AR-V7 protein in circulating tumor cells are likely to live longer if taxane chemotherapy is used.

Establishing Clinical Utility for Diagnostic Tests Using a Randomized Controlled, Virtual Patient Trial Design

Demonstrating clinical utility for diagnostic tests and securing coverage and reimbursement requires high quality and, ideally, randomized controlled trial (RCT) data. Traditional RCTs are often too costly, slow, and cumbersome for diagnostic firms. Alternative data options are needed. We evaluated four RCTs using virtual patients to demonstrate clinical utility. Each study used a similar pre-post intervention, two round design to facilitate comparison. Representative samples of physicians were recruited and randomized into control and intervention arms. All physicians were asked to care for their virtual patients during two assessment rounds, separated by a multi-week time interval. Between rounds, intervention physicians reviewed educational materials on the diagnostic test. All physician responses were scored against evidence-based care criteria. RCTs using virtual patients can demonstrate clinical utility for a variety of diagnostic test types, including: (1) an advanced multi-biomarker blood test, (2) a chromosomal microarray, (3) a proteomic assay analysis, and (4) a multiplex immunofluorescence imaging platform. In two studies, utility was demonstrated for all targeted patient populations, while in the other two studies, utility was only demonstrated for a select sub-segment of the intended patient population. Of these four tests, two received positive coverage decisions from Palmetto, one utilized the study results to support commercial payer adjudications, and the fourth company went out of business. RCTs using virtual patients are a cost-effective approach to demonstrate the presence or absence of clinical utility.

Effect of Positron Emission Tomography Imaging in Women With Locally Advanced Cervical Cancer: A Randomized Clinical Trial

IMPORTANCE

In women with locally advanced cancer of the cervix (LACC), staging defines disease extent and guides therapy. Currently, undetected disease outside the radiation field can result in undertreatment or, if disease is disseminated, overtreatment.

OBJECTIVE

To determine whether adding fludeoxyglucose F 18 positron emission tomography-computed tomography (PET-CT) to conventional staging with CT of the abdomen and pelvis affects therapy received in women with LACC.

DESIGN, SETTING, AND PARTICIPANTS

A randomized clinical trial was conducted. Women with newly diagnosed histologically confirmed International Federation of Gynecology and Obstetrics stage IB to IVA carcinoma of the cervix who were candidates for chemotherapy and radiation therapy (CRT) were allocated 2:1 to PET-CT plus CT of the abdomen and pelvis or CT alone. Enrollment occurred between April 2010 and June 2014 at 6 regional cancer centers in Ontario, Canada. The PET-CT scanners were at 6 associated academic institutions. The median follow-up at the time of the analysis was 3 years. The analysis was conducted on March 30, 2017.

INTERVENTIONS

Patients received either PET-CT plus CT of the abdomen and pelvis or CT of the abdomen and pelvis.

MAIN OUTCOMES AND MEASURES

Treatment delivered, defined as standard pelvic CRT vs more extensive CRT, ie, extended field radiotherapy or therapy with palliative intent.

RESULTS

One hundred seventy-one patients were allocated to PET-CT (n = 113) or CT (n = 58). The trial stopped early before the planned target of 288 was reached because of low recruitment. Mean (SD) age was 48.1 (11.2) years in the PET-CT group vs 48.9 (12.7) years in the CT group. In the 112 patients who received PET-CT, 68 (60.7%) received standard pelvic CRT, 38 (33.9%) more extensive CRT, and 6 (5.4%) palliative treatment. The corresponding data for the 56 patients who received CT alone were 42 (75.0%), 11 (19.6%), and 3 (5.4%). Overall, 44 patients (39.3%) in the PET-CT group received more extensive CRT or palliative treatment compared with 14 patients (25.0%) in the CT group (odds ratio, 2.05; 95% CI, 0.96-4.37; P = .06). Twenty-four patients in the PET-CT group (21.4%) received extended field radiotherapy to para-aortic nodes and 14 (12.5%) to common iliac nodes compared with 8 (14.3%) and 3 (5.4%), respectively, in the CT group (odds ratio, 1.64; 95% CI, 0.68-3.92; P = .27).

CONCLUSIONS AND RELEVANCE

There was a trend for more extensive CRT with PET-CT, but the difference was not significant because the trial was underpowered. This trial provides information on the utility of PET-CT for staging in LACC.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT00895349.

Impact of Biomarker-based Design Strategies on the Risk of False-Positive Findings in Targeted Therapy Evaluation

PURPOSE

When there is more than one potentially predictive biomarker for a new drug, the drug is often evaluated in different subpopulations defined by different biomarkers. We aim to (i) estimate the risk of false-positive findings with this approach and (ii) evaluate the cross-validated adaptive signature design (CVASD) as a potential alternative.

EXPERIMENTAL DESIGN

By using numerically simulated data, we compare the current approach and the CVASD across different settings and scenarios. We consider three strategies for CVASD. The first two CVASD strategies are different in terms of the partitioning of the overall significance level (between the population test and the subgroup test). In the third CVASD strategy, the order of the two tests is reversed, that is, the population test is realized when the prioritized subgroup test is not statistically significant.

RESULTS

The current approach results in a high risk of false-positive findings, whereas this risk is close to the nominal level of 5% once applying the CVASD, regardless of the strategy. When the treatment is equally effective to all patients, only the CVASD strategies could specify correctly the absence of a sensitive subgroup. When the treatment is only effective for some sensitive responders, the third CVASD strategy stands out by its ability to correctly identify the predictive biomarker(s).

CONCLUSIONS

The drug-biomarker coevaluation based on a series of independent enrichment trials can result in a high risk of false-positive findings. CVASD with some appropriate adjustments can be a good alternative to overcome this multiplicity issue.

EXAMINING EVIDENCE IN U.S. PAYER COVERAGE POLICIES FOR MULTI-GENE PANELS AND SEQUENCING TESTS

OBJECTIVES

The aim of this study was to examine the evidence payers cited in their coverage policies for multi-gene panels and sequencing tests (panels), and to compare these findings with the evidence payers cited in their coverage policies for other types of medical interventions.

METHODS

We used the University of California at San Francisco TRANSPERS Payer Coverage Registry to identify coverage policies for panels issued by five of the largest US private payers. We reviewed each policy and categorized the evidence cited within as: clinical studies, systematic reviews, technology assessments, cost-effectiveness analyses (CEAs), budget impact studies, and clinical guidelines. We compared the evidence cited in these coverage policies for panels with the evidence cited in policies for other intervention types (pharmaceuticals, medical devices, diagnostic tests and imaging, and surgical interventions) as reported in a previous study.

RESULTS

Fifty-five coverage policies for panels were included. On average, payers cited clinical guidelines in 84 percent of their coverage policies (range, 73-100 percent), clinical studies in 69 percent (50-87 percent), technology assessments 47 percent (33-86 percent), systematic reviews or meta-analyses 31 percent (7-71 percent), and CEAs 5 percent (0-7 percent). No payers cited budget impact studies in their policies. Payers less often cited clinical studies, systematic reviews, technology assessments, and CEAs in their coverage policies for panels than in their policies for other intervention types. Payers cited clinical guidelines in a comparable proportion of policies for panels and other technology types.

CONCLUSIONS

Payers in our sample less often cited clinical studies and other evidence types in their coverage policies for panels than they did in their coverage policies for other types of medical interventions.

Making genomic medicine evidence-based and patient-centered: a structured review and landscape analysis of comparative effectiveness research

Comparative effectiveness research (CER) in genomic medicine (GM) measures the clinical utility of using genomic information to guide clinical care in comparison to appropriate alternatives. We summarized findings of high-quality systematic reviews that compared the analytic and clinical validity and clinical utility of GM tests. We focused on clinical utility findings to summarize CER-derived evidence about GM and identify evidence gaps and future research needs. We abstracted key elements of study design, GM interventions, results, and study quality ratings from 21 systematic reviews published in 2010 through 2015. More than half (N = 13) of the reviews were of cancer-related tests. All reviews identified potentially important clinical applications of the GM interventions, but most had significant methodological weaknesses that largely precluded any conclusions about clinical utility. Twelve reviews discussed the importance of patient-centered outcomes, although few described evidence about the impact of genomic medicine on these outcomes. In summary, we found a very limited body of evidence about the effect of using genomic tests on health outcomes and many evidence gaps for CER to address.Genet Med advance online publication 13 April 2017.

Evidence for Treatment-by-Biomarker interaction for FDA-approved Oncology Drugs with Required Pharmacogenomic Biomarker Testing

For oncology drugs that were approved by the US Food and Drug Administration (FDA) and required pharmacogenomic biomarker testing, we describe 1) the use of enrichment (biomarker-positive patients) and a randomized controlled design by pre-approval trials and 2) the treatment-by-biomarker interaction. From the 137 drugs included in the FDA table, we selected the 22 oncology drugs with required genetic testing in their labels. These drugs corresponded to 35 approvals supported by 80 clinical studies included in the FDA medical officer reviews of efficacy. For two thirds of approvals (24/35, 69%), all clinical studies were restricted to biomarker-positive patients (enriched). Among the 11 remaining approvals with at least one non-enriched trial, for five approvals, the non-enriched studies were non-randomized. The treatment-by-biomarker interaction was statistically significant for three approvals and missing for two. Among the six approvals with a non-enriched randomized controlled trial, three featured a statistically significant treatment-by-biomarker interaction (p < 0.10), for an enhanced treatment effect in the biomarker-positive subgroup. For two thirds of FDA approvals of anticancer agents, the requirement for predictive biomarker testing was based on clinical development restricted to biomarker-positive patients. We found only few cases with clinical evidence that biomarker-negative patients would not benefit from treatment.

Payer decision making for next-generation sequencing-based genetic tests: insights from cell-free DNA prenatal screening

PURPOSE

Cell-free DNA (cfDNA) prenatal screening tests have been rapidly adopted into clinical practice, due in part to positive insurance coverage. We evaluated the framework payers used in making coverage decisions to describe a process that should be informative for other sequencing tests.

METHODS

We analyzed coverage policies from the 19 largest US private payers with publicly available policies through February 2016, building from the University of California San Francisco TRANSPERS Payer Coverage Policy Registry.

RESULTS

All payers studied cover cfDNA screening for detection of trisomies 21, 18, and 13 in high-risk, singleton pregnancies, based on robust clinical validity (CV) studies and modeled evidence of clinical utility (CU). Payers typically evaluated the evidence for each chromosomal abnormality separately, although results are offered as part of a panel. Starting in August 2015, 8 of the 19 payers also began covering cfDNA screening in average-risk pregnancies, citing recent CV studies and updated professional guidelines. Most payers attempted, but were unable, to independently assess analytic validity (AV).

CONCLUSION

Payers utilized the standard evidentiary framework (AV/CV/CU) when evaluating cfDNA screening but varied in their interpretation of the sufficiency of the evidence. Professional guidelines, large CV studies, and decision analytic models regarding health outcomes appeared highly influential in coverage decisions.Genet Med advance online publication 22 September 2016.

Clinical Genomic Profiling of a Diverse Array of Oncology Specimens at a Large Academic Cancer Center: Identification of Targetable Variants and Experience with Reimbursement

Large cancer panels are being increasingly used in the practice of precision medicine to generate genomic profiles of tumors with the goal of identifying targetable variants and guiding eligibility for clinical trials. To facilitate identification of mutations in a broad range of solid and hematological malignancies, a 467-gene oncology panel (Columbia Combined Cancer Panel) was developed in collaboration with pathologists and oncologists and is currently available and in use for clinical diagnostics. Herein, we share our experience with this testing in an academic medical center. Of 255 submitted specimens, which encompassed a diverse range of tumor types, we were able to successfully sequence 92%. The Columbia Combined Cancer Panel assay led to the detection of a targetable variant in 48.7% of cases. However, although we show good clinical performance and diagnostic yield, third-party reimbursement has been poor. Reimbursement from government and third-party payers using the 81455 Current Procedural Terminology code was at 19.4% of billed costs, and 55% of cases were rejected on first submission. Likely contributing factors to this low level of reimbursement are the delays in valuation of the 81455 Current Procedural Terminology code and in establishing national or local coverage determinations. In the absence of additional demonstrations of clinical utility and improved patient outcomes, we expect the reimbursement environment will continue to limit the availability of this testing more broadly.

The clinical utility of molecular genetic cancer profiling

INTRODUCTION

Next-Generation-Sequencing (NGS) has enabled gene mutation profiling - cataloguing sequence variants and modifications in clinical assays encompassing tens to thousands of genes in tumors and in germlines. The clinical benefit of applying multi-gene NGS to diverse applications in various malignancies remains to be demonstrated.

AREAS COVERED

Applications of gene mutation profiling in oncology include screening cancer-prone families, classification of malignancies, treatment selection, and monitoring the response to treatment of solid tumors (the 'liquid biopsy'). Google Scholar was used to search PubMed for the period 2011-2016 using combinations of the following search terms: 'clinical utility', NGS, 'molecular diagnostics'. Expert commentary: Clinical studies are in progress pairing mutation profiling with streamlined new trial designs to speed identification of promising drug-target combinations and to see if genotype-informed treatment selection will improve outcome across a spectrum of histologies. The analytical advantages and falling cost of NGS make focused gene panels likely to become the dominant modality in molecular diagnostic testing even if trials eventually discourage use of large panels to test all malignancies.