Statistical challenges in the development and evaluation of marker-based clinical tests

Circulating tumour cell gene expression and chemosensitivity analyses: predictive accuracy for response to multidisciplinary treatment of patients with unresectable refractory recurrent rectal cancer or unresectable refractory colorectal cancer liver metastases

Background

Patients with unresectable recurrent rectal cancer (RRC) or colorectal cancer (CRC) with liver metastases, refractory to at least two lines of traditional systemic therapy, may receive third line intraarterial chemotherapy (IC) and targeted therapy (TT) using drugs selected by chemosensitivity and tumor gene expression analyses of liquid biopsy-derived circulating tumor cells (CTCs).

Methods

In this retrospective study, 36 patients with refractory unresectable RRC or refractory unresectable CRC liver metastases were submitted for IC and TT with agents selected by precision oncotherapy chemosensitivity assays performed on liquid biopsy-derived CTCs, transiently cultured in vitro, and by tumor gene expression in the same CTC population, as a ratio to tumor gene expression in peripheral mononuclear blood cells (PMBCs) from the same individual. The endpoint was to evaluate the predictive accuracy of a specific liquid biopsy precision oncotherapy CTC purification and in vitro culture methodology for a positive RECIST 1.1 response to the therapy selected.

Results

Our analyses resulted in evaluations of 94.12% (95% CI 0.71–0.99) for sensitivity, 5.26% (95% CI 0.01–0.26) for specificity, a predictive value of 47.06% (95% CI 0.29–0.65) for a positive response, a predictive value of 50% (95% CI 0.01–0.98) for a negative response, with an overall calculated predictive accuracy of 47.22% (95% CI 0.30–0.64).

Conclusions

This is the first reported estimation of predictive accuracy derived from combining chemosensitivity and tumor gene expression analyses on liquid biopsy-derived CTCs, transiently cultured in vitro which, despite limitations, represents a baseline and benchmark which we envisage will be improve upon by methodological and technological advances and future clinical trials.

Association of B7-H4, PD-L1, and tumor infiltrating lymphocytes with outcomes in breast cancer

B7-H4 (VTCN1) is a member of the CD28/B7 family of immune co-inhibitory molecules. The relationship of tumor and stromal B7-H4 protein expression with PD-L1, tumor infiltrating lymphocytes (TILs) and its association with clinico-pathological variables are not well defined. Herein, we explore the expression level of B7-H4 protein in breast cancer and evaluate its association with TILs, levels of PD-L1 expression, and clinico-pathological characteristics in two independent populations. In this study, we used multiplexed automated quantitative immunofluorescence (QIF) to measure the levels of B7-H4 and PD-L1 protein and determined TILs through pathologist assessment of H&E-stained preparations in over a thousand breast cancer cases from two institutions represented in tissue microarray format. Associations between the marker levels, major clinico-pathological variables, and survival were analyzed. We detected B7-H4 protein was highly expressed in both breast cancer and stromal cells. Its expression was independent of breast cancer intrinsic subtypes. PD-L1 expression was higher in triple negative breast cancers. Neither B7-H4 nor PD-L1 were associated with survival in breast cancer. Our study shows there is a mutually exclusive pattern of B7-H4 with both tumor PD-L1 expression and TILs in all breast cancers, independent of breast cancer intrinsic subtype. This exclusive pattern suggests that some breast tumors may preferentially use one B7-related immune evasion mechanism/pathway. This could explain the clinical benefit that is seen only in a fraction of patients with immune checkpoint inhibitors directed exclusively towards PD-L1 in breast cancer.

Protein Biomarkers for Early Detection of Pancreatic Ductal Adenocarcinoma: Progress and Challenges

Approximately 75% of patients with pancreatic ductal adenocarcinoma are diagnosed with advanced cancer, which cannot be safely resected. The most commonly used biomarker CA19-9 has inadequate sensitivity and specificity for early detection, which we define as Stage I/II cancers. Therefore, progress in next-generation biomarkers is greatly needed. Recent reports have validated a number of biomarkers, including combination assays of proteins and DNA mutations; however, the history of translating promising biomarkers to clinical utility suggests that several major hurdles require careful consideration by the medical community. The first set of challenges involves nominating and verifying biomarkers. Candidate biomarkers need to discriminate disease from benign controls with high sensitivity and specificity for an intended use, which we describe as a two-tiered strategy of identifying and screening high-risk patients. Community-wide efforts to share samples, data, and analysis methods have been beneficial and progress meeting this challenge has been achieved. The second set of challenges is assay optimization and validating biomarkers. After initial candidate validation, assays need to be refined into accurate, cost-effective, highly reproducible, and multiplexed targeted panels and then validated in large cohorts. To move the most promising candidates forward, ideally, biomarker panels, head-to-head comparisons, meta-analysis, and assessment in independent data sets might mitigate risk of failure. Much more investment is needed to overcome these challenges. The third challenge is achieving clinical translation. To moonshot an early detection test to the clinic requires a large clinical trial and organizational, regulatory, and entrepreneurial know-how. Additional factors, such as imaging technologies, will likely need to improve concomitant with molecular biomarker development. The magnitude of the clinical translational challenge is uncertain, but interdisciplinary cooperation within the PDAC community is poised to confront it.

Planning clinically relevant biomarker validation studies using the "number needed to treat" concept

Purpose

Despite an explosion of translational research to exploit biomarkers in diagnosis, prediction and prognosis, the impact of biomarkers on clinical practice has been limited. The elusiveness of clinical utility may partly originate when validation studies are planned, from a failure to articulate precisely how the biomarker, if successful, will improve clinical decision-making for patients. Clarifying what performance would suffice if the test is to improve medical care makes it possible to design meaningful validation studies. But methods for tackling this part of validation study design are undeveloped, because it demands uncomfortable judgments about the relative values of good and bad outcomes resulting from a medical decision.

Methods

An unconventional use of “number needed to treat” (NNT) can structure communication for the trial design team, to elicit purely value-based outcome tradeoffs, conveyed as the endpoints of an NNT “discomfort range”. The study biostatistician can convert the endpoints into desired predictive values, providing criteria for designing a prospective validation study. Next, a novel “contra-Bayes” theorem converts those predictive values into target sensitivity and specificity criteria, to guide design of a retrospective validation study. Several examples demonstrate the approach.

Conclusion

In practice, NNT-guided dialogues have contributed to validation study planning by tying it closely to specific patient-oriented translational goals. The ultimate payoff comes when the report of the completed study includes motivation in the form of a biomarker test framework directly reflecting the clinical decision challenge to be solved. Then readers will understand better what the biomarker test has to offer patients.

Electronic supplementary material

The online version of this article (doi:10.1186/s12967-016-0862-4) contains supplementary material, which is available to authorized users.

Summary and Recommendations from the National Cancer Institute's Clinical Trials Planning Meeting on Novel Therapeutics for Non-Muscle Invasive Bladder Cancer

The NCI Bladder Cancer Task Force convened a Clinical Trials Planning Meeting (CTPM) Workshop focused on Novel Therapeutics for Non-Muscle Invasive Bladder Cancer (NMIBC). Meeting attendees included a broad and multi-disciplinary group of clinical and research stakeholders and included leaders from NCI, FDA, National Clinical Trials Network (NCTN), advocacy and the pharmaceutical and biotech industry. The meeting goals and objectives were to: 1) create a collaborative environment in which the greater bladder research community can pursue future optimally designed novel clinical trials focused on the theme of molecular targeted and immune-based therapies in NMIBC; 2) frame the clinical and translational questions that are of highest priority; and 3) develop two clinical trial designs focusing on immunotherapy and molecular targeted therapy. Despite successful development and implementation of large Phase II and Phase III trials in bladder and upper urinary tract cancers, there are no active and accruing trials in the NMIBC space within the NCTN. Disappointingly, there has been only one new FDA approved drug (Valrubicin) in any bladder cancer disease state since 1998. Although genomic-based data for bladder cancer are increasingly available, translating these discoveries into practice changing treatment is still to come. Recently, major efforts in defining the genomic characteristics of NMIBC have been achieved. Aligned with these data is the growing number of targeted therapy agents approved and/or in development in other organ site cancers and the multiple similarities of bladder cancer with molecular subtypes in these other cancers. Additionally, although bladder cancer is one of the more immunogenic tumors, some tumors have the ability to attenuate or eliminate host immune responses. Two trial concepts emerged from the meeting including a window of opportunity trial (Phase 0) testing an FGFR3 inhibitor and a second multi-arm multi-stage trial testing combinations of BCG or radiotherapy and immunomodulatory agents in patients who recur after induction BCG (BCG failure).

Designing a study to evaluate the benefit of a biomarker for selecting patient treatment

Biomarkers that predict the efficacy of treatment can potentially improve clinical outcomes and decrease medical costs by allowing treatment to be provided only to those most likely to benefit. We consider the design of a randomized clinical trial in which one objective is to evaluate a treatment selection marker. The marker may be measured prospectively or retrospectively using samples collected at baseline. We describe and contrast criteria around which the trial can be designed. An existing approach focuses on determining if there is a statistical interaction between the marker and treatment. We propose three alternative approaches based on estimating clinically relevant measures of improvement in outcomes with use of the marker. Importantly, our approaches accommodate the common scenario in which the marker-based rule for recommending treatment is developed with data from the trial. Sample sizes are calculated for powering a trial to assess these criteria in the context of adjuvant chemotherapy for the treatment of estrogen-receptor-positive, node-positive breast cancer. In this example, we find that larger sample sizes are generally required for assessing clinical impact than for simply evaluating if there is a statistical interaction between marker and treatment. We also find that retrospectively selecting a case-control subset of subjects for marker evaluation can lead to large efficiency gains, especially if cases and controls are matched on treatment assignment.

Target biomarker profile for the clinical management of paracetamol overdose

Paracetamol (acetaminophen) overdose is one of the most common causes of acute liver injury in the Western world. To improve patient care and reduce pressure on already stretched health care providers new biomarkers are needed that identify or exclude liver injury soon after an overdose of paracetamol is ingested. This review highlights the current state of paracetamol poisoning management and how novel biomarkers could improve patient care and save healthcare providers money. Based on the widely used concept of defining a target product profile, a target biomarker profile is proposed that identifies desirable and acceptable key properties for a biomarker in development to enable the improved treatment of this patient population. The current biomarker candidates, with improved hepatic specificity and based on the fundamental mechanistic basis of paracetamol-induced liver injury, are reviewed and their performance compared with our target profile.

Evaluation of a chemoresponse assay as a predictive marker in the treatment of recurrent ovarian cancer: further analysis of a prospective study

BACKGROUND

Recently, a prospective study reported improved clinical outcomes for recurrent ovarian cancer patients treated with chemotherapies indicated to be sensitive by a chemoresponse assay, compared with those patients treated with non-sensitive therapies, thereby demonstrating the assay's prognostic properties. Due to cross-drug response over different treatments and possible association of in vitro chemosensitivity of a tumour with its inherent biology, further analysis is required to ascertain whether the assay performs as a predictive marker as well.

METHODS

Women with persistent or recurrent epithelial ovarian cancer (n=262) were empirically treated with one of 15 therapies, blinded to assay results. Each patient's tumour was assayed for responsiveness to the 15 therapies. The assay's ability to predict progression-free survival (PFS) was assessed by comparing the association when the assayed therapy matches the administered therapy (match) with the association when the assayed therapy is randomly selected, not necessarily matching the administered therapy (mismatch).

RESULTS

Patients treated with assay-sensitive therapies had improved PFS vs patients treated with non-sensitive therapies, with the assay result for match significantly associated with PFS (hazard ratio (HR)=0.67, 95% confidence interval (CI)=0.50-0.91, P=0.009). On the basis of 3000 simulations, the mean HR for mismatch was 0.81 (95% range=0.66-0.99), with 3.4% of HRs less than 0.67, indicating that HR for match is lower than for mismatch. While 47% of tumours were non-sensitive to all assayed therapies and 9% were sensitive to all, 44% displayed heterogeneity in assay results. Improved outcome was associated with the administration of an assay-sensitive therapy, regardless of homogeneous or heterogeneous assay responses across all of the assayed therapies.

CONCLUSIONS

These analyses provide supportive evidence that this chemoresponse assay is a predictive marker, demonstrating its ability to discern specific therapies that are likely to be more effective among multiple alternatives.

Stratification of paracetamol overdose patients using new toxicity biomarkers: current candidates and future challenges

One of the most common causes of acute liver failure in the Western world is paracetamol (acetaminophen) overdose. Specific and sensitive detection of liver injury is important for the prompt and safe treatment of patients with the antidote N-acetylcysteine (NAC) and for the determination of NAC efficacy. Despite many years of intense research, the precise mechanisms of paracetamol-induced liver injury in humans are still not defined, and few studies have examined the optimal dosing regimen for clinical NAC use. It has been widely acknowledged that circulating biomarkers such as microRNA-122, keratin-18 and high mobility group box-1 hold potential to inform on the mechanistic-basis of human drug-induced liver injury. Here, we provide a perspective on the application of these mechanistic biomarkers to the deeper understanding of paracetamol hepatotoxicity in clinical and preclinical studies. Also, we discuss current barriers to using these experimental biomarkers to stratify patients presenting to hospital with this common medical emergency.

Collectives of diagnostic biomarkers identify high-risk subpopulations of hematuria patients: exploiting heterogeneity in large-scale biomarker data

BACKGROUND

Ineffective risk stratification can delay diagnosis of serious disease in patients with hematuria. We applied a systems biology approach to analyze clinical, demographic and biomarker measurements (n = 29) collected from 157 hematuric patients: 80 urothelial cancer (UC) and 77 controls with confounding pathologies.

METHODS

On the basis of biomarkers, we conducted agglomerative hierarchical clustering to identify patient and biomarker clusters. We then explored the relationship between the patient clusters and clinical characteristics using Chi-square analyses. We determined classification errors and areas under the receiver operating curve of Random Forest Classifiers (RFC) for patient subpopulations using the biomarker clusters to reduce the dimensionality of the data.

RESULTS

Agglomerative clustering identified five patient clusters and seven biomarker clusters. Final diagnoses categories were non-randomly distributed across the five patient clusters. In addition, two of the patient clusters were enriched with patients with 'low cancer-risk' characteristics. The biomarkers which contributed to the diagnostic classifiers for these two patient clusters were similar. In contrast, three of the patient clusters were significantly enriched with patients harboring 'high cancer-risk" characteristics including proteinuria, aggressive pathological stage and grade, and malignant cytology. Patients in these three clusters included controls, that is, patients with other serious disease and patients with cancers other than UC. Biomarkers which contributed to the diagnostic classifiers for the largest 'high cancer- risk' cluster were different than those contributing to the classifiers for the 'low cancer-risk' clusters. Biomarkers which contributed to subpopulations that were split according to smoking status, gender and medication were different.

CONCLUSIONS

The systems biology approach applied in this study allowed the hematuric patients to cluster naturally on the basis of the heterogeneity within their biomarker data, into five distinct risk subpopulations. Our findings highlight an approach with the promise to unlock the potential of biomarkers. This will be especially valuable in the field of diagnostic bladder cancer where biomarkers are urgently required. Clinicians could interpret risk classification scores in the context of clinical parameters at the time of triage. This could reduce cystoscopies and enable priority diagnosis of aggressive diseases, leading to improved patient outcomes at reduced costs.