Evaluating diagnostic strategies for early detection of cancer: the CanTest framework

BACKGROUND

Novel diagnostic triage and testing strategies to support early detection of cancer could improve clinical outcomes. Most apparently promising diagnostic tests ultimately fail because of inadequate performance in real-world, low prevalence populations such as primary care or general community populations. They should therefore be systematically evaluated before implementation to determine whether they lead to earlier detection, are cost-effective, and improve patient safety and quality of care, while minimising over-investigation and over-diagnosis.

METHODS

We performed a systematic scoping review of frameworks for the evaluation of tests and diagnostic approaches.

RESULTS

We identified 16 frameworks: none addressed the entire continuum from test development to impact on diagnosis and patient outcomes in the intended population, nor the way in which tests may be used for triage purposes as part of a wider diagnostic strategy. Informed by these findings, we developed a new framework, the 'CanTest Framework', which proposes five iterative research phases forming a clear translational pathway from new test development to health system implementation and evaluation.

CONCLUSION

This framework is suitable for testing in low prevalence populations, where tests are often applied for triage testing and incorporated into a wider diagnostic strategy. It has relevance for a wide range of stakeholders including patients, policymakers, purchasers, healthcare providers and industry.

Methods for the evaluation of biomarkers in patients with kidney and liver diseases: multicentre research programme including ELUCIDATE RCT

Background

Protein biomarkers with associations with the activity and outcomes of diseases are being identified by modern proteomic technologies. They may be simple, accessible, cheap and safe tests that can inform diagnosis, prognosis, treatment selection, monitoring of disease activity and therapy and may substitute for complex, invasive and expensive tests. However, their potential is not yet being realised.

Design and methods

The study consisted of three workstreams to create a framework for research: workstream 1, methodology – to define current practice and explore methodology innovations for biomarkers for monitoring disease; workstream 2, clinical translation – to create a framework of research practice, high-quality samples and related clinical data to evaluate the validity and clinical utility of protein biomarkers; and workstream 3, the ELF to Uncover Cirrhosis as an Indication for Diagnosis and Action for Treatable Event (ELUCIDATE) randomised controlled trial (RCT) – an exemplar RCT of an established test, the ADVIA Centaur® Enhanced Liver Fibrosis (ELF) test (Siemens Healthcare Diagnostics Ltd, Camberley, UK) [consisting of a panel of three markers – (1) serum hyaluronic acid, (2) amino-terminal propeptide of type III procollagen and (3) tissue inhibitor of metalloproteinase 1], for liver cirrhosis to determine its impact on diagnostic timing and the management of cirrhosis and the process of care and improving outcomes.

Results

The methodology workstream evaluated the quality of recommendations for using prostate-specific antigen to monitor patients, systematically reviewed RCTs of monitoring strategies and reviewed the monitoring biomarker literature and how monitoring can have an impact on outcomes. Simulation studies were conducted to evaluate monitoring and improve the merits of health care. The monitoring biomarker literature is modest and robust conclusions are infrequent. We recommend improvements in research practice. Patients strongly endorsed the need for robust and conclusive research in this area. The clinical translation workstream focused on analytical and clinical validity. Cohorts were established for renal cell carcinoma (RCC) and renal transplantation (RT), with samples and patient data from multiple centres, as a rapid-access resource to evaluate the validity of biomarkers. Candidate biomarkers for RCC and RT were identified from the literature and their quality was evaluated and selected biomarkers were prioritised. The duration of follow-up was a limitation but biomarkers were identified that may be taken forward for clinical utility. In the third workstream, the ELUCIDATE trial registered 1303 patients and randomised 878 patients out of a target of 1000. The trial started late and recruited slowly initially but ultimately recruited with good statistical power to answer the key questions. ELF monitoring altered the patient process of care and may show benefits from the early introduction of interventions with further follow-up. The ELUCIDATE trial was an ‘exemplar’ trial that has demonstrated the challenges of evaluating biomarker strategies in ‘end-to-end’ RCTs and will inform future study designs.

Conclusions

The limitations in the programme were principally that, during the collection and curation of the cohorts of patients with RCC and RT, the pace of discovery of new biomarkers in commercial and non-commercial research was slower than anticipated and so conclusive evaluations using the cohorts are few; however, access to the cohorts will be sustained for future new biomarkers. The ELUCIDATE trial was slow to start and recruit to, with a late surge of recruitment, and so final conclusions about the impact of the ELF test on long-term outcomes await further follow-up. The findings from the three workstreams were used to synthesise a strategy and framework for future biomarker evaluations incorporating innovations in study design, health economics and health informatics.

Trial registration

Current Controlled Trials ISRCTN74815110, UKCRN ID 9954 and UKCRN ID 11930.

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 3. See the NIHR Journals Library website for further project information.

RIPOSTE: a framework for improving the design and analysis of laboratory-based research

Lack of reproducibility is an ongoing problem in some areas of the biomedical sciences. Poor experimental design and a failure to engage with experienced statisticians at key stages in the design and analysis of experiments are two factors that contribute to this problem. The RIPOSTE (Reducing IrreProducibility in labOratory STudiEs) framework has been developed to support early and regular discussions between scientists and statisticians in order to improve the design, conduct and analysis of laboratory studies and, therefore, to reduce irreproducibility. This framework is intended for use during the early stages of a research project, when specific questions or hypotheses are proposed. The essential points within the framework are explained and illustrated using three examples (a medical equipment test, a macrophage study and a gene expression study). Sound study design minimises the possibility of bias being introduced into experiments and leads to higher quality research with more reproducible results.

A comparison of the analytical performance of five commercially available assays for neutrophil gelatinase-associated lipocalin using urine

BACKGROUND

Neutrophil gelatinase-associated lipocalin (NGAL) is a promising biomarker for acute kidney injury that is beginning to be used in clinical practice in addition to research studies. The current study describes an independent validation and comparison of five commercially available NGAL assays, focusing on urine samples. This is an essential step in the translation of this marker to clinical use in terms of allowing valid inter-study comparison and generation of robust results.

METHODS

Two CE (Conformité Européenne)-marked assays, the NGAL Test (BioPorto) on Siemens ADVIA(®) 1800 and the ARCHITECT Urine NGAL assay on i2000SR (Abbott Laboratories), and three research-use-only (RUO) ELISAs (R&D Systems, Hycult and BioPorto) were evaluated. Imprecision, parallelism, recovery, selectivity, limit of quantitation (LOQ), vulnerability to interference and hook effect were assessed and inter-assay agreement was determined using 68 urine samples from patients with various renal diseases and healthy controls.

RESULTS

The Abbott and R&D Systems assays demonstrated satisfactory performance for all parameters tested. However for the other three assays evaluated, problems were identified with LOQ (BioPorto/ADVIA(®)), parallelism (BioPorto ELISA) or several parameters (Hycult). Between-method agreement varied with the Hycult assay in particular being markedly different and highlighting issues with standardization and form of NGAL measured.

CONCLUSIONS

Variability exists between the five NGAL assays in terms of their performance and this should be taken into account when interpreting results from the various clinical or research studies measuring urinary NGAL.

Predicting response to bevacizumab in ovarian cancer: a panel of potential biomarkers informing treatment selection

PURPOSE

The aim of this study was to identify and validate novel predictive and/or prognostic serum proteomic biomarkers in patients with epithelial ovarian cancer (EOC) treated as part of the phase III international ICON7 clinical trial.

EXPERIMENTAL DESIGN

ICON7 was a phase III international trial in EOC which showed a modest but statistically significant benefit in progression-free survival (PFS) with the addition of bevacizumab to standard chemotherapy. Serum samples from 10 patients who received bevacizumab (five responders and five nonresponders) were analyzed by mass spectrometry to identify candidate biomarkers. Initial validation and exploration by immunoassay was undertaken in an independent cohort of 92 patients, followed by a second independent cohort of 115 patients (taken from across both arms of the trial).

RESULTS

Three candidate biomarkers were identified: mesothelin, fms-like tyrosine kinase-4 (FLT4), and α1-acid glycoprotein (AGP). Each showed evidence of independent prognostic potential when adjusting for high-risk status in initial (P < 0.02) and combined (P < 0.01) validation cohorts. In cohort I, individual biomarkers were not predictive of bevacizumab benefit; however, when combined with CA-125, a signature was developed that was predictive of bevacizumab response and discriminated benefit attributable to bevacizumab better than clinical characteristics. The signature showed weaker evidence of predictive ability in validation cohort II, but was still strongly predictive considering all samples (P = 0.001), with an improvement in median PFS of 5.5 months in signature-positive patients in the experimental arm compared with standard arm.

CONCLUSIONS

This study shows a discriminatory signature comprising mesothelin, FLT4, AGP, and CA-125 as potentially identifying those patients with EOC more likely to benefit from bevacizumab. These results require validation in further patient cohorts.

Predicting response to bevacizumab in ovarian cancer: a panel of potential biomarkers informing treatment selection

PURPOSE

The aim of this study was to identify and validate novel predictive and/or prognostic serum proteomic biomarkers in patients with epithelial ovarian cancer (EOC) treated as part of the phase III international ICON7 clinical trial.

EXPERIMENTAL DESIGN

ICON7 was a phase III international trial in EOC which showed a modest but statistically significant benefit in progression-free survival (PFS) with the addition of bevacizumab to standard chemotherapy. Serum samples from 10 patients who received bevacizumab (five responders and five nonresponders) were analyzed by mass spectrometry to identify candidate biomarkers. Initial validation and exploration by immunoassay was undertaken in an independent cohort of 92 patients, followed by a second independent cohort of 115 patients (taken from across both arms of the trial).

RESULTS

Three candidate biomarkers were identified: mesothelin, fms-like tyrosine kinase-4 (FLT4), and α1-acid glycoprotein (AGP). Each showed evidence of independent prognostic potential when adjusting for high-risk status in initial (P < 0.02) and combined (P < 0.01) validation cohorts. In cohort I, individual biomarkers were not predictive of bevacizumab benefit; however, when combined with CA-125, a signature was developed that was predictive of bevacizumab response and discriminated benefit attributable to bevacizumab better than clinical characteristics. The signature showed weaker evidence of predictive ability in validation cohort II, but was still strongly predictive considering all samples (P = 0.001), with an improvement in median PFS of 5.5 months in signature-positive patients in the experimental arm compared with standard arm.

CONCLUSIONS

This study shows a discriminatory signature comprising mesothelin, FLT4, AGP, and CA-125 as potentially identifying those patients with EOC more likely to benefit from bevacizumab. These results require validation in further patient cohorts.

Prognostic utility of pre-operative circulating osteopontin, carbonic anhydrase IX and CRP in renal cell carcinoma

Background:

Objectively measured circulating biomarkers of prognosis complementing existing clinicopathological models are needed in renal cell carcinoma (RCC).

Methods:

Blood samples collected from 216 RCC patients in Leeds before nephrectomy (median follow-up 7 years) were analysed for C-reactive protein (CRP), osteopontin (OPN) and carbonic anhydrase IX (CA9) and prognostic significance determined.

Results:

CA9, OPN and CRP were univariately prognostic for overall survival (OS), cancer-specific survival (CSS) and disease-free survival (DFS) with CRP and CA9 being independently prognostic for OS/CSS and OS, respectively. Including CA9, OPN and CRP with other conventional prognostic factors gave a superior predictive capacity when compared with a previously published pre-operative clinical nomogram (Karakiewicz et al, 2009). Osteopontin outperformed this nomogram and the post-operative SSIGN score for OS but not for CSS, being significantly predictive for non-cancer deaths. Osteopontin, CRP and CA9 outperformed stage (c-index 76% compared with 70% for stage) and OPN or CA9 identified several subsets of poor prognosis patients including in T1 patients, who may benefit from adjuvant therapy and increased surveillance.

Conclusion:

Circulating CA9, OPN and CRP add value to existing clinicopathological prognostic factors/models and support further studies to investigate their potential use in improving the clinical management of RCC.

Measuring carbonic anhydrase IX as a hypoxia biomarker: differences in concentrations in serum and plasma using a commercial enzyme-linked immunosorbent assay due to influences of metal ions

Background

There is increasing interest in measuring the soluble forms of carbonic anhydrase IX (CA IX) in blood as a marker of hypoxia for prognostic purposes or for predictive use in therapeutic trials in various cancers. Following our initial observations of marked differences in the measured concentrations of CA IX in EDTA plasma versus serum, we sought to investigate these further in order to determine their effects on results in published studies and to ensure accurate measurement in future studies.

Methods

Serum and EDTA plasma samples from healthy controls and patients with renal cancer were used in the validation of two commercially available enzyme-linked immunosorbent assays (ELISAs) for CA IX with examination of recovery, parallelism and specificity and comparison of paired plasma and serum.

Results

Successful validation of one of the ELISAs was not achieved with particular problems with parallelism and marked differences in measured CA IX concentrations between EDTA plasma and serum. This appeared to be due to a metal ion-dependent epitope on CA IX recognized by the detection antibody in this assay. The other commercially available ELISA examined was successfully validated and showed no difference in CA IX between EDTA plasma and serum.

Conclusions

These results have important consequences for published studies using this assay where the conclusions drawn from the measurements made may be invalid. This study highlights the need for stringent validation of commercially available assays, including examination of various sample types, before use in research studies.

The potential use of enamel matrix derivative for in situ anterior cruciate ligament tissue engineering: a translational in vitro investigation

Polyester scaffolds have been used as an alternative to autogenous tissues for the reconstruction of the anterior cruciate ligament (ACL). They are biocompatible and encourage tissue infiltration, leading to neoligament formation. However, rupture can occur, caused by abrasion of the scaffold against the bone tunnels through which it is implanted. Good early tissue induction is therefore considered essential to protect the scaffold from this abrasion. Enamel matrix derivative (EMD) is used clinically in the treatment of periodontal disease. It is a complex mix of proteins with growth factor-like activity, which enhances periodontal ligament fibroblast attachment, proliferation, and differentiation, leading to the regeneration of periodontal bone and ligament tissues. We hypothesized that EMD might, in a similar manner, enhance tissue induction around scaffolds used in ACL reconstruction. This preliminary investigation adopted a translational approach, modelling in vitro 3 possible clinical modes of EMD administration, to ascertain the suitability of each protocol for application in an animal model or clinically. Preliminary investigations in monolayer culture indicated that EMD had a significant dose-dependent stimulatory effect (p < 0.05, n = 6) on the proliferation of bovine primary synovial cells. However, pre-treating culture plates with EMD significantly inhibited cell attachment (p < 0.01, n = 6). EMD's effects on synovial cells, seeded onto ligament scaffolds, were then investigated in several in vitro experiments modelling 3 possible modes for clinical EMD administration (pre-, intra-, and post-operative). In the pre-operative model, EMD was adsorbed onto scaffolds before the addition of cells. In the intra-operative model, EMD and cells were added simultaneously to scaffolds in the culture medium. In the post-operative model, cells were pre-seeded onto scaffolds before EMD was administered. EMD significantly inhibited cell adhesion in the pre-operative model (p < 0.05, n = 6) and had no significant benefit in the intra-operative model. In the post-operative model, the addition of EMD to previously cell-seeded scaffolds significantly increased their total deoxyribonucleic acid content (p < 0.01, n = 5). EMD's stimulative effect on cell proliferation in vitro suggests that it may accelerate scaffold colonization by cells (and in turn tissue induction) in situ. However, its inhibitory effect on synovial cell attachment in vitro implies that it may only be suited to post-operative administration.

Global regulation of virulence and the stress response by CsrA in the highly adapted human gastric pathogen Helicobacter pylori

Although successful and persistent colonization of the gastric mucosa depends on the ability to respond to changing environmental conditions and co-ordinate the expression of virulence factors during the course of infection, Helicobacter pylori possesses relatively few transcriptional regulators. We therefore investigated the contribution of the regulatory protein CsrA to global gene regulation in this important human pathogen. CsrA was necessary for full motility and survival of H. pylori under conditions of oxidative stress. Loss of csrA expression deregulated the oxidant-induced transcriptional responses of napA and ahpC, the acid induction of napA, cagA, vacA, the urease operon, and fur, as well as the heat shock responses of napA, groESL and hspR. Although the level of napA transcript was higher in the csrA mutant, its stability was similar in the wild-type and mutant strains, and less NapA protein was produced in the mutant strain. Finally, H. pylori strains deficient in the production of CsrA were significantly attenuated for virulence in a mouse model of infection. This work provides evidence that CsrA has a broad role in regulating the physiology of H. pylori in response to environmental stimuli, and may be important in facilitating adaptation to the different environments encountered during colonization of the gastric mucosa. Furthermore, CsrA appears to mediate its effects in H. pylori at the post-transcriptional level by influencing the processing and translation of target transcripts, with minimal effect on the stability of the target mRNAs.