Chapter 11: challenges in and principles for conducting systematic reviews of genetic tests used as predictive indicators

Rapid evidence review of the comparative effectiveness, harms, and cost-effectiveness of pharmacogenomics-guided antidepressant treatment versus usual care for major depressive disorder

OBJECTIVE

This study aims to conduct an evidence review of the effectiveness, harms, and cost-effectiveness of pharmacogenomics-guided antidepressant treatment for major depressive disorder.

METHODS

We searched MEDLINE®, the Cochrane Central Registry of Controlled Trials, and PsycINFO through February 2017. We used prespecified criteria to select studies, abstract data, and rate internal validity and strength of the evidence (PROSPERO number CRD42016036358).

RESULTS

We included two randomized trials (RCT), five controlled cohort studies, and six modeling studies of mostly women in their mid-40s with few comorbidities. CNSDose (ABCB1, ABCC1, CYP2C19, CYP2D6, UGT1A1) is the only pharmacogenomics test that significantly improved remission (one additional remitting patient in 12 weeks per three genotyped, 95% CI 1.7 to 3.5) and reduced intolerability in an RCT. ABCB1 genotyping leads to one additional remitting patient in 5 weeks per three genotyped (95% CI 3 to 20), but tolerability was not reported. In an RCT, GeneSight (CYP2D6, CYPC19, CYP1A2, SLC6A4, HTR2A) did not statistically significantly improve remission, and evidence is inconclusive about its tolerability. Evidence is generally low strength because RCTs were few and underpowered. Cost-effectiveness is unclear due to lack of directly observed cost-effectiveness outcomes. We found no studies that evaluated whether pharmacogenomics shortens time to optimal treatment, whether improvements were due to switches to genetically congruent medication, or whether effectiveness varies based on test and patient characteristics.

CONCLUSIONS

Certain pharmacogenomics tools show promise of improving short-term remission rates in women in their mid-40s with few comorbidities. But, important evidence limitations preclude recommending their widespread use and indicate a need for further research.

The clinical effectiveness and cost-effectiveness of the PROGENSA® prostate cancer antigen 3 assay and the Prostate Health Index in the diagnosis of prostate cancer: a systematic review and economic evaluation

BACKGROUND

There is no single definitive test to identify prostate cancer in men. Biopsies are commonly used to obtain samples of prostate tissue for histopathological examination. However, this approach frequently misses cases of cancer, meaning that repeat biopsies may be necessary to obtain a diagnosis. The PROGENSA(®) prostate cancer antigen 3 (PCA3) assay (Hologic Gen-Probe, Marlborough, MA, USA) and the Prostate Health Index (phi; Beckman Coulter Inc., Brea, CA, USA) are two new tests (a urine test and a blood test, respectively) that are designed to be used to help clinicians decide whether or not to recommend a repeat biopsy.

OBJECTIVE

To evaluate the clinical effectiveness and cost-effectiveness of the PCA3 assay and the phi in the diagnosis of prostate cancer.

DATA SOURCES

Multiple publication databases and trial registers were searched in May 2014 (from 2000 to May 2014), including MEDLINE, EMBASE, The Cochrane Library, ISI Web of Science, Medion, Aggressive Research Intelligence Facility database, ClinicalTrials.gov, International Standard Randomised Controlled Trial Number Register and World Health Organization International Clinical Trials Registry Platform.

REVIEW METHODS

The assessment of clinical effectiveness involved three separate systematic reviews, namely reviews of the analytical validity, the clinical validity of these tests and the clinical utility of these tests. The assessment of cost-effectiveness comprised a systematic review of full economic evaluations and the development of a de novo economic model.

SETTING

The perspective of the evaluation was the NHS in England and Wales.

PARTICIPANTS

Men suspected of having prostate cancer for whom the results of an initial prostate biopsy were negative or equivocal.

INTERVENTIONS

The use of the PCA3 score or phi in combination with existing tests (including histopathology results, prostate-specific antigen level and digital rectal examination), multiparametric magnetic resonance imaging and clinical judgement.

RESULTS

In addition to documents published by the manufacturers, six studies were identified for inclusion in the analytical validity review. The review identified issues concerning the precision of the PCA3 assay measurements. It also highlighted issues relating to the storage requirements and stability of samples intended for analysis using the phi assay. Fifteen studies met the inclusion criteria for the clinical validity review. These studies reported results for 10 different clinical comparisons. There was insufficient evidence to enable the identification of appropriate test threshold values for use in a clinical setting. In addition, the implications of adding either the PCA3 assay or the phi to clinical assessment were not clear. Furthermore, the addition of the PCA3 assay or the phi to clinical assessment plus magnetic resonance imaging was not found to improve discrimination. No published papers met the inclusion criteria for either the clinical utility review or the cost-effectiveness review. The results from the cost-effectiveness analyses indicated that using either the PCA3 assay or the phi in the NHS was not cost-effective.

LIMITATIONS

The main limitations of the systematic review of clinical validity are that the review conclusions are over-reliant on findings from one study, the descriptions of clinical assessment vary widely within reviewed studies and many of the reported results for the clinical validity outcomes do not include either standard errors or confidence intervals.

CONCLUSIONS

The clinical benefit of using the PCA3 assay or the phi in combination with existing tests, scans and clinical judgement has not yet been confirmed. The results from the cost-effectiveness analyses indicate that the use of these tests in the NHS would not be cost-effective.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42014009595.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Chapter 9: options for summarizing medical test performance in the absence of a "gold standard"

The classical paradigm for evaluating test performance compares the results of an index test with a reference test. When the reference test does not mirror the "truth" adequately well (e.g. is an "imperfect" reference standard), the typical ("naïve") estimates of sensitivity and specificity are biased. One has at least four options when performing a systematic review of test performance when the reference standard is "imperfect": (a) to forgo the classical paradigm and assess the index test's ability to predict patient relevant outcomes instead of test accuracy (i.e., treat the index test as a predictive instrument); (b) to assess whether the results of the two tests (index and reference) agree or disagree (i.e., treat them as two alternative measurement methods); (c) to calculate "naïve" estimates of the index test's sensitivity and specificity from each study included in the review and discuss in which direction they are biased; (d) mathematically adjust the "naïve" estimates of sensitivity and specificity of the index test to account for the imperfect reference standard. We discuss these options and illustrate some of them through examples.