Reverse-Bayes methods for evidence assessment and research synthesis

Estimating the Prognostic Value of the NTRK Fusion Biomarker for Comparative Effectiveness Research in The Netherlands

OBJECTIVES

We evaluated the prognostic value of the neurotrophic tyrosine receptor kinase (NTRK) gene fusions by comparing the survival of patients with NTRK+ tumours with patients without NTRK+ tumours.

METHODS

We used genomic and clinical registry data from the Center for Personalized Cancer Treatment (CPCT-02) study containing a cohort of cancer patients who were treated in Dutch clinical practice between 2012 and 2020. We performed a propensity score matching analysis, where NTRK+ patients were matched to NTRK- patients in a 1:4 ratio. We subsequently analysed the survival of the matched sample of NTRK+ and NTRK- patients using the Kaplan-Meier method and Cox regression, and performed an analysis of credibility to evaluate the plausibility of our result.

RESULTS

Among 3556 patients from the CPCT-02 study with known tumour location, 24 NTRK+ patients were identified. NTRK+ patients were distributed across nine different tumour types: bone/soft tissue, breast, colorectal, head and neck, lung, pancreas, prostate, skin and urinary tract. NTRK fusions involving the NTRK3 gene (46%) and NTRK1 gene (33%) were most common. The survival analysis rendered a hazard ratio (HR) of 1.44 (95% CI 0.81-2.55) for NTRK+ patients. Using the point estimates of three prior studies on the prognostic value of NTRK fusions, our finding that the HR is > 1 was deemed plausible.

CONCLUSIONS

NTRK+ patients may have an increased risk of death compared with NTRK- patients. When using historic control data to assess the comparative effectiveness of TRK inhibitors, the prognostic value of the NTRK fusion biomarker should therefore be accounted for.

Reducing the false discovery rate of preclinical animal research with Bayesian statistical decision criteria

The success of preclinical research hinges on exploratory and confirmatory animal studies. Traditional null hypothesis significance testing is a common approach to eliminate the chaff from a collection of drugs, so that only the most promising treatments are funneled through to clinical research phases. Balancing the number of false discoveries and false omissions is an important aspect to consider during this process. In this paper, we compare several preclinical research pipelines, either based on null hypothesis significance testing or based on Bayesian statistical decision criteria. We build on a recently published large-scale meta-analysis of reported effect sizes in preclinical animal research and elicit a non-informative prior distribution under which both approaches are compared. After correcting for publication bias and shrinkage of effect sizes in replication studies, simulations show that (i) a shift towards statistical approaches which explicitly incorporate the minimum clinically important difference reduces the false discovery rate of frequentist approaches and (ii) a shift towards Bayesian statistical decision criteria can improve the reliability of preclinical animal research by reducing the number of false-positive findings. It is shown that these benefits hold while keeping the number of experimental units low which are required for a confirmatory follow-up study. Results show that Bayesian statistical decision criteria can help in improving the reliability of preclinical animal research and should be considered more frequently in practice.

Perspectives on scientific error

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

Comment on "Bayesian additional evidence for decision making under small sample uncertainty"

We examine the concept of Bayesian Additional Evidence (BAE) recently proposed by Sondhi et al. We derive simple closed-form expressions for BAE and compare its properties with other methods for assessing findings in the light of new evidence. We find that while BAE is easy to apply, it lacks both a compelling rationale and clarity of use needed for reliable decision-making.