The Effect of the UK Coordinating Centre for Cancer Research Anal Cancer Trial (ACT1) on Population-based Treatment and Survival for Squamous Cell Cancer of the Anus

Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

BACKGROUND

Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews.

METHODS

In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research  impact evaluation going forward.

RESULTS

In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated.

CONCLUSIONS

Impact evaluation can demonstrate if and why conducting cancer research  is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Temporal improvements in loco-regional failure and survival in patients with anal cancer treated with chemo-radiotherapy: treatment cohort study (1990-2014)

Background

We evaluated oncological changes in patients with squamous cell carcinoma of the anus (SCCA) treated by chemoradiotherapy (CRT) from a large UK institute, to derive estimates of contemporary outcomes.

Methods

We performed a treatment-cohort analysis in 560 patients with non-metastatic SCCA treated with CRT over 25 years. The primary outcomes were 3-year loco-regional failure (LRF), 5-year overall survival (OS), and 5-year cancer-specific survival (CSS). We developed prediction models; and overlaid estimates on published results from historic trials.

Results

Age distributions, proportions by gender and cT stage remained stable over time. The median follow-up was 61 (IQR: 36–79) months. Comparing the first period (1990–1994) with the last period (2010–2014), 3-year LRF declined from 33 to 16% (P_trends < 0.001); 5-year OS increased from 60% to 76% (P_trends = 0.001); and 5-year CCS increased from 62% in to 80% (P_trends = 0.001). For 2020, the models predicted a 3-year LRF of 14.7% (95% CIs: 0–31.3); 5-year OS of 74.7% (95% CIs: 54.6–94.9); and 5-year CSS of 85.7% (95% CIs: 75.3–96.0). Reported oncological outcomes from historic trials generally underestimated contemporary outcomes.

Conclusions

Current and predicted rates for 3-year LRF and 5-year survivals are considerably improved compared with those in historic trials.

Nodal stage migration and prognosis in anal cancer: a systematic review, meta-regression, and simulation study

BACKGROUND

In patients with squamous cell carcinoma of the anus (SCCA), lymph node positivity (LNP) indicates poor prognosis for survival and is central to radiotherapy planning. Over the past three decades, LNP proportion has increased, mainly reflecting enhanced detection with newer imaging modalities; a process known as nodal stage migration. If accompanied by constant T stage distributions, prognosis for both lymph node-positive and lymph node-negative groups may improve without any increase in overall survival for individual patients; a paradox termed the Will Rogers phenomenon. Here, we aim to systematically evaluate the impact of nodal stage migration on survival in SCCA and address a novel hypothesis that this phenomenon results in reduced prognostic discrimination.

METHODS

We did a systematic review and meta-regression to quantify changes in LNP over time and the impact of this change on survival and prognostic discrimination. We searched MEDLINE, Embase, and the Cochrane Library to identify randomised trials and observational studies in patients with SCCA published between Jan 1, 1970, and Oct 11, 2016. Studies were eligible if patients received chemoradiotherapy or radiotherapy as the main treatment, reported LNP proportions (all studies), and reported overall survival (not necessarily present in all studies). We excluded studies with fewer than 50 patients. We extracted study-level data with a standardised, piloted form. The primary outcome measure was 5-year overall survival. To investigate scenarios in which reduced prognostic discrimination might occur, we simulated varying true LNP proportions and true overall survival, and compared these with expected observed outcomes for varying levels of misclassification of true nodal state.

FINDINGS

We identified 62 studies reporting LNP proportions, which included 10 569 patients. From these, we included 45 studies (6302 patients) with whole cohort 5-year overall survival, 11 studies with 5-year survival stratified by nodal status, and 20 studies with hazard ratios in our analyses of temporal changes. In 62 studies, the LNP proportions increased from a mean estimate of 15·3% (95% CI 10·5-20·1) in 1980 to 37·1% (34·0-41·3) in 2012 (p<0·0001). In 11 studies with prognostic data, increasing LNP was associated with improved overall survival in both lymph node-positive and lymph node-negative categories, whereas the proportions with combined tumour stage T3 and T4 remained constant. In 20 studies, across a range of LNP proportions from 15% to 40%, the hazard ratios for overall survival of lymph node-positive versus lymph node-negative patients decreased significantly from 2·5 (95% CI 1·8-3·3) at 15% LNP to 1·3 (1·2-1·9; p=0·014) at 40% LNP. The simulated scenarios reproduced this effect if the true LNP proportions were 20% or 25%, but not if the true LNP proportions were 30% or greater.

INTERPRETATION

We describe a consequence of staging misclassification in anal cancer that we have termed reduced prognostic discrimination. We used this new observation to infer that the LNP proportions of more than 30% seen in modern clinical series (11 out of 15 studies with a median year since 2007) are higher than the true LNP proportion. The introduction of new staging technologies in oncology might misclassify true disease stage, spuriously informing disease management and ultimately increasing the risk of overtreatment.

FUNDING

Bowel Disease Research Foundation.