Risk scores and prognostic models in surgery: pancreas resection as a paradigm

Consensus Statement on Mandatory Measurements for Pancreatic Cancer Trials for Patients With Resectable or Borderline Resectable Disease (COMM-PACT-RB): A Systematic Review and Delphi Consensus Statement

Importance

Pancreatic cancer is the third most common cause of cancer death; however, randomized clinical trials (RCTs) of survival in patients with resectable pancreatic cancer lack mandatory measures for reporting baseline and prognostic factors, which hampers comparisons between outcome measures.

Objective

To develop a consensus on baseline and prognostic factors to be used as mandatory measurements in RCTs of resectable and borderline resectable pancreatic cancer.

Evidence Review

We performed a systematic literature search of the Cochrane Central Register of Controlled Trials (CENTRAL), PubMed, and Embase for RCTs on resectable and borderline resectable pancreatic cancer with overall survival as the primary outcome. We produced a systematic summary of all baseline and prognostic factors identified in the RCTs. A Delphi panel that included 13 experts was surveyed to reach a consensus on mandatory and recommended baseline and prognostic factors.

Findings

The 42 RCTs that met inclusion criteria reported a total of 60 baseline and 19 prognostic factors. After 2 Delphi rounds, agreement was reached on 50 mandatory baseline and 20 mandatory prognostic factors for future RCTs, with a distinction between studies of neoadjuvant vs adjuvant treatment.

Conclusion and Relevance

This findings of this systematic review and international expert consensus have produced this Consensus Statement on Mandatory Measurements in Pancreatic Cancer Trials for Resectable and Borderline Resectable Disease (COMM-PACT-RB). The baseline and prognostic factors comprising the mandatory measures will facilitate better comparison across RCTs and eventually will enable improved clinical practice among patients with resectable and borderline resectable pancreatic cancer.

G. H, Macleod A, Prasanth M, Zafar M, Hemadasa N, Neagle G, Rosindell I, Apollos J. A predictive Bayesian network that risk stratifies patients undergoing Barrett's surveillance for personalized risk of developing malignancy

BACKGROUND

Barrett's esophagus is strongly associated with esophageal adenocarcinoma. Considering costs and risks associated with invasive surveillance endoscopies better methods of risk stratification are required to assist decision-making and move toward more personalised tailoring of Barrett's surveillance.

METHODS

A Bayesian network was created by synthesizing data from published studies analysing risk factors for developing adenocarcinoma in Barrett's oesophagus through a two-stage weighting process.

RESULTS

Data was synthesized from 114 studies (n = 394,827) to create the Bayesian network, which was validated against a prospectively maintained institutional database (n = 571). Version 1 contained 10 variables (dysplasia, gender, age, Barrett's segment length, statin use, proton pump inhibitor use, BMI, smoking, aspirin and NSAID use) and achieved AUC of 0.61. Version 2 contained 4 variables with the strongest evidence of association with the development of adenocarcinoma in Barrett's (dysplasia, gender, age, Barrett's segment length) and achieved an AUC 0.90.

CONCLUSION

This Bayesian network is unique in the way it utilizes published data to translate the existing empirical evidence surrounding the risk of developing adenocarcinoma in Barrett's esophagus to make personalized risk predictions. Further work is required but this tool marks a vital step towards delivering a more personalized approach to Barrett's surveillance.

A prognostic Bayesian network that makes personalized predictions of poor prognostic outcome post resection of pancreatic ductal adenocarcinoma

BACKGROUND

The narrative surrounding the management of potentially resectable pancreatic cancer is complex. Surgical resection is the only potentially curative treatment. However resection rates are low, the risk of operative morbidity and mortality are high, and survival outcomes remain poor. The aim of this study was to create a prognostic Bayesian network that pre-operatively makes personalized predictions of post-resection survival time of 12months or less and also performs post-operative prognostic updating.

METHODS

A Bayesian network was created by synthesizing data from PubMed post-resection survival analysis studies through a two-stage weighting process. Input variables included: inflammatory markers, tumour factors, tumour markers, patient factors and, if applicable, response to neoadjuvant treatment for pre-operative predictions. Prognostic updating was performed by inclusion of post-operative input variables including: pathology results and adjuvant therapy.

RESULTS

77 studies (n = 31,214) were used to create the Bayesian network, which was validated against a prospectively maintained tertiary referral centre database (n = 387). For pre-operative predictions an Area Under the Curve (AUC) of 0.7 (P value: 0.001; 95% CI 0.589-0.801) was achieved accepting up to 4 missing data-points in the dataset. For prognostic updating an AUC 0.8 (P value: 0.000; 95% CI:0.710-0.870) was achieved when validated against a dataset with up to 6 missing pre-operative, and 0 missing post-operative data-points. This dropped to AUC: 0.7 (P value: 0.000; 95% CI:0.667-0.818) when the post-operative validation dataset had up to 2 missing data-points.

CONCLUSION

This Bayesian network is currently unique in the way it utilizes PubMed and patient level data to translate the existing empirical evidence surrounding potentially resectable pancreatic cancer to make personalized prognostic predictions. We believe such a tool is vital in facilitating better shared decision-making in clinical practice and could be further developed to offer a vehicle for delivering personalized precision medicine in the future.

A systematic review of methodological quality of model development studies predicting prognostic outcome for resectable pancreatic cancer

OBJECTIVES

To assess the methodological quality of prognostic model development studies pertaining to post resection prognosis of pancreatic ductal adenocarcinoma (PDAC).

DESIGN/SETTING

A narrative systematic review of international peer reviewed journals DATA SOURCE: Searches were conducted of: MEDLINE, Embase, PubMed, Cochrane database and Google Scholar for predictive modelling studies applied to the outcome of prognosis for patients with PDAC post resection. Predictive modelling studies in this context included prediction model development studies with and without external validation and external validation studies with model updating. Data was extracted following the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) checklist.

PRIMARY AND SECONDARY OUTCOME MEASURES

Primary outcomes were all components of the CHARMS checklist. Secondary outcomes included frequency of variables included across predictive models.

RESULTS

263 studies underwent full text review. 15 studies met the inclusion criteria. 3 studies underwent external validation. Multivariable Cox proportional hazard regression was the most commonly employed modelling method (n=13). 10 studies were based on single centre databases. Five used prospective databases, seven used retrospective databases and three used cancer data registry. The mean number of candidate predictors was 19.47 (range 7 to 50). The most commonly included variables were tumour grade (n=9), age (n=8), tumour stage (n=7) and tumour size (n=5). Mean sample size was 1367 (range 50 to 6400). 5 studies reached statistical power. None of the studies reported blinding of outcome measurement for predictor values. The most common form of presentation was nomograms (n=5) and prognostic scores (n=5) followed by prognostic calculators (n=3) and prognostic index (n=2).

CONCLUSIONS

Areas for improvement in future predictive model development have been highlighted relating to: general aspects of model development and reporting, applicability of models and sources of bias.

TRIAL REGISTRATION NUMBER

CRD42018105942.

Personalized Pancreatic Cancer Management: A Systematic Review of How Machine Learning Is Supporting Decision-making

This review critically analyzes how machine learning is being used to support clinical decision-making in the management of potentially resectable pancreatic cancer. Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, electronic searches of MEDLINE, Embase, PubMed, and Cochrane Database were undertaken. Studies were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) checklist. In total 89,959 citations were retrieved. Six studies met the inclusion criteria. Three studies were Markov decision-analysis models comparing neoadjuvant therapy versus upfront surgery. Three studies predicted survival time using Bayesian modeling (n = 1) and artificial neural network (n = 1), and one study explored machine learning algorithms including Bayesian network, decision trees, k-nearest neighbor, and artificial neural networks. The main methodological issues identified were limited data sources, which limits generalizability and potentiates bias; lack of external validation; and the need for transparency in methods of internal validation, consecutive sampling, and selection of candidate predictors. The future direction of research relies on expanding our view of the multidisciplinary team to include professionals from computing and data science with algorithms developed in conjunction with clinicians and viewed as aids, not replacement, to traditional clinical decision-making.

A simple matrix to predict treatment success and long-term survival among patients undergoing pancreatectomy

BACKGROUND

A more accurate measure of long-term survival among patients who have undergone a successful resection for pancreatic adenocarcinoma may be computed by accounting for time already survived during the initial treatment window.

METHODS

Patients diagnosed with pancreatic adenocarcinoma, from 2004 through 2013, were identified from the American College of Surgeons National Cancer Database (NCDB). A risk-stratification matrix was constructed including age, histopathologic factors and the use of adjuvant therapy, given successful treatment and survival at 3-month following diagnosis.

RESULTS

A total of 25,897 patients (50% male, 53% >65 years of age) presented with stage I-III pancreatic cancer. The majority of patients had tumors >2 cm size (82%), grade I/II (65%), lymphatic invasion (LI) (66%), and negative margins (76%). A survival advantage for adjuvant therapy was observed among all patients, independent of their risk-profile. For example, a patient ≤65 years of age, with early stage cancer (size ≤2 cm, grade I/II, -ve LI, -ve margins) who received adjuvant therapy had a 62% probability of being alive beyond three years (95%CI = 59%-66%). In contrast, the survival probability decreased to 53% (95%CI = 59%-66%) without adjuvant therapy.

CONCLUSIONS

These results provide surgeons and patients with more accurate information regarding long-term survival, as well as the benefit of opting for adjuvant therapy after successful pancreatic surgery.

Advances in Surgical Management of Pancreatic Diseases

The surgical management of pancreatic diseases is rapidly evolving, encompassing advances in evidence-driven selection of patients amenable for surgical therapy, preoperative risk stratification, refinements in the technical conduct of pancreatic operations, and quantification of postoperative morbidity. These advances have resulted in dramatic reductions in mortality following pancreatic surgery, particularly at high-volume pancreatic centers. Surgical decision making is complex, and requires an intimate understanding of disease pathobiology, host physiology, technical considerations, and evolving trends. This article highlights key developments in the contemporary surgical management of pancreatic diseases.

Calculating the risk of a pancreatic fistula after a pancreaticoduodenectomy: a systematic review

BACKGROUND

A post-operative pancreatic fistula (POPF) is a major cause of morbidity and mortality after a pancreaticoduodenectomy (PD). This systematic review aimed to identify all scoring systems to predict POPF after a PD, consider their clinical applicability and assess the study quality.

METHOD

An electronic search was performed of Medline (1946-2014) and EMBASE (1996-2014) databases. Results were screened according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, and quality assessed according to the QUIPS (quality in prognostic studies) tool.

RESULTS

Six eligible scoring systems were identified. Five studies used the International Study Group on Pancreatic Fistula (ISGPF) definition. The proposed scores feature between two and five variables and of the 16 total variables, the majority (12) featured in only one score. Three scores could be fully completed pre-operatively whereas 1 score included intra-operative and two studies post-operative variables. Four scores were internally validated and of these, two scores have been subject to subsequent multicentre review. The median QUIPS score was 38 out of 50 (range 16-50).

CONCLUSION

These scores show potential in calculating the individualized patient risk of POPF. There is, however, much variation in current scoring systems and further validation in large multicentre cohorts is now needed.

Feasibility of Comparing the Results of Pancreatic Resections between Surgeons: A Systematic Review and Meta-Analysis of Pancreatic Resections

Background. Indicators of operative outcomes could be used to identify underperforming surgeons for support and training. The feasibility of identifying HPB surgeons with poor operative performance ("outliers") based on the results of pancreatic resections is not known. Methods. A systematic review of Medline, Embase, and the Cochrane library was performed to identify studies on pancreatic resection including at least 100 patients and published between 2004 and 2014. Proportions that lay outside the upper 95% and 99.8% confidence intervals based on results of the systematic reviews were considered as "outliers." Results. In total, 30 studies reporting on 10712 patients were eligible for inclusion in this review. The average short-term mortality after pancreatic resections was 3.1% and proportion of patients with procedure-related complications was 47.0%. None of the classification systems assessed the long-term impact of the complications on patients. The surgeon-specific mortality should be 5 times the average mortality before he or she can be identified as an outlier with 0.1% false positive rate if he or she performs 50 surgeries a year. Conclusions. A valid risk prognostic model and a classification system of surgical complications are necessary before meaningful comparisons of the operative performance between pancreatic surgeons can be made.