Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews

The increasing need for systematic reviews of prognosis studies: strategies to facilitate review production and improve quality of primary research

Personalized, precision, and risk-based medicine are becoming increasingly important in medicine. These involve the use of information about the prognosis of a patient, to make individualized treatment decisions. This has led to an accumulating amount of literature available on prognosis studies. To summarize and evaluate this information overload, high-quality systematic reviews are essential, additionally helping us to facilitate interpretation and usability of prognosis study findings and to identify gaps in literature. Four types of prognosis studies can be identified: overall prognosis, prognostic factors, prognostic models, and predictors of treatment effect. Methodologists have focussed on developing methods and tools for every step of a systematic review for reviews of all four types of prognosis studies, from formulating the review question and writing a protocol to searching for studies, assessing risk of bias, meta-analysing results, and interpretation of results. The growing attention for prognosis research has led to the introduction of the Cochrane Prognosis Methods Group (PMG). Since 2016, reviews of prognosis studies are formally implemented within Cochrane. With these recent methodological developments and tools, and the implementation within Cochrane, it becomes increasingly feasible to perform high-quality reviews of prognosis studies that will have an impact on clinical practice.

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Prognostic models for predicting overall survival in metastatic castration-resistant prostate cancer: a systematic review

PURPOSE

Prognostic models are developed to estimate the probability of the occurrence of future outcomes incorporating multiple variables. We aimed to identify and summarize existing multivariable prognostic models developed for predicting overall survival in patients with metastatic castration-resistant prostate cancer (mCRPC).

METHODS

The protocol was prospectively registered (CRD42017064448). We systematically searched Medline and reference lists up to May 2018 and included experimental and observational studies, which developed and/or internally validated prognostic models for mCRPC patients and were further externally validated or updated. The outcome of interest was overall survival. Two authors independently performed literature screening and quality assessment.

RESULTS

We included 12 studies that developed models including 8750 patients aged 42-95 years. Models included 4-11 predictor variables, mostly hemoglobin, baseline PSA, alkaline phosphatase, performance status, and lactate dehydrogenase. Very few incorporated Gleason score. Two models included predictors related to docetaxel and mitoxantrone treatments. Model performance after internal validation showed similar discrimination power ranging from 0.62 to 0.73. Overall survival models were mainly constructed as nomograms or risk groups/score. Two models obtained an overall judgment of low risk of bias.

CONCLUSIONS

Most models were not suitable for clinical use due to methodological shortcomings and lack of external validation. Further external validation and/or model updating is required to increase prognostic accuracy and clinical applicability prior to their incorporation in clinical practice as a useful tool in patient management.

Clinical prediction rules: A systematic review of healthcare provider opinions and preferences

OBJECTIVE

The act of predicting clinical endpoints and patient trajectories based on past and current states is on the precipice of a technological revolution. This systematic review summarises the available evidence describing healthcare provider opinions and preferences with respect to the use of clinical prediction rules. The primary goal of this work is to inform the design and implementation of future systems, and secondarily to identify gaps for the development of clinician education programs.

METHODS

Five databases were systematically searched in May 2016 for studies collecting empirical opinions of healthcare providers regarding clinical prediction rule usage. Reference lists were scanned for additional eligible materials and an update search was made in August 2017. Data was extracted on high-level study features, before in-depth thematic analysis was performed.

RESULTS

45 articles were identified from 9 countries. Most studies utilised surveys (28) or interviews (14). Fewer employed focus groups (9) or formal usability testing (4). Three high-level themes were identified, which form the basis of healthcare provider opinions of clinical prediction rules and their implementation - utility, credibility and usability.

CONCLUSIONS

Some of the objections and preferences stated by healthcare providers are inherent to the nature of the clinical problem addressed, which may or may not be within the designer's capacity to change; however, others (in particular - actionability, validation, integration and provision of high quality education materials) should be considered by prediction rule designers and implementation teams, in order to increase user acceptance and improve uptake of these tools. We summarise these findings across the clinical prediction rule lifecycle and pose questions for the rule developers, in order to produce tools that are more likely to successfully translate into clinical practice.

Evaluating methodological quality of Prognostic models Including Patient-reported HeAlth outcomes iN oncologY (EPIPHANY): a systematic review protocol

INTRODUCTION

While there is mounting evidence of the independent prognostic value of patient-reported outcomes (PROs) for overall survival (OS) in patients with cancer, it is known that the conduct of these studies may hold a number of methodological challenges. The aim of this systematic review is to evaluate the quality of published studies in this research area, in order to identify methodological and statistical issues deserving special attention and to also possibly provide evidence-based recommendations.

METHODS AND ANALYSIS

An electronic search strategy will be performed in PubMed to identify studies developing or validating a prognostic model which includes PROs as predictors. Two reviewers will independently be involved in data collection using a predefined and standardised data extraction form including information related to study characteristics, PROs measures used and multivariable prognostic models. Studies selection will be reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, with data extraction form using fields from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist for multivariable models. Methodological quality assessment will also be performed and will be based on prespecified domains of the CHARMS checklist. As a substantial heterogeneity of included studies is expected, a narrative evidence synthesis will also be provided.

ETHICS AND DISSEMINATION

Given that this systematic review will use only published data, ethical permissions will not be required. Findings from this review will be published in peer-reviewed scientific journals and presented at major international conferences. We anticipate that this review will contribute to identify key areas of improvement for conducting and reporting prognostic factor analyses with PROs in oncology and will lay the groundwork for developing future evidence-based recommendations in this area of research.

PROSPERO REGISTRATION NUMBER

CRD42018099160.

Measurement error and timing of predictor values for multivariable risk prediction models are poorly reported

OBJECTIVE

Measurement error in predictor variables may threaten the validity of clinical prediction models. We sought to evaluate the possible extent of the problem. A secondary objective was to examine whether predictors are measured at the intended moment of model use.

METHODS

A systematic search of Medline was used to identify a sample of articles reporting the development of a clinical prediction model published in 2015. After screening according to a predefined inclusion criteria, information on predictors, strategies to control for measurement error, and intended moment of model use were extracted. Susceptibility to measurement error for each predictor was classified into low and high risks.

RESULTS

Thirty-three studies were reviewed, including 151 different predictors in the final prediction models. Fifty-one (33.7%) predictors were categorized as high risk of error; however, this was not accounted for in the model development. Only 8 (24.2%) studies explicitly stated the intended moment of model use and when the predictors were measured.

CONCLUSION

Reporting of measurement error and intended moment of model use is poor in prediction model studies. There is a need to identify circumstances where ignoring measurement error in prediction models is consequential and whether accounting for the error will improve the predictions.

Mortality after admission for heart failure in the UK compared with Japan

Objective

Mortality amongst patients hospitalised for heart failure (HHF) in Western and Asian countries may differ, but this has not been investigated using individual patient-level data (IPLD). We sought to remedy this through rigorous statistical analysis of HHF registries and variable selection from a systematic literature review.

Methods and results

IPLD from registries of HHF in Japan (n=3781) and the UK (n=894) were obtained. A systematic literature review identified 23 models for predicting outcome of HHF. Five variables appearing in 10 or more reports were strongly related to prognosis (systolic blood pressure, serum sodium concentration, age, blood urea nitrogen and creatinine). To compare mortality in the UK and Japan, variables were imputed in a propensity model using inverse probability of treatment weighting (IPTW) and IPTW with logistic regression (doubly robust IPTW). Overall, patients in the UK were sicker and in-patient and post-discharge mortalities were greater, suggesting that the threshold for hospital admission was higher. Covariate-adjusted in-hospital mortality was similar in the UK and Japan (IPTW OR: 1.14, 95% CI 0.70 to 1.86), but 180-day postdischarge mortality was substantially higher in the UK (doubly robust IPTW OR: 2.33, 95% CI 1.58 to 3.43).

Conclusions

Despite robust methods to adjust for differences in patient characteristics and disease severity, HHF patients in the UK have roughly twice the mortality at 180 days compared with those in Japan. Similar analyses should be done using other data sets and in other countries to determine the consistency of these findings and identify factors that might inform healthcare policy and improve outcomes.

Protease activity as a prognostic factor for wound healing in venous leg ulcers

BACKGROUND

Venous leg ulcers (VLUs) are a common type of complex wound that have a negative impact on people's lives and incur high costs for health services and society. It has been suggested that prolonged high levels of protease activity in the later stages of the healing of chronic wounds may be associated with delayed healing. Protease modulating treatments have been developed which seek to modulate protease activity and thereby promote healing in chronic wounds.

OBJECTIVES

To determine whether protease activity is an independent prognostic factor for the healing of venous leg ulcers.

SEARCH METHODS

In February 2018, we searched the following databases: Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE, Ovid Embase and CINAHL.

SELECTION CRITERIA

We included prospective and retrospective longitudinal studies with any follow-up period that recruited people with VLUs and investigated whether protease activity in wound fluid was associated with future healing of VLUs. We included randomised controlled trials (RCTs) analysed as cohort studies, provided interventions were taken into account in the analysis, and case-control studies if there were no available cohort studies. We also included prediction model studies provided they reported separately associations of individual prognostic factors (protease activity) with healing. Studies of any type of protease or combination of proteases were eligible, including proteases from bacteria, and the prognostic factor could be examined as a continuous or categorical variable; any cut-off point was permitted. The primary outcomes were time to healing (survival analysis) and the proportion of people with ulcers completely healed; the secondary outcome was change in ulcer size/rate of wound closure. We extracted unadjusted (simple) and adjusted (multivariable) associations between the prognostic factor and healing.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed studies for inclusion at each stage, and undertook data extraction, assessment of risk of bias and GRADE assessment. We collected association statistics where available. No study reported adjusted analyses: instead we collected unadjusted results or calculated association measures from raw data. We calculated risk ratios when both outcome and prognostic factor were dichotomous variables. When the prognostic factor was reported as continuous data and healing outcomes were dichotomous, we either performed regression analysis or analysed the impact of healing on protease levels, analysing as the standardised mean difference. When both prognostic factor and outcome were continuous data, we reported correlation coefficients or calculated them from individual participant data.We displayed all results on forest plots to give an overall visual representation. We planned to conduct meta-analyses where this was appropriate, otherwise we summarised narratively.

MAIN RESULTS

We included 19 studies comprising 21 cohorts involving 646 participants. Only 11 studies (13 cohorts, 522 participants) had data available for analysis. Of these, five were prospective cohort studies, four were RCTs and two had a type of case-control design. Follow-up time ranged from four to 36 weeks. Studies covered 10 different matrix metalloproteases (MMPs) and two serine proteases (human neutrophil elastase and urokinase-type plasminogen activators). Two studies recorded complete healing as an outcome; other studies recorded partial healing measures. There was clinical and methodological heterogeneity across studies; for example, in the definition of healing, the type of protease and its measurement, the distribution of active and bound protease species, the types of treatment and the reporting of results. Therefore, meta-analysis was not performed. No study had conducted multivariable analyses and all included evidence was of very low certainty because of the lack of adjustment for confounders, the high risk of bias for all studies except one, imprecision around the measures of association and inconsistency in the direction of association. Collectively the research indicated complete uncertainty as to the association between protease activity and VLU healing.

AUTHORS' CONCLUSIONS

This review identified very low validity evidence regarding any association between protease activity and VLU healing and there is complete uncertainty regarding the relationship. The review offers information for both future research and systematic review methodology.

Repeatability and Reproducibility of Radiomic Features: A Systematic Review

Purpose:

An ever-growing number of predictive models used to inform clinical decision making have included quantitative, computer-extracted imaging biomarkers, or “radiomic features.” Broadly generalizable validity of radiomics-assisted models may be impeded by concerns about reproducibility. We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features, derived from a systematic review of published peer-reviewed literature.

Methods and Materials:

The PubMed electronic database was searched using combinations of the broad Haynes and Ingui filters along with a set of text words specific to cancer, radiomics (including texture analyses), reproducibility, and repeatability. This review has been reported in compliance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. From each full-text article, information was extracted regarding cancer type, class of radiomic feature examined, reporting quality of key processing steps, and statistical metric used to segregate stable features.

Results:

Among 624 unique records, 41 full-text articles were subjected to review. The studies primarily addressed non-small cell lung cancer and oropharyngeal cancer. Only 7 studies addressed in detail every methodologic aspect related to image acquisition, preprocessing, and feature extraction. The repeatability and reproducibility of radiomic features are sensitive at various degrees to processing details such as image acquisition settings, image reconstruction algorithm, digital image preprocessing, and software used to extract radiomic features. First-order features were overall more reproducible than shape metrics and textural features. Entropy was consistently reported as one of the most stable first-order features. There was no emergent consensus regarding either shape metrics or textural features; however, coarseness and contrast appeared among the least reproducible.

Conclusions:

Investigations of feature repeatability and reproducibility are currently limited to a small number of cancer types. Reporting quality could be improved regarding details of feature extraction software, digital image manipulation (preprocessing), and the cutoff value used to distinguish stable features.

We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features. The repeatability and reproducibility of radiomic features are sensitive at various degrees to image quality and to software used to extract radiomic features. Investigations of feature repeatability and reproducibility are currently limited to a small number of cancer types. No consensus was found regarding the most repeatable and reproducible features with respect to different settings.

Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol

Substantial over-diagnosis and under-diagnosis of asthma in adults and children has recently been reported. As asthma is mostly diagnosed in non-specialist settings, a clinical prediction model (CPM) to aid the diagnosis of asthma in primary care may help improve diagnostic accuracy. We aim to systematically identify, describe, compare, and synthesise existing CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations. We will systematically search Medline, Embase and CINAHL from 1 January 1990 to present. Any CPM derived for use in a primary care population will be included. Equivalent populations in countries without a developed primary care service will also be included. The probability of asthma diagnosis will be the primary outcome. We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma. We will include derivation studies, and external model validation studies. Two reviewers will independently screen titles/abstracts and full texts for eligibility and extract data from included papers. The CHARMS checklist (or PROBAST if available) will be used to assess risk of bias within each study. Results will be summarised by narrative synthesis with meta-analyses completed if possible. This systematic review will provide comprehensive information about existing CPMs for the diagnosis of asthma in primary care and will inform the development of a future diagnostic model.

Sex-specific performance of pre-imaging diagnostic algorithms for pulmonary embolism

Essentials Decision rules for pulmonary embolism are used indiscriminately despite possible sex-differences. Various pre-imaging diagnostic algorithms have been investigated in several prospective studies. When analysed at an individual patient data level the algorithms perform similarly in both sexes. Estrogen use and male sex were associated with a higher prevalence in suspected pulmonary embolism.

SUMMARY

Background In patients suspected of pulmonary embolism (PE), clinical decision rules are combined with D-dimer testing to rule out PE, avoiding the need for imaging in those at low risk. Despite sex differences in several aspects of the disease, including its diagnosis, these algorithms are used indiscriminately in women and men. Objectives To compare the performance, defined as efficiency and failure rate, of three pre-imaging diagnostic algorithms for PE between women and men: the Wells rule with fixed or with age-adjusted D-dimer cut-off, and a recently validated algorithm (YEARS). A secondary aim was to determine the sex-specific prevalence of PE. Methods Individual patient data were obtained from six studies using the Wells rule (fixed D-dimer, n = 5; age adjusted, n = 1) and from one study using the YEARS algorithm. All studies prospectively enrolled consecutive patients with suspected PE. Main outcomes were efficiency (proportion of patients in which the algorithm ruled out PE without imaging) and failure rate (proportion of patients with PE not detected by the algorithm). Outcomes were estimated using (multilevel) logistic regression models. Results The main outcomes showed no sex differences in any of the separate algorithms. With all three, the prevalence of PE was lower in women (OR, 0.66, 0.68 and 0.74). In women, estrogen use, adjusted for age, was associated with lower efficiency and higher prevalence and D-dimer levels. Conclusions The investigated pre-imaging diagnostic algorithms for patients suspected of PE show no sex differences in performance. Male sex and estrogen use are both associated with a higher probability of having the disease.

Exacerbations in Adults with Asthma: A Systematic Review and External Validation of Prediction Models

BACKGROUND

Several prediction models assessing future risk of exacerbations in adult patients with asthma have been published. Applicability of these models is uncertain because their predictive performance has often not been assessed beyond the population in which they were derived.

OBJECTIVE

This study aimed to identify and critically appraise prediction models for asthma exacerbations and validate them in 2 clinically distinct populations.

METHODS

PubMed and EMBASE were searched to April 2017 for reports describing adult asthma populations in which multivariable models were constructed to predict exacerbations during any time frame. After critical appraisal, the models' predictive performances were assessed in a primary and a secondary care population for author-defined exacerbations and for American Thoracic Society/European Respiratory Society-defined severe exacerbations.

RESULTS

We found 12 reports from which 24 prediction models were evaluated. Three predictors (previous health care utilization, symptoms, and spirometry values) were retained in most models. Assessment was hampered by suboptimal methodology and reporting, and by differences in exacerbation outcomes. Discrimination (area under the receiver-operating characteristic curve [c-statistic]) of models for author-defined exacerbations was better in the primary care population (mean, 0.71) than in the secondary care population (mean, 0.60) and similar (0.65 and 0.62, respectively) for American Thoracic Society/European Respiratory Society-defined severe exacerbations. Model calibration was generally poor, but consistent between the 2 populations.

CONCLUSIONS

The preservation of 3 predictors in models derived from variable populations and the fairly consistent predictive properties of most models in 2 distinct validation populations suggest the feasibility of a generalizable model predicting severe exacerbations. Nevertheless, improvement of the models is warranted because predictive performances are below the desired level.

Clinical prediction rules for mortality in patients with pulmonary embolism and cancer to guide outpatient management: a meta-analysis

Essentials Clinical prediction rules (CPRs) can stratify patients with pulmonary embolism (PE) and cancer. A meta-analysis was done to assess prognostic accuracy in CPRs for mortality in these patients. Eight studies evaluating ten CPRs were included in this study. CPRs should continue to be used with other patient factors for mortality risk stratification.

SUMMARY

Background Cancer treatment is commonly complicated by pulmonary embolism (PE), which remains a leading cause of morbidity and mortality in these patients. Some guidelines recommend the use of clinical prediction rules (CPRs) to help clinicians identify patients at low risk of mortality and therefore guide care. Objective To determine and compare the accuracy of available CPRs for identifying cancer patients with PE at low risk of mortality. Methods A literature search of Medline and Scopus (January 2000 to August 2017) was performed. Studies deriving/validating ≥ 1 CPR for early post-PE all-cause mortality were included. A bivariate, random-effects model was used to pool sensitivity and specificity estimates for each CPR. Traditional random-effects meta-analysis was performed to estimate the weighted proportion of patients deemed at low risk of early mortality, mortality in low risk patients and odds ratios for death compared with higher-risk patients. Results Eight studies evaluating 10 CPRs were included. The highest sensitivities were observed with Hestia (98.1%, 95% confidence interval [CI] = 75.6-99.9%) and the EPIPHANY index (97.4%, 95% CI = 93.2-99.0%); sensitivities of remaining rules ranged from 59.9 to 96.6%. Of the six CPRs with sensitivities ≥ 95%, none had specificities > 33%. Random-effects meta-analysis suggested that 6.6-51.6% of cancer patients with PE were at low risk of mortality, 0-14.3% of low-risk patients died and low-risk patients had a 43-94% lower odds of death compared with those at higher risk. Conclusions Because of the limited total body of evidence regarding CPRs, their results, in conjunction with other pertinent patient-specific clinical factors, should continue to be used in identifying appropriate management for PE in patients with cancer.

Clinical prediction models for mortality and functional outcome following ischemic stroke: A systematic review and meta-analysis

Objective

We aim to identify and critically appraise clinical prediction models of mortality and function following ischaemic stroke.

Methods

Electronic databases, reference lists, citations were searched from inception to September 2015. Studies were selected for inclusion, according to pre-specified criteria and critically appraised by independent, blinded reviewers. The discrimination of the prediction models was measured by the area under the curve receiver operating characteristic curve or c-statistic in random effects meta-analysis. Heterogeneity was measured using I2. Appropriate appraisal tools and reporting guidelines were used in this review.

Results

31395 references were screened, of which 109 articles were included in the review. These articles described 66 different predictive risk models. Appraisal identified poor methodological quality and a high risk of bias for most models. However, all models precede the development of reporting guidelines for prediction modelling studies. Generalisability of models could be improved, less than half of the included models have been externally validated(n = 27/66). 152 predictors of mortality and 192 predictors and functional outcome were identified. No studies assessing ability to improve patient outcome (model impact studies) were identified.

Conclusions

Further external validation and model impact studies to confirm the utility of existing models in supporting decision-making is required. Existing models have much potential. Those wishing to predict stroke outcome are advised to build on previous work, to update and adapt validated models to their specific contexts opposed to designing new ones.

Ruling out pulmonary embolism across different subgroups of patients and healthcare settings: protocol for a systematic review and individual patient data meta-analysis (IPDMA)

BACKGROUND

Diagnosing pulmonary embolism in suspected patients is notoriously difficult as signs and symptoms are non-specific. Different diagnostic strategies have been developed, usually combining clinical probability assessment with D-dimer testing. However, their predictive performance differs across different healthcare settings, patient subgroups, and clinical presentation, which are currently not accounted for in the available diagnostic approaches.

METHODS

This is a protocol for a large diagnostic individual patient data meta-analysis (IPDMA) of currently available diagnostic studies in the field of pulmonary embolism. We searched MEDLINE (search date January 1, 1995, till August 25, 2016) to retrieve all primary diagnostic studies that had evaluated diagnostic strategies for pulmonary embolism. Two authors independently screened titles, abstracts, and subsequently full-text articles for eligibility from 3145 individual studies. A total of 40 studies were deemed eligible for inclusion into our IPDMA set, and principal investigators from these studies were invited to participate in a meeting at the 2017 conference from the International Society on Thrombosis and Haemostasis. All authors agreed on data sharing and participation into this project. The process of data collection of available datasets as well as potential identification of additional new datasets based upon personal contacts and an updated search will be finalized early 2018. The aim is to evaluate diagnostic strategies across three research domains: (i) the optimal diagnostic approach for different healthcare settings, (ii) influence of comorbidity on the predictive performance of each diagnostic strategy, and (iii) optimize and tailor the efficiency and safety of ruling out PE across a broad spectrum of patients with a new, patient-tailored clinical decision model that combines clinical items with quantitative D-dimer testing.

DISCUSSION

This pre-planned individual patient data meta-analysis aims to contribute in resolving remaining diagnostic challenges of time-efficient diagnosis of pulmonary embolism by tailoring available diagnostic strategies for different healthcare settings and comorbidity.

SYSTEMATIC REVIEW REGISTRATION

Prospero trial registration: ID 89366.

The Prognostic Value of MRI in Moderate and Severe Traumatic Brain Injury: A Systematic Review and Meta-Analysis

OBJECTIVES

Traumatic brain injury is a major cause of death and disability, yet many predictors of outcome are not precise enough to guide initial clinical decision-making. Although increasingly used in the early phase following traumatic brain injury, the prognostic utility of MRI remains uncertain. We thus undertook a systematic review and meta-analysis of studies evaluating the predictive value of acute MRI lesion patterns for discriminating clinical outcome in traumatic brain injury.

DATA SOURCES

MEDLINE, EMBASE, BIOSIS, and CENTRAL from inception to November 2015.

STUDY SELECTION

Studies of adults who had MRI in the acute phase following moderate or severe traumatic brain injury. Our primary outcomes were all-cause mortality and the Glasgow Outcome Scale.

DATA EXTRACTION

Two authors independently performed study selection and data extraction. We calculated pooled effect estimates with a random effects model, evaluated the risk of bias using a modified version of Quality in Prognostic Studies and determined the strength of evidence with the Grading of Recommendations, Assessment, Development, and Evaluation.

DATA SYNTHESIS

We included 58 eligible studies, of which 27 (n = 1,652) contributed data to meta-analysis. Brainstem lesions were associated with all-cause mortality (risk ratio, 1.78; 95% CI, 1.01-3.15; I = 43%) and unfavorable Glasgow Outcome Scale (risk ratio, 2.49; 95% CI, 1.72-3.58; I = 81%) at greater than or equal to 6 months. Diffuse axonal injury patterns were associated with an increased risk of unfavorable Glasgow Outcome Scale (risk ratio, 2.46; 95% CI, 1.06-5.69; I = 74%). MRI scores based on lesion depth demonstrated increasing risk of unfavorable neurologic outcome as more caudal structures were affected. Most studies were at high risk of methodological bias.

CONCLUSIONS

MRI following traumatic brain injury yields important prognostic information, with several lesion patterns significantly associated with long-term survival and neurologic outcome. Given the high risk of bias in the current body of literature, large well-controlled studies are necessary to better quantify the prognostic role of early MRI in moderate and severe traumatic brain injury.

A systematic review of neonatal treatment intensity scores and their potential application in low-resource setting hospitals for predicting mortality, morbidity and estimating resource use

BACKGROUND

Treatment intensity scores can predict mortality and estimate resource use. They may therefore be of interest for essential neonatal care in low resource settings where neonatal mortality remains high. We sought to systematically review neonatal treatment intensity scores to (1) assess the level of evidence on predictive performance in predicting clinical outcomes and estimating resource utilisation and (2) assess the applicability of the identified models to decision making for neonatal care in low resource settings.

METHODS

We conducted a systematic search of PubMed, EMBASE (OVID), CINAHL, Global Health Library (Global index, WHO) and Google Scholar to identify studies published up until 21 December 2016. Included were all articles that used treatments as predictors in neonatal models. Individual studies were appraised using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). In addition, Grading of Recommendations Assessment, Development, and Evaluation (GRADE) was used as a guiding framework to assess certainty in the evidence for predicting outcomes across studies.

RESULTS

Three thousand two hundred forty-nine articles were screened, of which ten articles were included in the review. All of the studies were conducted in neonatal intensive care units with sample sizes ranging from 22 to 9978, with a median of 163. Two articles reported model development, while eight reported external application of existing models to new populations. Meta-analysis was not possible due heterogeneity in the conduct and reporting of the identified studies. Discrimination as assessed by area under receiver operating characteristic curve was reported for in-hospital mortality, median 0.84 (range 0.75-0.96, three studies), early adverse outcome and late adverse outcome (0.78 and 0.59, respectively, one study).

CONCLUSION

Existing neonatal treatment intensity models show promise in predicting mortality and morbidity. There is however low certainty in the evidence on their performance in essential neonatal care in low resource settings as all studies had methodological limitations and were conducted in intensive care. The approach may however be developed further for low resource settings like Kenya because treatment data may be easier to obtain compared to measures of physiological status.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016034205.

Predictors of multidisciplinary rehabilitation outcomes in patients with chronic musculoskeletal pain: protocol for a systematic review and meta-analysis

BACKGROUND

Chronic musculoskeletal pain is a major public health problem. Early prediction for optimal treatment results has received growing attention, but there is presently a lack of evidence regarding what information such proactive management should be based on. This study protocol, therefore, presents our planned systematic review and meta-analysis on important predictive factors for health and work-related outcomes following multidisciplinary rehabilitation (MDR) in patients with chronic musculoskeletal pain.

METHODS

We aim to perform a synthesis of the available evidence together with a meta-analysis of published peer-reviewed original research that includes predictive factors preceding MDR. Included are prospective studies of adults with benign, chronic (> 3 months) musculoskeletal pain diagnoses who have taken part in MDR. In the studies, associations between personal and rehabilitation-based factors and the outcomes of interest are reported. Outcome domains are pain, physical functioning including health-related quality of life, and work ability with follow-ups of 6 months or more. We will use a broad, explorative approach to any presented predictive factors (demographic, symptoms-related, physical, psychosocial, work-related, and MDR-related) and these will be analyzed through (a) narrative synthesis for each outcome domain and (b) if sufficient studies are available, a quantitative synthesis in which variance-weighted pooled proportions will be computed using a random effects model for each outcome domain. The strength of the evidence will be evaluated using the Grading of Recommendations, Assessment, Development and Evaluation.

DISCUSSION

The strength of this systematic review is that it aims for a meta-analysis of prospective cohort or randomized controlled studies by performing an extensive search of multiple databases, using an explorative study approach to predictive factors, rather than building on single predictor impact on the outcome or on predefined hypotheses. In this way, an overview of factors central to MDR outcome can be made and will help strengthen the evidence base and inform a wide readership including health care practitioners and policymakers.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016025339.

Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations

OBJECTIVE

Critically appraise prediction models for hospital-acquired acute kidney injury (HA-AKI) in general populations.

DESIGN

Systematic review.

DATA SOURCES

Medline, Embase and Web of Science until November 2016.

ELIGIBILITY

Studies describing development of a multivariable model for predicting HA-AKI in non-specialised adult hospital populations. Published guidance followed for data extraction reporting and appraisal.

RESULTS

14 046 references were screened. Of 53 HA-AKI prediction models, 11 met inclusion criteria (general medicine and/or surgery populations, 474 478 patient episodes) and five externally validated. The most common predictors were age (n=9 models), diabetes (5), admission serum creatinine (SCr) (5), chronic kidney disease (CKD) (4), drugs (diuretics (4) and/or ACE inhibitors/angiotensin-receptor blockers (3)), bicarbonate and heart failure (4 models each). Heterogeneity was identified for outcome definition. Deficiencies in reporting included handling of predictors, missing data and sample size. Admission SCr was frequently taken to represent baseline renal function. Most models were considered at high risk of bias. Area under the receiver operating characteristic curves to predict HA-AKI ranged 0.71-0.80 in derivation (reported in 8/11 studies), 0.66-0.80 for internal validation studies (n=7) and 0.65-0.71 in five external validations. For calibration, the Hosmer-Lemeshow test or a calibration plot was provided in 4/11 derivations, 3/11 internal and 3/5 external validations. A minority of the models allow easy bedside calculation and potential electronic automation. No impact analysis studies were found.

CONCLUSIONS

AKI prediction models may help address shortcomings in risk assessment; however, in general hospital populations, few have external validation. Similar predictors reflect an elderly demographic with chronic comorbidities. Reporting deficiencies mirrors prediction research more broadly, with handling of SCr (baseline function and use as a predictor) a concern. Future research should focus on validation, exploration of electronic linkage and impact analysis. The latter could combine a prediction model with AKI alerting to address prevention and early recognition of evolving AKI.

Models to predict relapse in psychosis: A systematic review

Background

There is little evidence on the accuracy of psychosis relapse prediction models. Our objective was to undertake a systematic review of relapse prediction models in psychosis.

Method

We conducted a literature search including studies that developed and/or validated psychosis relapse prediction models, with or without external model validation. Models had to target people with psychosis and predict relapse. The key databases searched were; Embase, Medline, Medline In-Process Citations & Daily Update, PsychINFO, BIOSIS Citation Index, CINAHL, and Science Citation Index, from inception to September 2016. Prediction modelling studies were assessed for risk of bias and applicability using the PROBAST tool.

Results

There were two eligible studies, which included 33,088 participants. One developed a model using prodromal symptoms and illness-related variables, which explained 14% of relapse variance but was at high risk of bias. The second developed a model using administrative data which was moderately discriminative (C = 0.631) and associated with relapse (OR 1.11 95% CI 1.10, 1.12) and achieved moderately discriminative capacity when validated (C = 0.630). The risk of bias was low.

Conclusions

Due to a lack of high quality evidence it is not possible to make any specific recommendations about the predictors that should be included in a prognostic model for relapse. For instance, it is unclear whether prodromal symptoms are useful for predicting relapse. The use of routine data to develop prediction models may be a more promising approach, although we could not empirically compare the two included studies.

The prognostic role of tumor infiltrating T-lymphocytes in squamous cell carcinoma of the head and neck: A systematic review and meta-analysis

Background - The presence of tumor-infiltrating lymphocytes (TILs) in the tumor microenvironment is associated with an improved prognosis and a better response to therapy in different types of cancer. In this systematic review and meta-analysis, we investigated the prognostic value of T cells in head and neck squamous cell carcinoma (HNSCC). Methods - In a systematic review, Pubmed and Embase were searched for publications that investigated the prognostic value of T cells in HNSCC. A meta-analysis was performed including all studies assessing the association between CD3+, CD4+, CD8+, and FoxP3+ TILs and overall survival (OS), disease-free survival (DFS), or locoregional control (LRC). Results - A pooled analysis indicated a favorable, prognostic role for CD3+ TILs (HR 0.64 (95%CI 0.47-0.85) for OS, HR 0.63 (95%CI 0.49-0.82) for DFS) and CD8+ TILs (HR 0.67 (95%CI 0.58-0.79) for OS, HR 0.50 (95%CI 0.37-0.68) for DFS, and HR 0.82 (95%CI 0.70-0.96) for LRC) in the clinical outcome of HNSCC. FoxP3+ TILs were also associated with better OS (HR 0.80 (95%CI 0.70-0.92)). Conclusion - This systematic review and meta-analysis confirmed the favorable, prognostic role of CD3+ and CD8+ T cell infiltration in HNSCC patients and found an association between FoxP3+ TILs and improved overall survival. Future studies using homogeneous patient cohorts with regard to tumor subsite, stage and treatment are necessary to provide more insight in the predictive value of TILs in HNSCC.

External validation of a claims-based and clinical approach for predicting post-pulmonary embolism outcomes among United States veterans

The In-hospital Mortality for PulmonAry embolism using Claims daTa (IMPACT) rule can accurately identify pulmonary embolism (PE) patients at low risk of early complications using claims data. We sought to externally validate the IMPACT and simplified Pulmonary Embolism Severity Index (sPESI) tools for predicting all-cause mortality and readmission. We used Veteran Health Administration data (10/1/2010-9/30/2015) to identify adults with ≥1 inpatient diagnosis code for acute PE, ≥12 months continuous medical and pharmacy benefits prior to the index PE, ≥90 days of post-event follow-up (unless death occurred) and ≥1 claim for an anticoagulant during the index PE stay. Prognostic accuracies of IMPACT and sPESI for 30- and 90-day all-cause mortality and 90-day readmission were estimated. Of 6,746 PE patients, 7.5 and 12.6% died at 30 and 90 days. Within 90 days, 20.1% were readmitted for any reason. Hospitalization for recurrent VTE and major bleeding occurred in 5.6 and 1.7% of patients. IMPACT classified 15.2% as low risk, while 28.4% were low risk per sPESI. Both tools displayed sensitivity >90% and negative predictive values (NPVs) >97% for 30-day mortality, but low specificity (range 16.2-30.0) and positive predictive values (PPVs) (range 8.7-9.5); with similar results observed for 90-day mortality. IMPACT's sensitivity for all-cause readmission was numerically higher than sPESI (88.2 vs. 79.0%), but both had comparable NPVs (85.1 vs. 84.2%). Similar trends were observed for VTE or major bleeding readmissions. IMPACT classified patients for post-PE outcomes with similar accuracy as sPESI. IMPACT appears useful for identifying PE patients at low risk for early mortality or readmission in claims-based studies.

Little Evidence for Usefulness of Biomarkers for Predicting Successful Dose Reduction or Discontinuation of a Biologic Agent in Rheumatoid Arthritis: A Systematic Review

OBJECTIVE

To systematically review studies addressing prediction of successful dose reduction or discontinuation of a biologic agent in rheumatoid arthritis (RA).

METHODS

PubMed, Embase, and Cochrane Library databases were searched for studies that examined the predictive value of biomarkers for successful dose reduction or discontinuation of a biologic agent in RA. Two reviewers independently selected studies, and extracted data and assessed the risk of bias. A biomarker was classified as a "potential predictor" if the univariate association was either strong (odds ratio or hazard ratio >2.0 or <0.5) or statistically significant. For biomarkers that were studied multiple times, qualitative best-evidence synthesis was performed separately for the prediction of successful dose reduction and discontinuation. Biomarkers that were defined in ≥75% of the studies as potential predictors were regarded as "predictor" for the purposes of our study.

RESULTS

Of 3,029 nonduplicate articles initially searched, 16 articles regarding 15 cohorts were included in the present study. Overall, 17 biomarkers were studied multiple times for the prediction of successful dose reduction, and 33 for the prediction of successful discontinuation of a biologic agent. Three predictors were identified: higher adalimumab trough level for successful dose reduction and lower Sharp/van der Heijde erosion score and shorter symptom duration at the start of a biologic agent for successful discontinuation.

CONCLUSION

The predictive value of a wide variety of biomarkers for successful dose reduction or discontinuation of biologic treatment in RA has been investigated. We identified only 3 biomarkers as predictors, in just 2 studies. The strength of the evidence is limited by the low quality of the included studies and the likelihood of reporting bias and multiple testing.

Predictive performance of the CHA2DS2-VASc rule in atrial fibrillation: a systematic review and meta-analysis

Essentials The widely recommended CHA2DS2-VASc shows conflicting results in contemporary validation studies. We performed a systematic review and meta-analysis of 19 studies validating CHA2DS2-VASc. There was high heterogeneity in stroke risks for different CHA2DS2-VASc scores. This was not explained by differences between setting of care, or by performing meta-regression.

SUMMARY

Background The CHA2DS2-VASc decision rule is widely recommended for estimating stroke risk in patients with atrial fibrillation (AF), although validation studies show ambiguous and conflicting results. Objectives To: (i) review existing studies validating CHA2DS2-VASc in AF patients who are not (yet) anticoagulated; (ii) meta-analyze estimates of stroke risk per score; and (iii) explore sources of heterogeneity across the validation studies. Methods We performed a systematic literature review and random effects meta-analysis of studies externally validating CHA2DS2-VASc in AF patients not receiving anticoagulants. To explore between-study heterogeneity in stroke risk, we stratified studies to the clinical setting in which patient enrollment started, and performed meta-regression. Results In total, 19 studies were evaluated, with over two million person-years of follow-up. In studies recruiting AF patients in hospitals, stroke risks for scores of 0, 1 and 2 were 0.4% (approximate 95% prediction interval [PI] 0.2-3.2%), 1.2% (95% PI 0.1-3.8%), and 2.2% (95% PI 0.03-7.8%), respectively. These were consistently higher than those in studies recruiting patients from the open general population, with risks of 0.2% (95% PI 0.0-0.9%), 0.7% (95% PI 0.3-1.2%) and 1.5% (95% PI 0.4-3.3%) for scores of 0, 1, and 2, respectively. Heterogeneity, as reflected by the wide PIs, could not be fully explained by meta-regression. Conclusions Studies validating CHA2DS2-VASc show high heterogeneity in predicted stroke risks for different scores.

Protocol: a systematic review of studies developing and/or evaluating search strategies to identify prognosis studies

BACKGROUND

Prognosis research is on the rise, its importance recognised because chronic health conditions and diseases are increasingly common and costly. Prognosis systematic reviews are needed to collate and synthesise these research findings, especially to help inform effective clinical decision-making and healthcare policy. A detailed, comprehensive search strategy is central to any systematic review. However, within prognosis research, this is challenging due to poor reporting and inconsistent use of available indexing terms in electronic databases. Whilst many published search filters exist for finding clinical trials, this is not the case for prognosis studies. This systematic review aims to identify and compare existing methodological filters developed and evaluated to identify prognosis studies of any of the three main types: overall prognosis, prognostic factors, and prognostic [risk prediction] models.

METHODS

Primary studies reporting the development and/or evaluation of methodological search filters to retrieve any type of prognosis study will be included in this systematic review. Multiple electronic bibliographic databases will be searched, grey literature will be sought from relevant organisations and websites, experts will be contacted, and citation tracking of key papers and reference list checking of all included papers will be undertaken. Titles will be screened by one person, and abstracts and full articles will be reviewed for inclusion independently by two reviewers. Data extraction and quality assessment will also be undertaken independently by two reviewers with disagreements resolved by discussion or by a third reviewer if necessary. Filters' characteristics and performance metrics reported in the included studies will be extracted and tabulated. To enable comparisons, filters will be grouped according to database, platform, type of prognosis study, and type of filter for which it was intended.

DISCUSSION

This systematic review will identify all existing validated prognosis search filters and synthesise evidence about their applicability and performance. These findings will identify if current filters provide a proficient means of searching electronic bibliographic databases or if further prognosis filters are needed and can feasibly be developed for systematic searches of prognosis studies.

Risk Factor Models for Neurodevelopmental Outcomes in Children Born Very Preterm or With Very Low Birth Weight: A Systematic Review of Methodology and Reporting

The prediction of long-term outcomes in surviving infants born very preterm (VPT) or with very low birth weight (VLBW) is necessary to guide clinical management, provide information to parents, and help target and evaluate interventions. There is a large body of literature describing risk factor models for neurodevelopmental outcomes in VPT/VLBW children, yet few, if any, have been developed for use in routine clinical practice or adopted for use in research studies or policy evaluation. We sought to systematically review the methods and reporting of studies that have developed a multivariable risk factor model for neurodevelopment in surviving VPT/VLBW children. We searched the MEDLINE, Embase, and PsycINFO databases from January 1, 1990, to June 1, 2014, and identified 78 studies reporting 222 risk factor models. Most studies presented risk factor analyses that were not intended to be used for prediction, confirming that there is a dearth of specifically designed prognostic modeling studies for long-term outcomes in surviving VPT/VLBW children. We highlight the strengths and weaknesses of the research methodology and reporting to date, and provide recommendations for the design and analysis of future studies seeking to analyze risk prediction or develop prognostic models for VPT/VLBW children.

Risk prediction models for delirium in the intensive care unit after cardiac surgery: a systematic review and independent external validation

Numerous risk prediction models are available for predicting delirium after cardiac surgery, but few have been directly compared with one another or been validated in an independent data set. We conducted a systematic review to identify validated risk prediction models of delirium (using the Confusion Assessment Method-Intensive Care Unit tool) after cardiac surgery and assessed the transportability of the risk prediction models on a prospective cohort of 600 consecutive patients undergoing cardiac surgery at a university hospital in Hong Kong from July 2013 to July 2015. The discrimination (c-statistic), calibration (GiViTI calibration belt), and clinical usefulness (decision curve analysis) of the risk prediction models were examined in a stepwise manner. Three published high-quality intensive care unit delirium risk prediction models (n=5939) were identified: Katznelson, the original PRE-DELIRIC, and the international recalibrated PRE-DELIRIC model. Delirium occurred in 83 patients (13.8%, 95% CI: 11.2-16.9%). After updating the intercept and regression coefficients in the Katznelson model, there was fair discrimination (0.62, 95% CI: 0.58-0.66) and good calibration. As the original PRE-DELIRIC model was already validated externally and recalibrated in six countries, we performed a logistic calibration on the recalibrated model and found acceptable discrimination (0.75, 95% CI: 0.72-0.79) and good calibration. Decision curve analysis demonstrated that the recalibrated PRE-DELIRIC risk model was marginally more clinically useful than the Katznelson model. Current models predict delirium risk in the intensive care unit after cardiac surgery with only fair to moderate accuracy and are insufficient for routine clinical use.

Clinical prediction rules for prognosis and treatment prescription in neck pain: A systematic review

Clinical prediction rules (CPRs) developed to identify sub-groups of people with neck pain for different prognoses (i.e. prognostic) or response to treatments (i.e. prescriptive) have been recommended as a research priority to improve health outcomes for these conditions. A systematic review was undertaken to identify prognostic and prescriptive CPRs relevant to the conservative management of adults with neck pain and to appraise stage of development, quality and readiness for clinical application. Six databases were systematically searched from inception until 4th July 2016. Two independent reviewers assessed eligibility, risk of bias (PEDro and QUIPS), methodological quality and stage of development. 9840 records were retrieved and screened for eligibility. Thirty-two studies reporting on 26 CPRs were included in this review. Methodological quality of included studies varied considerably. Most prognostic CPR development studies employed appropriate designs. However, many prescriptive CPR studies (n = 12/13) used single group designs and/or analysed controlled trials using methods that were inadequate for identifying treatment effect moderators. Most prognostic (n = 11/15) and all prescriptive (n = 11) CPRs have not progressed beyond the derivation stage of development. Four prognostic CPRs relating to acute whiplash (n = 3) or non-traumatic neck pain (n = 1) have undergone preliminary validation. No CPRs have undergone impact analysis. Most prognostic and prescriptive CPRs for neck pain are at the initial stage of development and therefore routine clinical use is not yet supported. Further validation and impact analyses of all CPRs are required before confident conclusions can be made regarding clinical utility.

Prediction models for the risk of gestational diabetes: a systematic review

BACKGROUND

Numerous prediction models for gestational diabetes mellitus (GDM) have been developed, but their methodological quality is unknown. The objective is to systematically review all studies describing first-trimester prediction models for GDM and to assess their methodological quality.

METHODS

MEDLINE and EMBASE were searched until December 2014. Key words for GDM, first trimester of pregnancy, and prediction modeling studies were combined. Prediction models for GDM performed up to 14 weeks of gestation that only include routinely measured predictors were eligible.Data was extracted by the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). Data on risk predictors and performance measures were also extracted. Each study was scored for risk of bias.

RESULTS

Our search yielded 7761 articles, of which 17 were eligible for review (14 development studies and 3 external validation studies). The definition and prevalence of GDM varied widely across studies. Maternal age and body mass index were the most common predictors. Discrimination was acceptable for all studies. Calibration was reported for four studies. Risk of bias for participant selection, predictor assessment, and outcome assessment was low in general. Moderate to high risk of bias was seen for the number of events, attrition, and analysis.

CONCLUSIONS

Most studies showed moderate to low methodological quality, and few prediction models for GDM have been externally validated. External validation is recommended to enhance generalizability and assess their true value in clinical practice.

Development and testing of a medline search filter for identifying patient and public involvement in health research

BACKGROUND

Research involving the public as partners often proves difficult to locate due to the variations in terms used to describe public involvement, and inability of medical databases to index this concept effectively.

OBJECTIVE

To design a search filter to identify literature where patient and public involvement (PPI) was used in health research.

METHODS

A reference standard of 172 PPI papers was formed. The references were divided into a development set and a test set. Search terms were identified from common words, phrases and synonyms in the development set. These terms were combined as a search strategy for medline via OvidSP, which was then tested for sensitivity against the test set. The resultant search filter was then assessed for sensitivity, specificity and precision using a previously published systematic review.

RESULTS

The search filter was found to be highly sensitive 98.5% in initial testing. When tested against results generated by a 'real-life' systematic review, the filter had a specificity of 81%. However, sensitivity dropped to 58%. Adjustments to the population group of terms increased the sensitivity to 73%.

CONCLUSION

The PPI filter designed for medline via OvidSP could aid information specialists and researchers trying to find literature specific to PPI.

Published diagnostic models safely excluded colorectal cancer in an independent primary care validation study

OBJECTIVE

To validate published diagnostic models for their ability to safely reduce unnecessary endoscopy referrals in primary care patients suspected of significant colorectal disease.

STUDY DESIGN AND SETTING

Following a systematic literature search, we independently validated the identified diagnostic models in a cross-sectional study of 810 Dutch primary care patients with persistent lower abdominal complaints referred for endoscopy. We estimated diagnostic accuracy measures for colorectal cancer (N = 37) and significant colorectal disease (N = 141; including colorectal cancer, inflammatory bowel disease, diverticulitis, or >1-cm adenomas).

RESULTS

We evaluated 18 models-12 specific for colorectal cancer-, of which most were able to safely rule out colorectal cancer: the best model (National Institute for Health and Care Excellence-1) prevented 59% of referrals (95% confidence interval [CI]: 56-63%), with 96% sensitivity (95% CI: 83-100%), 100% negative predictive value (NPV; 95% CI: 99-100%), and an area under the receiver operating characteristics curve (AUC) of 0.86 (95% CI: 0.80-0.92). The models performed less for significant colorectal disease: the best model (Brazer) prevented 23% of referrals (95% CI: 20-26%), with 95% sensitivity (95% CI: 90-98%), 96% NPV (95% CI: 92-98%), and an AUC of 0.73 (95% CI: 0.69-0.78).

CONCLUSION

Most models safely excluded colorectal cancer in many primary care patients with lower gastrointestinal complaints referred for endoscopy. Models performed less well for significant colorectal disease.

Risk scores for outcome in bacterial meningitis: Systematic review and external validation study

OBJECTIVES

To perform an external validation study of risk scores, identified through a systematic review, predicting outcome in community-acquired bacterial meningitis.

METHODS

MEDLINE and EMBASE were searched for articles published between January 1960 and August 2014. Performance was evaluated in 2108 episodes of adult community-acquired bacterial meningitis from two nationwide prospective cohort studies by the area under the receiver operating characteristic curve (AUC), the calibration curve, calibration slope or Hosmer-Lemeshow test, and the distribution of calculated risks.

FINDINGS

Nine risk scores were identified predicting death, neurological deficit or death, or unfavorable outcome at discharge in bacterial meningitis, pneumococcal meningitis and invasive meningococcal disease. Most studies had shortcomings in design, analyses, and reporting. Evaluation showed AUCs of 0.59 (0.57-0.61) and 0.74 (0.71-0.76) in bacterial meningitis, 0.67 (0.64-0.70) in pneumococcal meningitis, and 0.81 (0.73-0.90), 0.82 (0.74-0.91), 0.84 (0.75-0.93), 0.84 (0.76-0.93), 0.85 (0.75-0.95), and 0.90 (0.83-0.98) in meningococcal meningitis. Calibration curves showed adequate agreement between predicted and observed outcomes for four scores, but statistical tests indicated poor calibration of all risk scores.

INTERPRETATION

One score could be recommended for the interpretation and design of bacterial meningitis studies. None of the existing scores performed well enough to recommend routine use in individual patient management.

Prognostic Models in Adults Undergoing Physical Therapy for Rotator Cuff Disorders: Systematic Review

BACKGROUND

Rotator cuff-related disorders represent the largest subgroup of shoulder complaints. Despite the availability of various conservative and surgical treatment options, the precise indications for these options remain unclear.

PURPOSE

The purpose of this systematic review was to synthesize the available research on prognostic models for predicting outcomes in adults undergoing physical therapy for painful rotator cuff disorders.

DATA SOURCES

The MEDLINE, EMBASE, CINAHL, Cochrane CENTRAL, and PEDro databases and the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) up to October 2015 were searched.

STUDY SELECTION

The review included primary studies exploring prognostic models in adults undergoing physical therapy, with or without other conservative measures, for painful rotator cuff disorders. Primary outcomes were pain, disability, and adverse events. Inclusion was limited to prospective investigations of prognostic factors elicited at the baseline assessment. Study selection was independently performed by 2 reviewers.

DATA EXTRACTION

A pilot-tested form was used to extract data on key aspects of study design, characteristics, analyses, and results. Risk of bias and applicability were independently assessed by 2 reviewers using the Prediction Study Risk of Bias Assessment tool (PROBAST).

DATA SYNTHESIS

Five studies were included in the review. These studies were extremely heterogeneous in many aspects of design, conduct, and analysis. The findings were analyzed narratively.

LIMITATIONS

All included studies were rated as at high risk of bias, and none of the resulting prognostic models was found to be usable in clinical practice.

CONCLUSIONS

There are no prognostic models ready to inform clinical practice in the context of the review question, highlighting the need for further research on prognostic models for predicting outcomes in adults who undergo physical therapy for painful rotator cuff disorders. The design and conduct of future studies should be receptive to developing methods.

Echocardiographic findings associated with mortality ortransplant in patients with pulmonary arterial hypertension:A systematic review and meta-analysis

Background

Identification of patients at risk of deterioration is essential to guide clinical management in pulmonary arterial hypertension (PAH). This study aims to provide a comprehensive overview of well-investigated echocardiographic findings that are associated with clinical deterioration in PAH.

Methods

MEDLINE and EMBASE databases were systematically searched for longitudinal studies published by April 2015 that reported associations between echocardiographic findings and mortality, transplant or clinical worsening. Meta-analysis using random effect models was performed for echocardiographic findings investigated by four or more studies. In case of statistical heterogeneity a sensitivity analysis was conducted.

Results

Thirty-seven papers investigating 51 echocardiographic findings were included. Meta-analysis of univariable hazard ratios (HRs) and sensitivity analysis showed that presence of pericardial effusion (pooled HR 1.70; 95 % CI 1.44–1.99), right atrial area (pooled HR 1.71; 95 % CI 1.38–2.13) and tricuspid annular plane systolic excursion (TAPSE; pooled HR 1.72; 95 % CI 1.34–2.20) were the most well-investigated and robust predictors of mortality or transplant.

Conclusions

This meta-analysis substantiates the clinical yield of specific echocardiographic findings in the prognostication of PAH patients in day-to-day practice. In particular, pericardial effusion, right atrial area and TAPSE are of prognostic value.

Electronic supplementary material

The online version of this article (doi: 10.1007/s12471-016-0845-3) contains supplementary material, which is available to authorized users.

Prognostic factors for cerebral palsy and motor impairment in children born very preterm or very low birthweight: a systematic review

AIM

There is a large literature reporting risk factor analyses for poor neurodevelopment in children born very preterm (VPT: ≤32wks) or very low birthweight (VLBW: ≤1250g), which to date has not been formally summarized. The aim of this paper was to identify prognostic factors for cerebral palsy (CP) and motor impairment in children born VPT/VLBW.

METHOD

A systematic review was conducted using Medline, Embase, and Pyscinfo databases to identify studies published between 1 January 1990 and 1 June 2014 reporting multivariable prediction models for poor neurodevelopment in VPT/VLBW children (registration number CRD42014006943). Twenty-eight studies for motor outcomes were identified.

RESULTS

There was strong evidence that intraventricular haemorrhage and periventricular leukomalacia, and some evidence that the use of postnatal steroids and non-use of antenatal steroids, were prognostic factors for CP. Male sex and gestational age were of limited use as prognostic factors for CP in cohorts restricted to ≤32 weeks gestation; however, in children older than 5 years with no major disability, there was evidence that male sex was a predictive factor for motor impairment.

INTERPRETATION

This review has identified factors which may be of prognostic value for CP and motor impairment in VPT/VLBW children and will help to form the basis of future prognostic research.

Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review

OBJECTIVE

Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction, presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR based risk prediction modeling through a systematic review of clinical prediction studies using EHR data.

METHODS

We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation.

RESULTS

We identified 107 articles from 15 different countries. Studies were generally very large (median sample size = 26 100) and utilized a diverse array of predictors. Most used validation techniques (n = 94 of 107) and reported model coefficients for reproducibility (n = 83). However, studies did not fully leverage the breadth of EHR data, as they uncommonly used longitudinal information (n = 37) and employed relatively few predictor variables (median = 27 variables). Less than half of the studies were multicenter (n = 50) and only 26 performed validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71).

CONCLUSIONS

EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.

Performance of search strategies to retrieve systematic reviews of diagnostic test accuracy from the Cochrane Library

AIM

To compare the performance of search strategies to retrieve systematic reviews of diagnostic test accuracy from The Cochrane Library.

METHODS

Databases of Cochrane Database of Systematic Reviews and Cochrane Database of Abstracts of Reviews of Effects in the Cochrane Library were searched for systematic reviews of diagnostic test accuracy published between 2008 and 2012 through nine search strategies. Each strategy consists of one group or combination of groups of searching filters about diagnostic test accuracy. Four groups of diagnostic filters were used. The strategy combing all the filters was used as the reference to determine the sensitivity, precision, and the sensitivity × precision product for another eight strategies.

RESULTS

The reference strategy retrieved 8029 records, of which 832 were eligible. The strategy only composed of MeSH terms about "accuracy measures" achieved the highest values in both precision (69.71%) and product (52.45%) with a moderate sensitivity (75.24%). The combination of MeSH terms and free text words about "accuracy measures" contributed little to increasing the sensitivity. Strategies composed of filters about "diagnosis" had similar sensitivity but lower precision and product to those composed of filters about "accuracy measures." MeSH term "exp'diagnosis'" achieved the lowest precision (9.78%) and product (7.91%), while its hyponym retrieved only half the number of records at the expense of missing 53 target articles. The precision was negatively correlated with sensitivities among the nine strategies.

CONCLUSIONS

Compared to the filters about "diagnosis," the filters about "accuracy measures" achieved similar sensitivities but higher precision. When combining both terms, sensitivity of the strategy was enhanced obviously. The combination of MeSH terms and free text words about the same concept seemed to be meaningless for enhancing sensitivity.

Costs, effects and implementation of routine data emergency admission risk prediction models in primary care for patients with, or at risk of, chronic conditions: a systematic review protocol

INTRODUCTION

Emergency admission risk prediction models are increasingly used to identify patients, typically with one or more chronic conditions, for proactive management in primary care to avoid admissions, save costs and improve patient experience.

AIM

To identify and review the published evidence on the costs, effects and implementation of emergency admission risk prediction models in primary care for patients with, or at risk of, chronic conditions.

METHODS

We shall search for studies of healthcare interventions using routine data-generated emergency admission risk models. We shall report: the effects on emergency admissions and health costs; clinician and patient views; and implementation findings. We shall search ASSIA, CINAHL, the Cochrane Library, HMIC, ISI Web of Science, MEDLINE and Scopus from 2005, review references in and citations of included articles, search key journals and contact experts. Study selection, data extraction and quality assessment will be performed by two independent reviewers.

ETHICS AND DISSEMINATION

No ethical permissions are required for this study using published data. Findings will be disseminated widely, including publication in a peer-reviewed journal and through conferences in primary and emergency care and chronic conditions. We judge our results will help a wide audience including primary care practitioners and commissioners, and policymakers.

TRIAL REGISTRATION NUMBER

CRD42015016874; Pre-results.

Cardiac magnetic resonance findings predicting mortality in patients with pulmonary arterial hypertension: a systematic review and meta-analysis

Objectives

To provide a comprehensive overview of all reported cardiac magnetic resonance (CMR) findings that predict clinical deterioration in pulmonary arterial hypertension (PAH).

Methods

MEDLINE and EMBASE electronic databases were systematically searched for longitudinal studies published by April 2015 that reported associations between CMR findings and adverse clinical outcome in PAH. Studies were appraised using previously developed criteria for prognostic studies. Meta-analysis using random effect models was performed for CMR findings investigated by three or more studies.

Results

Eight papers (539 patients) investigating 21 different CMR findings were included. Meta-analysis showed that right ventricular (RV) ejection fraction was the strongest predictor of mortality in PAH (pooled HR 1.23 [95 % CI 1.07–1.41], p = 0.003) per 5 % decrease. In addition, RV end-diastolic volume index (pooled HR 1.06 [95 % CI 1.00–1.12], p = 0.049), RV end-systolic volume index (pooled HR 1.05 [95 % CI 1.01–1.09], p = 0.013) and left ventricular end-diastolic volume index (pooled HR 1.16 [95 % CI 1.00–1.34], p = 0.045) were of prognostic importance. RV and LV mass did not provide prognostic information (p = 0.852 and p = 0.983, respectively).

Conclusion

This meta-analysis substantiates the clinical yield of specific CMR findings in the prognostication of PAH patients. Decreased RV ejection is the strongest and most well established predictor of mortality.

Key Points

• Cardiac magnetic resonance imaging is useful for prognostication in pulmonary arterial hypertension.

• Right ventricular ejection fraction is the strongest predictor of mortality.

• Serial CMR evaluation seems to be of additional prognostic importance.

• Accurate prognostication can aid in adequate and timely intensification of PAH-specific therapy.

Electronic supplementary material

The online version of this article (doi:10.1007/s00330-016-4217-6) contains supplementary material, which is available to authorized users.

Prognostic Factors for Behavioral Problems and Psychiatric Disorders in Children Born Very Preterm or Very Low Birth Weight: A Systematic Review

OBJECTIVE

Risk factors associated with adverse behavioral outcomes in very preterm (VPT) or very low birth weight (VLBW) infants are poorly understood. The aim of this article is to identify prognostic factors for behavioral problems and psychiatric disorders in children born ≤32 weeks gestational age or with birth weight ≤1250 g.

METHOD

A systematic review was conducted using MEDLINE, Embase, and Pyscinfo databases to identify studies published between January 1, 1990 and June 1, 2014 reporting multivariable prediction models for behavioral problems or psychiatric disorders in VPT/VLBW children. Fifteen studies were identified and 2 independent reviewers extracted key information on study design, outcome definition, risk factor selection, model development, reporting, and conducted a risk of bias assessment.

RESULTS

The 15 studies included reported risk factor analyses for the following domains: general behavioral problems (n = 8), any psychiatric disorder (n = 2), autism spectrum symptoms/disorders (n = 5), and attention deficit/hyperactivity disorder (n = 1). Findings were inconclusive because of the following: small number of studies in each domain, heterogeneity in outcome measures, lack of overlap in the risk factors examined, and differences in strategies for dealing with children with neurological impairments.

CONCLUSION

There is a lack of evidence concerning risk factors for behavior problems and psychiatric disorders among VPT/VLBW survivors. This review has identified the need for further research examining the etiology of disorders of psychological development in the VPT/VLBW population to refine risk prediction and identify targets for intervention. Large well-conducted studies that use standard diagnostic evaluations to assess psychiatric disorders throughout childhood and adolescence are required.

Predicting surgical outcome in patients with International Federation of Gynecology and Obstetrics stage III or IV ovarian cancer using computed tomography: a systematic review of prediction models

OBJECTIVE

Maximal cytoreduction to no residual disease is an important predictor of prognosis in patients with advanced-stage epithelial ovarian cancer. Preoperative prediction of outcome of surgery should guide treatment decisions, for example, primary debulking or neoadjuvant chemotherapy followed by interval debulking surgery. The objective of this study was to systematically review studies evaluating computed tomography imaging based models predicting the amount of residual tumor after cytoreductive surgery for advanced-stage epithelial ovarian cancer.

METHODS

We systematically searched the literature for studies investigating multivariable models that predicted the amount of residual disease after cytoreductive surgery in advanced-stage epithelial ovarian cancer using computed tomography imaging. Detected studies were scored for quality and classified as model derivation or validation studies. We summarized their performance in terms of discrimination when possible.

RESULTS

We identified 11 studies that described 13 models. The 4 models that were externally validated all had a poor discriminative capacity (sensitivity, 15%-79%; specificity, 32%-64%). The only internal validated model had an area under the receiver operating characteristic curve of 0.67. Peritoneal thickening, mesenterial and diaphragm disease, and ascites were most often used as predictors in the final models. We did not find studies that assessed the impact of prediction model on outcomes.

CONCLUSIONS

Currently, there are no external validated studies with a good predictive performance for residual disease. Studies of better quality are needed, especially studies that focus on predicting any residual disease after surgery.