Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

Anticholinergic burden for prediction of cognitive decline or neuropsychiatric symptoms in older adults with mild cognitive impairment or dementia

BACKGROUND

Medications with anticholinergic properties are commonly prescribed to older adults with a pre-existing diagnosis of dementia or cognitive impairment. The cumulative anticholinergic effect of all the medications a person takes is referred to as the anticholinergic burden because of its potential to cause adverse effects. It is possible that a high anticholinergic burden may be a risk factor for further cognitive decline or neuropsychiatric disturbances in people with dementia. Neuropsychiatric disturbances are the most frequent complication of dementia that require hospitalisation, accounting for almost half of admissions; hence, identification of modifiable prognostic factors for these outcomes is crucial. There are various scales available to measure anticholinergic burden but agreement between them is often poor.

OBJECTIVES

Our primary objective was to assess whether anticholinergic burden, as defined at the level of each individual scale, was a prognostic factor for further cognitive decline or neuropsychiatric disturbances in older adults with pre-existing diagnoses of dementia or cognitive impairment. Our secondary objective was to investigate whether anticholinergic burden was a prognostic factor for other adverse clinical outcomes, including mortality, impaired physical function, and institutionalisation.

SEARCH METHODS

We searched these databases from inception to 29 November 2021: MEDLINE OvidSP, Embase OvidSP, PsycINFO OvidSP, CINAHL EBSCOhost, and ISI Web of Science Core Collection on ISI Web of Science.

SELECTION CRITERIA

We included prospective and retrospective longitudinal cohort and case-control observational studies, with a minimum of one-month follow-up, which examined the association between an anticholinergic burden measurement scale and the above stated adverse clinical outcomes, in older adults with pre-existing diagnoses of dementia or cognitive impairment.   DATA COLLECTION AND ANALYSIS: Two review authors independently assessed studies for inclusion, and undertook data extraction, risk of bias assessment, and GRADE assessment. We summarised risk associations between anticholinergic burden and all clinical outcomes in a narrative fashion. We also evaluated the risk association between anticholinergic burden and mortality using a random-effects meta-analysis.  We established adjusted pooled rates for the anticholinergic cognitive burden (ACB) scale; then, as an exploratory analysis, established pooled rates on the prespecified association across scales.  MAIN RESULTS: We identified 18 studies that met our inclusion criteria (102,684 older adults). Anticholinergic burden was measured using five distinct measurement scales: 12 studies used the ACB scale; 3 studies used the Anticholinergic Risk Scale (ARS); 1 study used the Anticholinergic Drug Scale (ADS); 1 study used the Anticholinergic Effect on Cognition (AEC) Scale; and 2 studies used a list developed by Tune and Egeli.  Risk associations between anticholinergic burden and adverse clinical outcomes were highly heterogenous. Four out of 10 (40%) studies reported a significantly increased risk of greater long-term cognitive decline for participants with an anticholinergic burden compared to participants with no or minimal anticholinergic burden. No studies investigated neuropsychiatric disturbance outcomes. One out of four studies (25%) reported a significant association with reduced physical function for participants with an anticholinergic burden versus participants with no or minimal anticholinergic burden. No study (out of one investigating study) reported a significant association between anticholinergic burden and risk of institutionalisation. Six out of 10 studies (60%) found a significantly increased risk of mortality for those with an anticholinergic burden compared to those with no or minimal anticholinergic burden. Pooled analysis of adjusted mortality hazard ratios (HR) measured anticholinergic burden with the ACB scale, and suggested a significantly increased risk of death for those with a high ACB score relative to those with no or minimal ACB scores (HR 1.153, 95% confidence interval (CI) 1.030 to 1.292; 4 studies, 48,663 participants). An exploratory pooled analysis of adjusted mortality HRs across anticholinergic burden scales also suggested a significantly increased risk of death for those with a high anticholinergic burden (HR 1.102, 95% CI 1.044 to 1.163; 6 studies, 68,381 participants).   Overall GRADE evaluation of results found low- or very low-certainty evidence for all outcomes.  AUTHORS' CONCLUSIONS: There is low-certainty evidence that older adults with dementia or cognitive impairment who have a significant anticholinergic burden may be at increased risk of death. No firm conclusions can be drawn for risk of accelerated cognitive decline, neuropsychiatric disturbances, decline in physical function, or institutionalisation.

How to conduct a systematic review and meta-analysis of prognostic model studies

BACKGROUND

Prognostic models are typically developed to estimate the risk that an individual in a particular health state will develop a particular health outcome, to support (shared) decision making. Systematic reviews of prognostic model studies can help identify prognostic models that need to further be validated or are ready to be implemented in healthcare.

OBJECTIVES

To provide a step-by-step guidance on how to conduct and read a systematic review of prognostic model studies and to provide an overview of methodology and guidance available for every step of the review progress.

SOURCES

Published, peer-reviewed guidance articles.

CONTENT

We describe the following steps for conducting a systematic review of prognosis studies: 1) Developing the review question using the Population, Index model, Comparator model, Outcome(s), Timing, Setting format, 2) Searching and selection of articles, 3) Data extraction using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist, 4) Quality and risk of bias assessment using the Prediction model Risk Of Bias ASsessment (PROBAST) tool, 5) Analysing data and undertaking quantitative meta-analysis, and 6) Presenting summary of findings, interpreting results, and drawing conclusions. Guidance for each step is described and illustrated using a case study on prognostic models for patients with COVID-19.

IMPLICATIONS

Guidance for conducting a systematic review of prognosis studies is available, but the implications of these reviews for clinical practice and further research highly depend on complete reporting of primary studies.

Short- and long-term prognostic factors associated with functional recovery in elderly patients with hip fracture: A systematic review

This systematic review aimed to identify short- and long-term associated factors to functional recovery of elderly hip fracture patients after discharge. We identified 43 studies reporting 74 associated factors to functional recovery; most of them were biological, sociodemographic, or inherent factors to patients' baseline characteristics, including their pre-facture functional capacity.

PURPOSE

This systematic review aimed to identify short- and long-term associated factors to functional recovery of elderly hip fracture patients after hospital discharge. We assessed the use of the hip fracture core-set and key-performance indicators for secondary fracture reduction.

METHODS

A search was performed in seven electronic databases. Observational studies reporting predictors after usual care of elderly patients with hip fracture diagnoses receiving surgical or conservative treatment were included. Primary outcomes considered were part of the domains corresponding to functional capacity.

RESULTS

Of 3873 references identified, and after the screening and selection process, 43 studies were included. Sixty-one functional measures were identified for ten functional outcomes, including BADLs, IADLs, ambulation, and mobility. Biological characteristics such as age, sex, comorbidities, cognitive status, nutritional state, and biochemical parameters are significantly associated. Determinants such as contact and size of social network and those related to institutional care quality are relevant for functional recovery at six and 12 months. Age, pre-fracture function, cognitive status, and complications continue to be associated five years after discharge. We found 74 associated factors to functional recovery of elderly hip fracture patients. Ten of the studies reported rehabilitation programs as suggested in KPI 9; none used the complete hip fracture core-set.

CONCLUSION

Most of the associated factors for functional recovery of elderly hip fracture were biological, sociodemographic, or inherent factors to patients' baseline characteristics, including their pre-facture functional capacity. For the core-set and KPI's, we found an insufficient use and report. This study reports 61 different instruments to measure functional capacity.

REGISTRATION NUMBER

PROSPERO (CRD42020149563).

Maternal metabolic factors and the association with gestational diabetes: A systematic review and meta-analysis

Gestational diabetes (GDM) is associated with several adverse outcomes for the mother and child. Higher levels of individual lipids are associated with risk of GDM and metabolic syndrome (MetS), a clustering of risk factors also increases risk for GDM. Metabolic factors can be modified by diet and lifestyle. This review comprehensively evaluates the association between MetS and its components, measured in early pregnancy, and risk for GDM. Databases (Cumulative Index to Nursing and Allied Health Literature, PubMed, Embase, and Cochrane Library) were searched from inception to 5 May 2021. Eligible studies included ≥1 metabolic factor (waist circumference, blood pressure, fasting plasma glucose (FPG), triglycerides, and high-density lipoprotein cholesterol), measured at <16 weeks' gestation. At least two authors independently screened potentially eligible studies. Heterogeneity was quantified using I² . Data were pooled by random-effects models and expressed as odds ratio and 95% confidence intervals (CIs). Of 7213 articles identified, 40 unique articles were included in meta-analysis. In analyses adjusting for maternal age and body mass index, GDM was increased with increasing FPG (odds ratios [OR] 1.92; 95% CI 1.39-2.64, k = 7 studies) or having MetS (OR 2.52; 1.65, 3.84, k = 3). Women with overweight (OR 2.17; 95% CI 1.89, 2.50, k = 12) or obesity (OR 4.34; 95% CI 2.79-6.74, k = 9) also were at increased risk for GDM. Early pregnancy assessment of glucose or the MetS, offers a potential opportunity to detect and treat individual risk factors as an approach towards GDM prevention; weight loss for pregnant women with overweight or obesity is not recommended. Systematic review registration: PROSPERO CRD42020199225.

Elevated Cardiac Troponin to Detect Acute Cellular Rejection After Cardiac Transplantation: A Systematic Review and Meta-Analysis

Cardiac troponin is well known as a highly specific marker of cardiomyocyte damage, and has significant diagnostic accuracy in many cardiac conditions. However, the value of elevated recipient troponin in diagnosing adverse outcomes in heart transplant recipients is uncertain. We searched MEDLINE (Ovid), Embase (Ovid), and the Cochrane Library from inception until December 2020. We generated summary sensitivity, specificity, and Bayesian areas under the curve (BAUC) using bivariate Bayesian modelling, and standardised mean differences (SMDs) to quantify the diagnostic relationship of recipient troponin and adverse outcomes following cardiac transplant. We included 27 studies with 1,684 cardiac transplant recipients. Patients with acute rejection had a statistically significant late elevation in standardised troponin measurements taken at least 1 month postoperatively (SMD 0.98, 95% CI 0.33–1.64). However, pooled diagnostic accuracy was poor (sensitivity 0.414, 95% CrI 0.174–0.696; specificity 0.785, 95% CrI 0.567–0.912; BAUC 0.607, 95% CrI 0.469–0.723). In summary, late troponin elevation in heart transplant recipients is associated with acute cellular rejection in adults, but its stand-alone diagnostic accuracy is poor. Further research is needed to assess its performance in predictive modelling of adverse outcomes following cardiac transplant.

Systematic Review Registration: identifier CRD42021227861

Reproducibility of prediction models in health services research

The field of health services research studies the health care system by examining outcomes relevant to patients and clinicians but also health economists and policy makers. Such outcomes often include health care spending, and utilization of care services. Building accurate prediction models using reproducible research practices for health services research is important for evidence-based decision making. Several systematic reviews have summarized prediction models for outcomes relevant to health services research, but these systematic reviews do not present a thorough assessment of reproducibility and research quality of the prediction modelling studies. In the present commentary, we discuss how recent advances in prediction modelling in other medical fields can be applied to health services research. We also describe the current status of prediction modelling in health services research, and we summarize available methodological guidance for the development, update, external validation and systematic appraisal of prediction models.

Multivariable prediction models for long-term outcomes after hip fracture: A protocol for a systematic review

Background: Hip fracture results in high mortality and, for many survivors, long-term functional limitations. Multivariable prediction models for hip fracture outcomes have the potential to aid clinical-decision making as well as risk-adjustment in national audits of care. The aim of this study is to identify, critically appraise and synthesise published multivariable prediction models for long-term outcomes after hip fracture. Protocol: The systematic review will include a literature search of electronic databases (MEDLINE, Embase, Scopus, Web of Science and CINAHL) for journal articles. Search terms related to hip fracture, prognosis and outcomes will be included. Study selection criteria includes studies of people with hip fracture where the study aimed to predict one or more long-term outcomes through derivation or validation of a multivariable prediction model. Studies will be excluded if they focus only on the predictive value of individual factors, or only include patients with periprosthetic fractures, fractures managed non-surgically or younger patients. Covidence software will be used for data management. Two review authors will independently conduct study selection, data extraction and appraisal. Data will be extracted based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist. Risk of bias assessment will be conducted using the Prediction model Risk of Bias Assessment Tool (PROBAST). Characteristics and results of all studies will be narratively synthesised and presented in tables. Where the same model has been validated in multiple studies, a meta-analysis of discrimination and calibration will be conducted. Conclusions: This systematic review will aim to identify multivariable models for hip fracture outcome prognosis that have been derived using high quality methods. Results will highlight if current models have the potential for further assessment for use in both clinical decision making and improving methods of national hip fracture audits. PROSPERO registration: CRD42022330019 (25th May 2022).

Factors associated with concurrent malignancy risk among patients with incidental solitary pulmonary nodule: A systematic review taskforce for developing rapid recommendations

OBJECTIVE

To assess the association between prespecified factors and the malignancy risk of solitary pulmonary nodules (SPNs) to support the development of rapid recommendations for daily use in the Chinese setting.

METHODS

The expert panel for the rapid recommendations voted for 12 candidate factors based on published guidelines, selected publications, and clinical experiences. We then searched Medline, Embase, and Web of Science up to October 17, 2021, for studies investigating the association between these factors and the diagnosis of malignant SPNs in patients with CT-identified SPNs through multivariable regression analysis. The risk of bias was assessed using the Agency for Healthcare Research and Quality (AHRQ) Checklist. We pooled adjusted odds ratios (aOR) between candidate factors and the diagnosis of the malignant SPNs.

RESULTS

A total of 32 cross-sectional studies were included. Nine factors were statistically associated with malignant SPNs: age (aOR 1.06, 95% confidence interval [CI]: 1.05-1.07), smoking history (2.83, 1.84-4.36), history of extrathoracic malignancy (5.66, 2.80-11.46), history of malignancy (4.64, 3.37-6.39), family history of malignancy (3.11, 1.66-5.83), nodule diameter (1.23, 1.17-1.31), spiculation (3.41, 2.64-4.41), lobulation (3.85, 2.47-6.01), and mixed ground-glass opacity (mGGO) density of the nodule (5.56, 2.47-12.52). No statistical association was found between family history of lung cancer, emphysema, nodule border, and malignant SPNs.

CONCLUSION

Nine prespecified factors were associated with the concurrent malignancy risk among patients with SPNs. Risk stratification for SPNs is warranted in clinical practice.

Prediction models for functional status in community dwelling older adults: a systematic review

Background

Disability poses a burden for older persons, and is associated with poor outcomes and high societal costs. Prediction models could potentially identify persons who are at risk for disability. An up to date review of such models is missing.

Objective

To identify models developed for the prediction of functional status in community dwelling older persons.

Methods

A systematic review was performed including studies of older persons that developed and/or validated prediction models for the outcome functional status. Medline and EMBASE were searched, and reference lists and prospective citations were screened for additional references. Risk of bias was assessed using the PROBAST-tool. The performance of models was described and summarized, and the use of predictors was collated using the bag-of-words text mining procedure.

Results

Forty-three studies were included and reported 167 evaluations of prediction models. The median c-statistic values for the multivariable development models ranged between 0.65 and 0.76 (minimum = 0.58, maximum = 0.90), and were consistently higher than the values of the validation models for which median c-statistic values ranged between 0.6 and 0.68 (minimum = 0.50, maximum = 0.81). A total of 559 predictors were used in the models. The five predictors most frequently used were gait speed (n = 47), age (n = 38), cognition (n = 27), frailty (n = 24), and gender (n = 22).

Conclusions

No model can be recommended for implementation in practice. However, frailty models appear to be the most promising, because frailty components (e.g. gait speed) and frailty indexes demonstrated good to excellent predictive performance. However, the risk of study bias was high. Substantial improvements can be made in the methodology.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03156-7.

Quality and transparency of reporting derivation and validation prognostic studies of recurrent stroke in patients with TIA and minor stroke: a systematic review

BACKGROUND

Clinical prediction models/scores help clinicians make optimal evidence-based decisions when caring for their patients. To critically appraise such prediction models for use in a clinical setting, essential information on the derivation and validation of the models needs to be transparently reported. In this systematic review, we assessed the quality of reporting of derivation and validation studies of prediction models for the prognosis of recurrent stroke in patients with transient ischemic attack or minor stroke.

METHODS

MEDLINE and EMBASE databases were searched up to February 04, 2020. Studies reporting development or validation of multivariable prognostic models predicting recurrent stroke within 90 days in patients with TIA or minor stroke were included. Included studies were appraised for reporting quality and conduct using a select list of items from the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) Statement.

RESULTS

After screening 7026 articles, 60 eligible articles were retained, consisting of 100 derivation and validation studies of 27 unique prediction models. Four models were newly derived while 23 were developed by validating and updating existing models. Of the 60 articles, 15 (25%) reported an informative title. Among the 100 derivation and validation studies, few reported whether assessment of the outcome (24%) and predictors (12%) was blinded. Similarly, sample size justifications (49%), description of methods for handling missing data (16.1%), and model calibration (5%) were seldom reported. Among the 96 validation studies, 17 (17.7%) clearly reported on similarity (in terms of setting, eligibility criteria, predictors, and outcomes) between the validation and the derivation datasets. Items with the highest prevalence of adherence were the source of data (99%), eligibility criteria (93%), measures of discrimination (81%) and study setting (65%).

CONCLUSIONS

The majority of derivation and validation studies for the prognosis of recurrent stroke in TIA and minor stroke patients suffer from poor reporting quality. We recommend that all prediction model derivation and validation studies follow the TRIPOD statement to improve transparency and promote uptake of more reliable prediction models in practice.

TRIAL REGISTRATION

The protocol for this review was registered with PROSPERO (Registration number CRD42020201130 ).

Systematic review and meta-analysis of predictors of adjustment disorders in adults

BACKGROUND

The diagnosis of adjustment disorder is common in clinical practice, yet there is lack of research on the etiology and epidemiology of adjustment disorders. The goal of this systematic review was to evaluate predictors of adjustment disorders in adults.

METHODS

We conducted systematic searches in MEDLINE, EMBASE, and PsycINFO. We included 70 studies that examined thirteen theoretically-derived and predefined predictors of adjustment disorders with a total of 3,449,374 participants.

RESULTS

We found that female gender, younger age, unemployed status, stress, physical illness and injury, low social support, and a history of mental health disorders predicted adjustment disorders. Most of these predictors differentiated individuals with adjustment disorders from individuals with no mental health disorders. Participants with adjustment disorders were more likely to have experienced accidents than were those with posttraumatic stress disorder but were less likely to have experienced assaults and abuse, neglect, and maltreatment. More research is needed to identify factors that differentiate adjustment disorders from other mental health disorders.

LIMITATIONS

Because very few studies adjusted for confounders (e.g., demographic variables, mental health histories, and a variety of stressors), it was not possible to identify independent associations between predictors and adjustment disorders.

CONCLUSIONS

We identified a number of factors that predicted adjustment disorders compared to no mental health diagnosis. The majority of studies were rated as moderate or high in risk of bias, suggesting that more rigorous research is needed to confirm the relationships we detected.

Prediction Models for Radiation-Induced Neurocognitive Decline in Adult Patients With Primary or Secondary Brain Tumors: A Systematic Review

Purpose

Although an increasing body of literature suggests a relationship between brain irradiation and deterioration of neurocognitive function, it remains as the standard therapeutic and prophylactic modality in patients with brain tumors. This review was aimed to abstract and evaluate the prediction models for radiation-induced neurocognitive decline in patients with primary or secondary brain tumors.

Methods

MEDLINE was searched on October 31, 2021 for publications containing relevant truncation and MeSH terms related to “radiotherapy,” “brain,” “prediction model,” and “neurocognitive impairments.” Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool.

Results

Of 3,580 studies reviewed, 23 prediction models were identified. Age, tumor location, education level, baseline neurocognitive score, and radiation dose to the hippocampus were the most common predictors in the models. The Hopkins verbal learning (n = 7) and the trail making tests (n = 4) were the most frequent outcome assessment tools. All studies used regression (n = 14 linear, n = 8 logistic, and n = 4 Cox) as machine learning method. All models were judged to have a high risk of bias mainly due to issues in the analysis.

Conclusion

Existing models have limited quality and are at high risk of bias. Following recommendations are outlined in this review to improve future models: developing cognitive assessment instruments taking into account the peculiar traits of the different brain tumors and radiation modalities; adherence to model development and validation guidelines; careful choice of candidate predictors according to the literature and domain expert consensus; and considering radiation dose to brain substructures as they can provide important information on specific neurocognitive impairments.

Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE

Background

Systematic reviews (SRs) are valuable resources as they address specific clinical questions by summarizing all existing relevant studies. However, finding all information to include in systematic reviews can be challenging. Methodological search filters have been developed to find articles related to specific clinical questions. To our knowledge, no filter exists for finding studies on the role of prognostic factor (PF). We aimed to develop and evaluate a search filter to identify PF studies in Ovid MEDLINE that has maximum sensitivity.

Methods

We followed current recommendations for the development of a search filter by first identifying a reference set of PF studies included in relevant systematic reviews on the topic, and by selecting search terms using a word frequency analysis complemented with an expert panel discussion. We evaluated filter performance using the relative recall methodology.

Results

We constructed a reference set of 73 studies included in six systematic reviews from a larger sample. After completing a word frequency analysis using the reference set studies, we compiled a list of 80 of the frequent methodological terms. This list of terms was evaluated by the Delphi panel for inclusion in the filter, resulting in a final set of 8 appropriate terms. The consecutive connection of these terms with the Boolean operator OR produced the filter. We then evaluated the filter using the relative recall method against the reference set, comparing the references included in the SRs with our new search using the filter. The overall sensitivity of the filter was calculated to be 95%, while the overall specificity was 41%. The precision of the filter varied considerably, ranging from 0.36 to 17%. The NNR (number needed to read) value varied largely from 6 to 278. The time saved by using the filter ranged from 13–70%.

Conclusions

We developed a search filter for OVID-Medline with acceptable performance that could be used in systematic reviews of PF studies. Using this filter could save as much as 40% of the title and abstract screening task. The specificity of the filter could be improved by defining additional terms to be included, although it is important to evaluate any modification to guarantee the filter is still highly sensitive.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01595-9.

No prognostic model for people with recent-onset low back pain has yet been demonstrated to be suitable for use in clinical practice: a systematic review

OBJECTIVE

What model development and external validation studies exist that focus on the prognosis of patients with recent-onset low back pain (LBP)? What is the performance (in terms of discrimination and calibration) of these clinical prediction models?

METHODS

Systematic searches on MEDLINE, Embase and CINAHL were conducted. Model development and/or external validation studies of patients with recent-onset LBP were selected. Models predicting outcomes of pain, disability, sick leave, work absence and self-reported recovery, with at least 12 weeks of follow-up, were included. Risk of bias was assessed using the PROBAST instrument.

RESULTS

We identified 17 prognostic models developed to predict outcomes in people with recent-onset LBP: six models were in the development phase and 11 were in the validation phase. The most assessed prediction model was the Original Örebro Musculoskeletal Pain Questionnaire. The Da Silva Clinical Prediction Model was the only model, from a study with low risk of bias, that presented acceptable discrimination, demonstrating 'good' performance in predicting recovery from pain (C-statistic 0.71, 95% CI 0.63 to 0.78) and overall acceptable agreement in calibration.

CONCLUSION

Most prediction models for prognosis of patients with recent-onset LBP did not perform well at discrimination, few studies reported calibration and their performance varied across studies. It seems premature to advocate use of the available models, at their current state of development and validation, for low back pain in primary care, considering their generally poor methods and performance.

REGISTRATION

CRD42020160988.

Implementation of prognostic machine learning algorithms in paediatric chronic respiratory conditions: a scoping review

Machine learning (ML) holds great potential for predicting clinical outcomes in heterogeneous chronic respiratory diseases (CRD) affecting children, where timely individualised treatments offer opportunities for health optimisation. This paper identifies rate-limiting steps in ML prediction model development that impair clinical translation and discusses regulatory, clinical and ethical considerations for ML implementation. A scoping review of ML prediction models in paediatric CRDs was undertaken using the PRISMA extension scoping review guidelines. From 1209 results, 25 articles published between 2013 and 2021 were evaluated for features of a good clinical prediction model using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines.

Most of the studies were in asthma (80%), with few in cystic fibrosis (12%), bronchiolitis (4%) and childhood wheeze (4%). There were inconsistencies in model reporting and studies were limited by a lack of validation, and absence of equations or code for replication. Clinician involvement during ML model development is essential and diversity, equity and inclusion should be assessed at each step of the ML pipeline to ensure algorithms do not promote or amplify health disparities among marginalised groups. As ML prediction studies become more frequent, it is important that models are rigorously developed using published guidelines and take account of regulatory frameworks which depend on model complexity, patient safety, accountability and liability.

Prediction models for venous thromboembolism in ambulatory adults with pancreatic and gastro-oesophageal cancer: protocol for systematic review and meta-analysis

INTRODUCTION

Venous thromboembolism (VTE) is a common complication of cancer. Pancreatic and gastro-oesophageal cancers are among malignancies that have the highest rates of VTE occurrence. VTE can increase cancer-related morbidity and mortality and disrupt cancer treatment. The risk of VTE can be managed with measures such as using anticoagulant drugs, although the risk of bleeding may be an impeding factor. Therefore, a VTE risk assessment should be performed before the start of anticoagulation in individual patients. Several prediction models have been published, but most of them have low sensitivity and unknown clinical applicability in pancreatic or gastro-oesphageal cancers. We intend to do this systematic review to identify all applicable published predictive models and compare their performance in those types of cancer.

METHODS AND ANALYSIS

All studies in which a prediction model for VTE have been developed, validated or compared using adult ambulatory patients with pancreatic or gastro-oesphageal cancers will be identified and the reported predictive performance indicators will be extracted. Full text peer-reviewed journal articles of observational or experimental studies published in English will be included. Five databases (Medline, EMBASE, Web of Science, CINAHL and Cochrane) will be searched. Two reviewers will independently undertake each of the phases of screening, data extraction and risk of bias assessment. The quality of the selected studies will be assessed using Prediction model Risk Of Bias Assessment Tool. The results from the review will be used for a narrative information synthesis, and if the same models have been validated in multiple studies, meta-analyses will be done to pool the predictive performance measures.

ETHICS AND DISSEMINATION

There is no need for ethics approval because the review will use previously peer-reviewed articles. The results will be published.

PROSPERO REGISTRATION NUMBER

CRD42021253887.

The perceived importance of prognostic aspects considered by physicians during work disability evaluation: a survey

Background

Assessing prognosis is challenging for many physicians in various medical fields. Research shows that physicians who perform disability assessments consider six areas when evaluating a prognosis: disease, treatment, course of the disease, external information, patient-related and physician-related aspects. We administered a questionnaire to evaluate how physicians rate the importance of these six prognosis areas during work disability evaluation and to explore what kind of support they would like during prognosis assessment.

Methods

Seventy-six physicians scored the importance of 23 prognostic aspects distributed over six prognosis areas. Participants scored the importance of each aspect both “in general” and from the perspective of a case vignette of a worker with a severe degenerative disease. The questionnaire also covered needs and suggestions for support during the evaluation of prognoses.

Results

Medical areas that are related to the disease, or the treatment or course of the disease, appeared important (scores of 7.0–9.0), with less differing opinions among participants (IQR 1.0–3.0). Corresponding verbatim remarks supported the importance of disease and treatment as prognostic aspects. In comparison, patient- and physician-related aspects scored somewhat lower, with more variability (range 4.0–8.0, with IQR 2.0–5.0 for patient- and physician-related considerations). Participants indicated a need for a tool or online database that includes prognostic aspects and prognostic evidence.

Conclusions

Despite some variation in scores, the physicians rated all six prognosis areas as important for work disability evaluations. This study provides suggested aids to prognosis assessment, including an online support tool based on evidence-based medicine features.

Donor Cardiac Troponin for Prognosis of Adverse Outcomes in Cardiac Transplantation Recipients: a Systematic Review and Meta-analysis

BACKGROUND

Cardiac troponin is a highly specific and widely available marker of myocardial injury, and elevations in cardiac transplant donors may influence donor selection. We aimed to investigate whether elevated donor troponin has a role as a prognostic biomarker in cardiac transplantation.

METHODS

In a systematic review and meta-analysis, we searched MEDLINE, Embase, and the Cochrane Library, without language restriction, from inception to December 2020. We included studies reporting the association of elevated donor troponin with recipient outcome after cardiac transplant. We generated summary odds ratios and hazard ratios for the association of elevated donor troponin with short- and long-term adverse outcomes. Methodological quality was monitored using the Quality In Prognosis Studies tool, and interstudy heterogeneity was assessed using a series of sensitivity and subgroup analyses.

RESULTS

We included 17 studies involving 15 443 patients undergoing cardiac transplantation. Elevated donor troponin was associated with increased odds of graft rejection at 1 y (odds ratio, 2.54; 95% confidence interval, 1.22-5.28). No significant prognostic relationship was found between donor troponin and primary graft failure, short- to long-term mortality, cardiac allograft vasculopathy, and pediatric graft loss.

CONCLUSIONS

Elevated donor troponin is not associated with an increased short- or long-term mortality postcardiac transplant despite increasing the risk of graft rejection at 1 y. Accordingly, an elevated donor troponin in isolation should not exclude donation.

The comparative and added prognostic value of biomarkers to the Revised Cardiac Risk Index for preoperative prediction of major adverse cardiac events and all-cause mortality in patients who undergo noncardiac surgery

BACKGROUND

The Revised Cardiac Risk Index (RCRI) is a widely acknowledged prognostic model to estimate preoperatively the probability of developing in-hospital major adverse cardiac events (MACE) in patients undergoing noncardiac surgery. However, the RCRI does not always make accurate predictions, so various studies have investigated whether biomarkers added to or compared with the RCRI could improve this.

OBJECTIVES

Primary: To investigate the added predictive value of biomarkers to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery. Secondary: To investigate the prognostic value of biomarkers compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery. Tertiary: To investigate the prognostic value of other prediction models compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery.

SEARCH METHODS

We searched MEDLINE and Embase from 1 January 1999 (the year that the RCRI was published) until 25 June 2020. We also searched ISI Web of Science and SCOPUS for articles referring to the original RCRI development study in that period.

SELECTION CRITERIA

We included studies among adults who underwent noncardiac surgery, reporting on (external) validation of the RCRI and: - the addition of biomarker(s) to the RCRI; or - the comparison of the predictive accuracy of biomarker(s) to the RCRI; or - the comparison of the predictive accuracy of the RCRI to other models. Besides MACE, all other adverse outcomes were considered for inclusion.

DATA COLLECTION AND ANALYSIS

We developed a data extraction form based on the CHARMS checklist. Independent pairs of authors screened references, extracted data and assessed risk of bias and concerns regarding applicability according to PROBAST. For biomarkers and prediction models that were added or compared to the RCRI in ≥ 3 different articles, we described study characteristics and findings in further detail. We did not apply GRADE as no guidance is available for prognostic model reviews.

MAIN RESULTS

We screened 3960 records and included 107 articles.   Over all objectives we rated risk of bias as high in ≥ 1 domain in 90% of included studies, particularly in the analysis domain. Statistical pooling or meta-analysis of reported results was impossible due to heterogeneity in various aspects: outcomes used, scale by which the biomarker was added/compared to the RCRI, prediction horizons and studied populations.  Added predictive value of biomarkers to the RCRI Fifty-one studies reported on the added value of biomarkers to the RCRI. Sixty-nine different predictors were identified derived from blood (29%), imaging (33%) or other sources (38%). Addition of NT-proBNP, troponin or their combination improved the RCRI for predicting MACE (median delta c-statistics: 0.08, 0.14 and 0.12 for NT-proBNP, troponin and their combination, respectively). The median total net reclassification index (NRI) was 0.16 and 0.74 after addition of troponin and NT-proBNP to the RCRI, respectively. Calibration was not reported. To predict myocardial infarction, the median delta c-statistic when NT-proBNP was added to the RCRI was 0.09, and 0.06 for prediction of all-cause mortality and MACE combined. For BNP and copeptin, data were not sufficient to provide results on their added predictive performance, for any of the outcomes. Comparison of the predictive value of biomarkers to the RCRI  Fifty-one studies assessed the predictive performance of biomarkers alone compared to the RCRI. We identified 60 unique predictors derived from blood (38%), imaging (30%) or other sources, such as the American Society of Anesthesiologists (ASA) classification (32%). Predictions were similar between the ASA classification and the RCRI for all studied outcomes. In studies different from those identified in objective 1, the median delta c-statistic was 0.15 and 0.12 in favour of  BNP and NT-proBNP alone, respectively, when compared to the RCRI, for the prediction of MACE. For C-reactive protein, the predictive performance was similar to the RCRI. For other biomarkers and outcomes, data were insufficient to provide summary results. One study reported on calibration and none on reclassification. Comparison of the predictive value of other prognostic models to the RCRI   Fifty-two articles compared the predictive ability of the RCRI to other prognostic models. Of these, 42% developed a new prediction model, 22% updated the RCRI, or another prediction model, and 37% validated an existing prediction model. None of the other prediction models showed better performance in predicting MACE than the RCRI. To predict myocardial infarction and cardiac arrest, ACS-NSQIP-MICA had a higher median delta c-statistic of 0.11 compared to the RCRI. To predict all-cause mortality, the median delta c-statistic was 0.15 higher in favour of ACS-NSQIP-SRS compared to the RCRI. Predictive performance was not better for CHADS₂, CHA₂DS₂-VASc, R₂CHADS₂, Goldman index, Detsky index or VSG-CRI compared to the RCRI for any of the outcomes. Calibration and reclassification were reported in only one and three studies, respectively.

AUTHORS' CONCLUSIONS

Studies included in this review suggest that the predictive performance of the RCRI in predicting MACE is improved when NT-proBNP, troponin or their combination are added. Other studies indicate that BNP and NT-proBNP, when used in isolation, may even have a higher discriminative performance than the RCRI. There was insufficient evidence of a difference between the predictive accuracy of the RCRI and other prediction models in predicting MACE. However, ACS-NSQIP-MICA and ACS-NSQIP-SRS outperformed the RCRI in predicting myocardial infarction and cardiac arrest combined, and all-cause mortality, respectively. Nevertheless, the results cannot be interpreted as conclusive due to high risks of bias in a majority of papers, and pooling was impossible due to heterogeneity in outcomes, prediction horizons, biomarkers and studied populations. Future research on the added prognostic value of biomarkers to existing prediction models should focus on biomarkers with good predictive accuracy in other settings (e.g. diagnosis of myocardial infarction) and identification of biomarkers from omics data. They should be compared to novel biomarkers with so far insufficient evidence compared to established ones, including NT-proBNP or troponins. Adherence to recent guidance for prediction model studies (e.g. TRIPOD; PROBAST) and use of standardised outcome definitions in primary studies is highly recommended to facilitate systematic review and meta-analyses in the future.

Identifying older adults with frailty approaching end-of-life: A systematic review

BACKGROUND

People with frailty may have specific needs for end-of-life care, but there is no consensus on how to identify these people in a timely way, or whether they will benefit from intervention.

AIM

To synthesise evidence on identification of older people with frailty approaching end-of-life, and whether associated intervention improves outcomes.

DESIGN

Systematic review (PROSPERO: CRD42020462624).

DATA SOURCES

Six databases were searched, with no date restrictions, for articles reporting prognostic or intervention studies. Key inclusion criteria were adults aged 65 and over, identified as frail via an established measure. End-of-life was defined as the final 12 months. Key exclusion criteria were proxy definitions of frailty, or studies involving people with cancer, even if also frail.

RESULTS

Three articles met the inclusion criteria. Strongest evidence came from one study in English primary care, which showed distinct trajectories in electronic Frailty Index scores in the last 12 months of life, associated with increased risk of death. We found no studies evaluating established clinical tools (e.g. Gold Standards Framework) with existing frail populations. We found no intervention studies; the literature on advance care planning with people with frailty has relied on proxy definitions of frailty.

CONCLUSION

Clear implications for policy and practice are hindered by the lack of studies using an established approach to assessing frailty. Future end-of-life research needs to use explicit approaches to the measurement and reporting of frailty, and address the evidence gap on interventions. A focus on models of care that incorporate a palliative approach is essential.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation

Background

A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence.

Objective

The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature.

Methods

Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores.

Results

Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022.

Conclusions

The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching.

International Registered Report Identifier (IRRID)

DERR1-10.2196/29398

Optimal surveillance strategies for patients with stage 1 cutaneous melanoma post primary tumour excision: three systematic reviews and an economic model

BACKGROUND

Malignant melanoma is the fifth most common cancer in the UK, with rates continuing to rise, resulting in considerable burden to patients and the NHS.

OBJECTIVES

The objectives were to evaluate the effectiveness and cost-effectiveness of current and alternative follow-up strategies for stage IA and IB melanoma.

REVIEW METHODS

Three systematic reviews were conducted. (1) The effectiveness of surveillance strategies. Outcomes were detection of new primaries, recurrences, metastases and survival. Risk of bias was assessed using the Cochrane Collaboration's Risk-of-Bias 2.0 tool. (2) Prediction models to stratify by risk of recurrence, metastases and survival. Model performance was assessed by study-reported measures of discrimination (e.g. D-statistic, Harrel's c-statistic), calibration (e.g. the Hosmer-Lemeshow 'goodness-of-fit' test) or overall performance (e.g. Brier score, R ²). Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). (3) Diagnostic test accuracy of fine-needle biopsy and ultrasonography. Outcomes were detection of new primaries, recurrences, metastases and overall survival. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Review data and data from elsewhere were used to model the cost-effectiveness of alternative surveillance strategies and the value of further research.

RESULTS

(1) The surveillance review included one randomised controlled trial. There was no evidence of a difference in new primary or recurrence detected (risk ratio 0.75, 95% confidence interval 0.43 to 1.31). Risk of bias was considered to be of some concern. Certainty of the evidence was low. (2) Eleven risk prediction models were identified. Discrimination measures were reported for six models, with the area under the operating curve ranging from 0.59 to 0.88. Three models reported calibration measures, with coefficients of ≥ 0.88. Overall performance was reported by two models. In one, the Brier score was slightly better than the American Joint Committee on Cancer scheme score. The other reported an R ² of 0.47 (95% confidence interval 0.45 to 0.49). All studies were judged to have a high risk of bias. (3) The diagnostic test accuracy review identified two studies. One study considered fine-needle biopsy and the other considered ultrasonography. The sensitivity and specificity for fine-needle biopsy were 0.94 (95% confidence interval 0.90 to 0.97) and 0.95 (95% confidence interval 0.90 to 0.97), respectively. For ultrasonography, sensitivity and specificity were 1.00 (95% confidence interval 0.03 to 1.00) and 0.99 (95% confidence interval 0.96 to 0.99), respectively. For the reference standards and flow and timing domains, the risk of bias was rated as being high for both studies. The cost-effectiveness results suggest that, over a lifetime, less intensive surveillance than recommended by the National Institute for Health and Care Excellence might be worthwhile. There was considerable uncertainty. Improving the diagnostic performance of cancer nurse specialists and introducing a risk prediction tool could be promising. Further research on transition probabilities between different stages of melanoma and on improving diagnostic accuracy would be of most value.

LIMITATIONS

Overall, few data of limited quality were available, and these related to earlier versions of the American Joint Committee on Cancer staging. Consequently, there was considerable uncertainty in the economic evaluation.

CONCLUSIONS

Despite adoption of rigorous methods, too few data are available to justify changes to the National Institute for Health and Care Excellence recommendations on surveillance. However, alternative strategies warrant further research, specifically on improving estimates of incidence, progression of recurrent disease; diagnostic accuracy and health-related quality of life; developing and evaluating risk stratification tools; and understanding patient preferences.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42018086784.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol 25, No. 64. See the NIHR Journals Library website for further project information.

Anticholinergic burden for prediction of cognitive decline or neuropsychiatric symptoms in older adults with mild cognitive impairment or dementia

This is a protocol for a Cochrane Review (prognosis). The objectives are as follows:

Primary objective

To assess whether anticholinergic burden, as defined at the level of each individual scale, is a prognostic factor for further cognitive decline or neuropsychiatric disturbances in people with mild cognitive impairment (MCI) or dementia.

Secondary objective

To compare the prognostic validity of different anticholinergic burden scales.

To examine the effect of type of dementia and severity of dementia on the association between anticholinergic burden and rate of cognitive decline or neuropsychiatric disturbances.

To examine the effect of setting (care home versus non‐care home) on the association between anticholinergic burden and rate of cognitive decline or neuropsychiatric disturbances.

To examine whether anticholinergic burden is a prognostic factor for other clinical outcomes in people with MCI or dementia

Prognostic prediction models for chronic postsurgical pain in adults: a systematic review

ABSTRACT

Chronic postsurgical pain (CPSP) affects an estimated 10% to 50% of adults depending on the type of surgical procedure. Clinical prediction models can help clinicians target preventive strategies towards patients at high risk for CPSP. Therefore, the objective of this systematic review was to identify and describe existing prediction models for CPSP in adults. A systematic search was performed in MEDLINE, Embase, PsychINFO, and the Cochrane Database of Systematic Reviews in March 2020 for English peer-reviewed studies that used data collected between 2000 and 2020. Studies that developed, validated, or updated a prediction model in adult patients who underwent any surgical procedure were included. Two reviewers independently screened titles, abstracts, and full texts for eligibility; extracted data; and assessed risk of bias using the Prediction model Risk of Bias Assessment Tool. The search identified 2037 records; 28 articles were reviewed in full text. Fifteen studies reporting on 19 prediction models were included; all were at high risk of bias. Model discrimination, measured by the area under receiver operating curves or c-statistic, ranged from 0.690 to 0.816. The most common predictors identified in final prediction models included preoperative pain in the surgical area, preoperative pain in other areas, age, sex or gender, and acute postsurgical pain. Clinical prediction models may support prevention and management of CPSP, but existing models are at high risk of bias that affects their reliability to inform practice and generalizability to wider populations. Adherence to standardized guidelines for clinical prediction model development is necessary to derive a prediction model of value to clinicians.

Prognostic Models for Predicting Remission of Diabetes Following Bariatric Surgery: A Systematic Review and Meta-analysis

BACKGROUND

Remission of type 2 diabetes following bariatric surgery is well established, but identifying patients who will go into remission is challenging.

PURPOSE

To perform a systematic review of currently available diabetes remission prediction models, compare their performance, and evaluate their applicability in clinical settings.

DATA SOURCES

A comprehensive systematic literature search of MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL) was undertaken. The search was restricted to studies published in the last 15 years and in the English language.

STUDY SELECTION

All studies developing or validating a prediction model for diabetes remission in adults after bariatric surgery were included.

DATA EXTRACTION

The search identified 4,165 references, of which 38 were included for data extraction. We identified 16 model development and 22 validation studies.

DATA SYNTHESIS

Of the 16 model development studies, 11 developed scoring systems and 5 proposed logistic regression models. In model development studies, 10 models showed excellent discrimination with area under the receiver operating characteristic curve ≥0.800. Two of these prediction models, ABCD and DiaRem, were widely externally validated in different populations, in a variety of bariatric procedures, and for both short- and long-term diabetes remission. Newer prediction models showed excellent discrimination in test studies, but external validation was limited.

LIMITATIONS

While the key messages were consistent, a large proportion of the studies were conducted in small cohorts of patients with short duration of follow-up.

CONCLUSIONS

Among the prediction models identified, the ABCD and DiaRem models were the most widely validated and showed acceptable to excellent discrimination. More studies validating newer models and focusing on long-term diabetes remission are needed.

Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review

The evidence of the impact of traditional statistical (TS) and artificial intelligence (AI) tool interventions in clinical practice was limited. This study aimed to investigate the clinical impact and quality of randomized controlled trials (RCTs) involving interventions evaluating TS, machine learning (ML), and deep learning (DL) prediction tools. A systematic review on PubMed was conducted to identify RCTs involving TS/ML/DL tool interventions in the past decade. A total of 65 RCTs from 26,082 records were included. A majority of them had model development studies and generally good performance was achieved. The function of TS and ML tools in the RCTs mainly included assistive treatment decisions, assistive diagnosis, and risk stratification, but DL trials were only conducted for assistive diagnosis. Nearly two-fifths of the trial interventions showed no clinical benefit compared to standard care. Though DL and ML interventions achieved higher rates of positive results than TS in the RCTs, in trials with low risk of bias (17/65) the advantage of DL to TS was reduced while the advantage of ML to TS disappeared. The current applications of DL were not yet fully spread performed in medicine. It is predictable that DL will integrate more complex clinical problems than ML and TS tools in the future. Therefore, rigorous studies are required before the clinical application of these tools.

Prognostic factors for outcome following lumbar spine fusion surgery: a systematic review and narrative synthesis

PURPOSE

The objective of this study was to identify and evaluate the value of prognostic factors related to disability, pain and quality of life (QoL) for adult patients undergoing lumbar spine fusion surgery (LSFS).

METHODS

Two reviewers independently searched the literature, assessed eligibility, extracted data and assessed risk of bias and certainty of evidence. Key electronic databases were searched [PubMed, CINAHL, EMBASE, MEDLINE, PEDro and ZETOC] using pre-defined terms [e.g. LSFS] to 20/9/2020; with additional searching of journals, reference lists and unpublished literature. Prospective cohort studies with ≥ 12-month follow-up after LSFS were included. Narrative synthesis was based on recommendations by Cochrane Consumers and Communication Review Group. The GRADE tool enabled assessment of certainty of evidence. Prognostic factors and outcome were analysed and summarised when examined in ≥ 2 studies and when results pointed in the same direction in ≥ 75% of studies.

RESULTS

Sixteen studies (n =  8388, 2 low and 14 high risk of bias) were included with 39 prognostic factors identified. There is low certainty evidence that higher pre-operative severity of leg pain predicts greater improvement of leg pain and that pre-operative working predicts less post-operative disability both at 1-2-year follow-up. Other found associations were of very low certainty evidence.

CONCLUSION

No moderate to high certainty evidence exists. Use of leg pain and pre-operative working may be valuable predictors of outcome to inform clinical decision-making and advice regarding LSFS surgery. There is need for adequately powered low-risk-of-bias prospective observational studies to further investigate candidate prognostic factors.

The accuracy of diagnostic indicators for coeliac disease: A systematic review and meta-analysis

BACKGROUND

The prevalence of coeliac disease (CD) is around 1%, but diagnosis is challenged by varied presentation and non-specific symptoms and signs. This study aimed to identify diagnostic indicators that may help identify patients at a higher risk of CD in whom further testing is warranted.

METHODS

International guidance for systematic review methods were followed and the review was registered at PROSPERO (CRD42020170766). Six databases were searched until April 2021. Studies investigating diagnostic indicators, such as symptoms or risk conditions, in people with and without CD were eligible for inclusion. Risk of bias was assessed using the QUADAS-2 tool. Summary sensitivity, specificity, and positive predictive values were estimated for each diagnostic indicator by fitting bivariate random effects meta-analyses.

FINDINGS

191 studies reporting on 26 diagnostic indicators were included in the meta-analyses. We found large variation in diagnostic accuracy estimates between studies and most studies were at high risk of bias. We found strong evidence that people with dermatitis herpetiformis, migraine, family history of CD, HLA DQ2/8 risk genotype, anaemia, type 1 diabetes, osteoporosis, or chronic liver disease are more likely than the general population to have CD. Symptoms, psoriasis, epilepsy, inflammatory bowel disease, systemic lupus erythematosus, fractures, type 2 diabetes, and multiple sclerosis showed poor diagnostic ability. A sensitivity analysis revealed a 3-fold higher risk of CD in first-degree relatives of CD patients.

CONCLUSIONS

Targeted testing of individuals with dermatitis herpetiformis, migraine, family history of CD, HLA DQ2/8 risk genotype, anaemia, type 1 diabetes, osteoporosis, or chronic liver disease could improve case-finding for CD, therefore expediting appropriate treatment and reducing adverse consequences. Migraine and chronic liver disease are not yet included as a risk factor in all CD guidelines, but it may be appropriate for these to be added. Future research should establish the diagnostic value of combining indicators.

Which biopsychosocial factors are associated with the development of chronic musculoskeletal pain? Protocol for an umbrella review of systematic reviews

INTRODUCTION

Recent systematic reviews have identified many biopsychosocial factors associated with the development of chronic musculoskeletal pain (CMP). Despite often being specific to a particular musculoskeletal condition, findings are similar across systematic reviews. Research is needed to aggregate these findings to identify consistent factors across musculoskeletal disorders that are associated with the development of CMP. The objective of this study is to provide a meta-level synthesis of all biopsychosocial factors associated with the development of CMP.

METHODS AND ANALYSIS

An umbrella review and meta-level narrative synthesis±meta-analysis has been designed informed by Joanna Briggs Institute and Cochrane guidance. This protocol is reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis-P. Sources will include Ovid Medline, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, PsycINFO, CINAHL, PEDro, PROSPERO, Google Scholar and grey literature.

INCLUSION CRITERIA

any systematic review which investigates biopsychosocial factors which may be associated with the development of CMP through prospective longitudinal methods. The outcome is musculoskeletal pain lasting beyond 3 months. Two independent reviewers will be involved in all stages; screening, selection, data extraction and risk of bias evaluation using the Assessing the Methodological Quality of Systematic Reviews-2 guidelines. A meta-level narrative synthesis will be conducted based on (a) factors associated with development of CMP, (b) the range of musculoskeletal disorders for which the same/similar findings have been established and (c) the quality of studies informing these findings. Where possible, meta-analysis will be performed. The Grading of Recommendations, Assessment, Development and Evaluation guidelines will be followed to determine the level of evidence for each biopsychosocial factor.

ETHICS AND DISSEMINATION

This umbrella review does not require ethical approval. Findings will be presented at conferences and published in a peer reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42020193081.

Prognostic factors of depression and depressive symptoms after hip fracture surgery: systematic review

Background

Patients with hip fracture and depression are less likely to recover functional ability. This review sought to identify prognostic factors of depression or depressive symptoms up to 1 year after hip fracture surgery in adults. This review also sought to describe proposed underlying mechanisms for their association with depression or depressive symptoms.

Methods

We searched for published (MEDLINE, Embase, PsychInfo, CINAHL and Web of Science Core Collection) and unpublished (OpenGrey, Greynet, BASE, conference proceedings) studies. We did not impose any date, geographical, or language limitations. Screening (Covidence), extraction (Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies, adapted for use with prognostic factors studies Checklist), and quality appraisal (Quality in Prognosis Studies tool) were completed in duplicate. Results were summarised narratively.

Results

In total, 37 prognostic factors were identified from 12 studies included in this review. The quality of the underlying evidence was poor, with all studies at high risk of bias in at least one domain. Most factors did not have a proposed mechanism for the association. Where factors were investigated by more than one study, the evidence was often conflicting.

Conclusion

Due to conflicting and low quality of available evidence it is not possible to make clinical recommendations based on factors prognostic of depression or depressive symptoms after hip fracture. Further high-quality research investigating prognostic factors is warranted to inform future intervention and/or stratified approaches to care after hip fracture.

Trial registration

Prospero registration: CRD42019138690.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-021-02514-1.

The potential of prediction models of functioning remains to be fully exploited: A scoping review in the field of spinal cord injury rehabilitation

OBJECTIVE

The study aimed to explore existing prediction models of functioning in spinal cord injury (SCI).

STUDY DESIGN AND SETTING

The databases PubMed, EBSCOhost CINAHL Complete, and IEEE Xplore were searched for relevant literature. The search strategy included published search filters for prediction model and impact studies, index terms and keywords for SCI, and relevant outcome measures able to assess functioning as reflected in the International Classification of Functioning, Disability and Health (ICF). The search was completed in October 2020.

RESULTS

We identified seven prediction model studies reporting twelve prediction models of functioning. The identified prediction models were mainly envisioned to be used for rehabilitation planning, however, also other possible applications were stated. The method predominantly used was regression analysis and the investigated predictors covered mainly the ICF-components of body functions and activities and participation, next to characteristics of the health condition and health interventions.

CONCLUSION

Findings suggest that the development of prediction models of functioning for use in clinical practice remains to be fully exploited. By providing a comprehensive overview of what has been done, this review informs future research on prediction models of functioning in SCI and contributes to an efficient use of research evidence.

Perioperative Neutrophil-Lymphocyte Ratio Predicts Mortality After Cardiac Surgery: Systematic Review and Meta-Analysis

OBJECTIVES

Neutrophil-lymphocyte ratio (NLR) is an inflammatory biomarker that has been evaluated across a variety of surgical disciplines and is widely predictive of poor postoperative outcome, but its value in cardiac surgery is unclear. The authors did this systematic review and meta-analysis to determine the impact of elevated perioperative NLR on survival after cardiac surgery.

DESIGN

Systematic review and meta-analysis of study-level data.

SETTING

Multiple hospitals involved in an international pool of studies.

PARTICIPANTS

Adults undergoing cardiac surgery.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The authors searched multiple databases from inception until November 2020. They generated summary hazard ratios (HR) and odds ratios (OR) for the association of elevated preoperative NLR with long-term and short-term mortality following cardiac surgery. They separately reported on elevated postoperative NLR. Between-study heterogeneity was explored using metaregression. The authors included 12 studies involving 13,262 patients undergoing cardiac surgery. Elevated preoperative NLR was associated with worse long-term (>30 days) (hazard ratio [HR] 1.56; 95% CI [confidence interval], 1.18-2.06; 8 studies) and short-term (<30 days) mortality (OR 3.18; 95% CI, 1.90-5.30; 7 studies). One study reported the association of elevated postoperative NLR with long-term mortality (HR 8.58; 95% CI, 2.55-28.85). There was considerable between-study heterogeneity for the analysis of long-term mortality (I² statistic 94.39%), which mostly was explained by study-level variables, such as the number of variables adjusted for by included studies and how many of these significantly increased the risk of long-term mortality, high risk of bias, and number of study centers, as well as participant level factors, such as average participant age and hypertension prevalence.

CONCLUSIONS

Perioperative NLR is an independent predictor of short-term and long-term postoperative mortality following cardiac surgery. Further research is required to determine which patient-level factors modify the prognostic value of NLR and to evaluate its role in routine clinical practice.

Search methods for prognostic factor systematic reviews: a methodologic investigation

Objective

This study retroactively investigated the search used in a 2019 review by Hayden et al., one of the first systematic reviews of prognostic factors that was published in the Cochrane Library. The review was designed to address recognized weaknesses in reviews of prognosis by using multiple supplementary search methods in addition to traditional electronic database searching.

Methods

The authors used four approaches to comprehensively assess aspects of systematic review literature searching for prognostic factor studies: (1) comparison of search recall of broad versus focused electronic search strategies, (2) linking of search methods of origin for eligible studies, (3) analysis of impact of supplementary search methods on meta-analysis conclusions, and (4) analysis of prognosis filter performance.

Results

The review's focused electronic search strategy resulted in a 91% reduction in recall, compared to a broader version. Had the team relied on the focused search strategy without using supplementary search methods, they would have missed 23 of 58 eligible studies that were indexed in MEDLINE; additionally, the number of included studies in 2 of the review's primary outcome meta-analyses would have changed. Using a broader strategy without supplementary searches would still have missed 5 studies. The prognosis filter used in the review demonstrated the highest sensitivity of any of the filters tested.

Conclusions

Our study results support recommendations for supplementary search methods made by prominent systematic review methodologists. Leaving out any supplemental search methods would have resulted in missed studies, and these omissions would not have been prevented by using a broader search strategy or any of the other prognosis filters tested.

Prognostic factors for chronic post-surgical pain after lung or pleural surgery: a protocol for a systematic review and meta-analysis

INTRODUCTION

Chronic post-surgical pain (CPSP) after lung or pleural surgery is a common complication and associated with a decrease in quality of life, long-term use of pain medication and substantial economic costs. An abundant number of primary prognostic factor studies are published each year, but findings are often inconsistent, methods heterogeneous and the methodological quality questionable. Systematic reviews and meta-analyses are therefore needed to summarise the evidence.

METHODS AND ANALYSIS

The reporting of this protocol adheres to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) checklist. We will include retrospective and prospective studies with a follow-up of at least 3 months reporting patient-related factors and surgery-related factors for any adult population. Randomised controlled trials will be included if they report on prognostic factors for CPSP after lung or pleural surgery. We will exclude case series, case reports, literature reviews, studies that do not report results for lung or pleural surgery separately and studies that modified the treatment or prognostic factor based on pain during the observation period. MEDLINE, Scopus, Web of Science, Embase, Cochrane, CINAHL, Google Scholar and relevant literature reviews will be searched. Independent pairs of two reviewers will assess studies in two stages based on the PICOTS criteria. We will use the Quality in Prognostic Studies tool for the quality assessment and the CHARMS-PF checklist for the data extraction of the included studies. The analyses will all be conducted separately for each identified prognostic factor. We will analyse adjusted and unadjusted estimated measures separately. When possible, evidence will be summarised with a meta-analysis and otherwise narratively. We will quantify heterogeneity by calculating the Q and I² statistics. The heterogeneity will be further explored with meta-regression and subgroup analyses based on clinical knowledge. The quality of the evidence obtained will be evaluated according to the Grades of Recommendation Assessment, Development and Evaluation guideline 28.

ETHICS AND DISSEMINATION

Ethical approval will not be necessary, as all data are already in the public domain. Results will be published in a peer-reviewed scientific journal.

PROSPERO REGISTRATION NUMBER

CRD42021227888.

Anticholinergic burden measures and older people's falls risk: a systematic prognostic review

Introduction

Several adverse outcomes have been associated with anticholinergic burden (ACB), and these risks increase with age. Several approaches to measuring this burden are available but, to date, no comparison of their prognostic abilities has been conducted. This PROSPERO-registered systematic review (CRD42019115918) compared the evidence behind ACB measures in relation to their ability to predict risk of falling in older people.

Methods

Medline (OVID), EMBASE (OVID), CINAHL (EMBSCO) and PsycINFO (OVID) were searched using comprehensive search terms and a validated search filter for prognostic studies. Inclusion criteria included: participants aged 65 years and older, use of one or more ACB measure(s) as a prognostic factor, cohort or case-control in design, and reporting falls as an outcome. Risk of bias was assessed using the Quality in Prognosis Studies (QUIPS) tool.

Results

Eight studies reporting temporal associations between ACB and falls were included. All studies were rated high risk of bias in ⩾1 QUIPS tool categories, with five rated high risk ⩾3 categories. All studies (274,647 participants) showed some degree of association between anticholinergic score and increased risk of falls. Findings were most significant with moderate to high levels of ACB. Most studies (6/8) utilised the anticholinergic cognitive burden scale. No studies directly compared two or more ACB measures and there was variation in how falls were measured for analysis.

Conclusion

The evidence supports an association between moderate to high ACB and risk of falling in older people, but no conclusion can be made regarding which ACB scale offers best prognostic value in older people.

Plain language summary

A review of published studies to explore which anticholinergic burden scale is best at predicting the risk of falls in older people Introduction: One third of older people will experience a fall. Falls have many consequences including fractures, a loss of independence and being unable to enjoy life. Many things can increase the chances of having a fall. This includes some medications. One type of medication, known as anticholinergic medication, may increase the risk of falls. These medications are used to treat common health issues including depression and bladder problems. Anticholinergic burden is the term used to describe the total effects from taking these medications. Some people may use more than one of these medications. This would increase their anticholinergic burden. It is possible that reducing the use of these medications could reduce the risk of falls. We need to carry out studies to see if this is possible. To do this, we need to be able to measure anticholinergic burden. There are several scales available, but we do not know which is best.Methods: We wanted to answer: 'Which anticholinergic scale is best at predicting the risk of falling in older people?'. We reviewed studies that could answer this. We did this in a systematic way to capture all published studies. We restricted the search in several ways. We only included studies relevant to our question.Results: We found eight studies. We learned that people who are moderate to high users of these medications (often people who will use more than one of these medications) had a higher risk of falling. It was less clear if people who have a lower burden (often people who only use one of these medications) had an increased risk of falling. The low number of studies prevented us from determining if one scale was better than another.Conclusion: These findings suggest that we should reduce use of these medications. This could reduce the number falls and improve the well-being of older people.

Anticholinergic burden (prognostic factor) for prediction of dementia or cognitive decline in older adults with no known cognitive syndrome

BACKGROUND

Medications with anticholinergic properties are commonly prescribed to older adults. The cumulative anticholinergic effect of all the medications a person takes is referred to as the 'anticholinergic burden' because of its potential to cause adverse effects. It is possible that high anticholinergic burden may be a risk factor for development of cognitive decline or dementia. There are various scales available to measure anticholinergic burden but agreement between them is often poor.

OBJECTIVES

To assess whether anticholinergic burden, as defined at the level of each individual scale, is a prognostic factor for future cognitive decline or dementia in cognitively unimpaired older adults.

SEARCH METHODS

We searched the following databases from inception to 24 March 2021: MEDLINE (OvidSP), Embase (OvidSP), PsycINFO (OvidSP), CINAHL (EBSCOhost), and ISI Web of Science Core Collection (ISI Web of Science).

SELECTION CRITERIA

We included prospective and retrospective longitudinal cohort and case-control observational studies with a minimum of one year' follow-up that examined the association between an anticholinergic burden measurement scale and future cognitive decline or dementia in cognitively unimpaired older adults.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed studies for inclusion, and undertook data extraction, assessment of risk of bias, and GRADE assessment. We extracted odds ratios (OR) and hazard ratios, with 95% confidence intervals (CI), and linear data on the association between anticholinergic burden and cognitive decline or dementia. We intended to pool each metric separately; however, only OR-based data were suitable for pooling via a random-effects meta-analysis. We initially established adjusted and unadjusted pooled rates for each available anticholinergic scale; then, as an exploratory analysis, established pooled rates on the prespecified association across scales. We examined variability based on severity of anticholinergic burden.

MAIN RESULTS

We identified 25 studies that met our inclusion criteria (968,428 older adults). Twenty studies were conducted in the community care setting, two in primary care clinics, and three in secondary care settings. Eight studies (320,906 participants) provided suitable data for meta-analysis. The Anticholinergic Cognitive Burden scale (ACB scale) was the only scale with sufficient data for 'scale-based' meta-analysis. Unadjusted ORs suggested an increased risk for cognitive decline or dementia in older adults with an anticholinergic burden (OR 1.47, 95% CI 1.09 to 1.96) and adjusted ORs similarly suggested an increased risk for anticholinergic burden, defined according to the ACB scale (OR 2.63, 95% CI 1.09 to 6.29). Exploratory analysis combining adjusted ORs across available scales supported these results (OR 2.16, 95% CI 1.38 to 3.38), and there was evidence of variability in risk based on severity of anticholinergic burden (ACB scale 1: OR 2.18, 95% CI 1.11 to 4.29; ACB scale 2: OR 2.71, 95% CI 2.01 to 3.56; ACB scale 3: OR 3.27, 95% CI 1.41 to 7.61); however, overall GRADE evaluation of certainty of the evidence was low.

AUTHORS' CONCLUSIONS

There is low-certainty evidence that older adults without cognitive impairment who take medications with anticholinergic effects may be at increased risk of cognitive decline or dementia.

Predicting falls in community-dwelling older adults: a systematic review of prognostic models

OBJECTIVE

To systematically review and critically appraise prognostic models for falls in community-dwelling older adults.

ELIGIBILITY CRITERIA

Prospective cohort studies with any follow-up period. Studies had to develop or validate multifactorial prognostic models for falls in community-dwelling older adults (60+ years). Models had to be applicable for screening in a general population setting.

INFORMATION SOURCE

MEDLINE, EMBASE, CINAHL, The Cochrane Library, PsycINFO and Web of Science for studies published in English, Danish, Norwegian or Swedish until January 2020. Sources also included trial registries, clinical guidelines, reference lists of included papers, along with contacting clinical experts to locate published studies.

DATA EXTRACTION AND RISK OF BIAS

Two authors performed all review stages independently. Data extraction followed the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. Risk of bias assessments on participants, predictors, outcomes and analysis methods followed Prediction study Risk Of Bias Assessment Tool.

RESULTS

After screening 11 789 studies, 30 were eligible for inclusion (n=86 369 participants). Median age of participants ranged from 67.5 to 83.0 years. Falls incidences varied from 5.9% to 59%. Included studies reported 69 developed and three validated prediction models. Most frequent falls predictors were prior falls, age, sex, measures of gait, balance and strength, along with vision and disability. The area under the curve was available for 40 (55.6%) models, ranging from 0.49 to 0.87. Validated models' The area under the curve ranged from 0.62 to 0.69. All models had a high risk of bias, mostly due to limitations in statistical methods, outcome assessments and restrictive eligibility criteria.

CONCLUSIONS

An abundance of prognostic models on falls risk have been developed, but with a wide range in discriminatory performance. All models exhibited a high risk of bias rendering them unreliable for prediction in clinical practice. Future prognostic prediction models should comply with recent recommendations such as Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis.

PROSPERO REGISTRATION NUMBER

CRD42019124021.

Factors Associated With Violence Against Children in Low- and Middle-Income Countries: A Systematic Review and Meta-Regression of Nationally Representative Data

BACKGROUND

This systematic review and meta-regression sought to identify the relative importance of factors associated with physical, emotional, and sexual violence against children in low- and middle-income countries. Understanding of factors associated with violence is important for targeted programming and prevention on the population level.

METHODS

We searched 17 electronic databases from 1989 to 2018 and reports from child violence surveys. Nationally representative studies that described evidence on potential factors associated with violence against children under 18 years old were included. The search was restricted to the English language. Factors were synthesized quantitatively using robust variance estimation, with 95% confidence intervals, for each violence type.

RESULTS

We identified 8,346 unduplicated studies, and 103 publications met our eligibility criteria. The data distribution was uneven across region, country income status, factors, and violence types. Of the 94 eligible studies quantitatively synthesized, no specific factors were significant for physical violence. Lower household socioeconomic status, being a girl, and primary education of mothers and adults in the household were associated with emotional violence, and being a girl was associated with sexual violence.

CONCLUSION

A broad spectrum of factors merit consideration for physical violence policy and prevention among the general population of children in low- and middle-income countries. Conversely, a tailored approach may be warranted for preventing emotional and sexual violence. Information is unequally distributed across countries, factors, and violence types. Greater emphasis should be placed on collecting representative data on the general population and vulnerable subgroups to achieve national reductions in violence against children.

Externally validated model predicting gait independence after stroke showed fair performance and improved after updating

OBJECTIVE

To externally validate recent prognostic models that predict independent gait following stroke.

STUDY DESIGN AND SETTING

A systematic search identified recent models (<10 years) that predicted independent gait in adult stroke patients, using easily obtainable predictors. Predictors from the original models were assigned proxies when required, and model performance was evaluated in the validation cohort (n = 957). Models were updated to determine if performance could be improved.

RESULTS

Three prognostic models met our criteria, all with high Risk of Bias. Validation data was only available for the Australian model. This model used National Institute of Health Stroke Scale (NIHSS) and age to predict independent gait, using Motor Assessment Scale (MAS) walking item. For validation, Scandinavian Stroke Scale (SSS) was a proxy for NIHSS, and Functional Independence Measure (FIM) locomotion item was a proxy for MAS. The Area Under the Curve was 0.77 (0.74-0.80) and had good calibration in the validation dataset. Adjustment of the intercept and regression coefficients slightly improved discrimination. By adding paretic leg strength, the model further improved (AUC 0.82).

CONCLUSION

External validation of the Australian model with proxies showed fair discrimination and good calibration. Updating the model by adding paretic leg strength further improved model performance.

Optimizing a literature surveillance strategy to retrieve sound overall prognosis and risk assessment model papers

OBJECTIVE

Our aim was to develop an efficient search strategy for prognostic studies and clinical prediction guides (CPGs), optimally balancing sensitivity and precision while independent of MeSH terms, as relying on them may miss the most current literature.

MATERIALS AND METHODS

We combined 2 Hedges-based search strategies, modified to remove MeSH terms for overall prognostic studies and CPGs, and ran the search on 269 journals. We read abstracts from a random subset of retrieved references until ≥ 20 per journal were reviewed and classified them as positive when fulfilling standardized quality criteria, thereby assembling a standard dataset used to calibrate the search strategy. We determined performance characteristics of our new search strategy against the Hedges standard and performance characteristics of published search strategies against the standard dataset.

RESULTS

Our search strategy retrieved 16 089 references from 269 journals during our study period. One hundred fifty-four journals yielded ≥ 20 references and ≥ 1 prognostic study or CPG. Against the Hedges standard, the new search strategy had sensitivity/specificity/precision/accuracy of 84%/80%/2%/80%, respectively. Existing published strategies tested against our standard dataset had sensitivities of 36%-94% and precision of 5%-10%.

DISCUSSION

We developed a new search strategy to identify overall prognosis studies and CPGs independent of MeSH terms. These studies are important for medical decision-making, as they identify specific populations and individuals who may benefit from interventions.

CONCLUSION

Our results may benefit literature surveillance and clinical guideline efforts, as our search strategy performs as well as published search strategies while capturing literature at the time of publication.

Reviewing the use and quality of machine learning in developing clinical prediction models for cardiovascular disease

Cardiovascular disease (CVD) is one of the leading causes of death across the world. CVD can lead to angina, heart attacks, heart failure, strokes, and eventually, death; among many other serious conditions. The early intervention with those at a higher risk of developing CVD, typically with statin treatment, leads to better health outcomes. For this reason, clinical prediction models (CPMs) have been developed to identify those at a high risk of developing CVD so that treatment can begin at an earlier stage. Currently, CPMs are built around statistical analysis of factors linked to developing CVD, such as body mass index and family history. The emerging field of machine learning (ML) in healthcare, using computer algorithms that learn from a dataset without explicit programming, has the potential to outperform the CPMs available today. ML has already shown exciting progress in the detection of skin malignancies, bone fractures and many other medical conditions. In this review, we will analyse and explain the CPMs currently in use with comparisons to their developing ML counterparts. We have found that although the newest non-ML CPMs are effective, ML-based approaches consistently outperform them. However, improvements to the literature need to be made before ML should be implemented over current CPMs.

Systematic review of prediction models for pulmonary tuberculosis treatment outcomes in adults

OBJECTIVE

To systematically review and critically evaluate prediction models developed to predict tuberculosis (TB) treatment outcomes among adults with pulmonary TB.

DESIGN

Systematic review.

DATA SOURCES

PubMed, Embase, Web of Science and Google Scholar were searched for studies published from 1 January 1995 to 9 January 2020.

STUDY SELECTION AND DATA EXTRACTION

Studies that developed a model to predict pulmonary TB treatment outcomes were included. Study screening, data extraction and quality assessment were conducted independently by two reviewers. Study quality was evaluated using the Prediction model Risk Of Bias Assessment Tool. Data were synthesised with narrative review and in tables and figures.

RESULTS

14 739 articles were identified, 536 underwent full-text review and 33 studies presenting 37 prediction models were included. Model outcomes included death (n=16, 43%), treatment failure (n=6, 16%), default (n=6, 16%) or a composite outcome (n=9, 25%). Most models (n=30, 81%) measured discrimination (median c-statistic=0.75; IQR: 0.68-0.84), and 17 (46%) reported calibration, often the Hosmer-Lemeshow test (n=13). Nineteen (51%) models were internally validated, and six (16%) were externally validated. Eighteen (54%) studies mentioned missing data, and of those, half (n=9) used complete case analysis. The most common predictors included age, sex, extrapulmonary TB, body mass index, chest X-ray results, previous TB and HIV. Risk of bias varied across studies, but all studies had high risk of bias in their analysis.

CONCLUSIONS

TB outcome prediction models are heterogeneous with disparate outcome definitions, predictors and methodology. We do not recommend applying any in clinical settings without external validation, and encourage future researchers adhere to guidelines for developing and reporting of prediction models.

TRIAL REGISTRATION

The study was registered on the international prospective register of systematic reviews PROSPERO (CRD42020155782).

Prognostic factors and prediction models for acute aortic dissection: a systematic review

OBJECTIVE

Our study aimed to systematically review the methodological characteristics of studies that identified prognostic factors or developed or validated models for predicting mortalities among patients with acute aortic dissection (AAD), which would inform future work.

DESIGN/SETTING

A methodological review of published studies.

METHODS

We searched PubMed and EMBASE from inception to June 2020 for studies about prognostic factors or prediction models on mortality among patients with AAD. Two reviewers independently collected the information about methodological characteristics. We also documented the information about the performance of the prognostic factors or prediction models.

RESULTS

Thirty-two studies were included, of which 18 evaluated the performance of prognostic factors, and 14 developed or validated prediction models. Of the 32 studies, 23 (72%) were single-centre studies, 22 (69%) used data from electronic medical records, 19 (59%) chose retrospective cohort study design, 26 (81%) did not report missing predictor data and 5 (16%) that reported missing predictor data used complete-case analysis. Among the 14 prediction model studies, only 3 (21%) had the event per variable over 20, and only 5 (36%) reported both discrimination and calibration statistics. Among model development studies, 3 (27%) did not report statistical methods, 3 (27%) exclusively used statistical significance threshold for selecting predictors and 7 (64%) did not report the methods for handling continuous predictors. Most prediction models were considered at high risk of bias. The performance of prognostic factors showed varying discrimination (AUC 0.58 to 0.95), and the performance of prediction models also varied substantially (AUC 0.49 to 0.91). Only six studies reported calibration statistic.

CONCLUSIONS

The methods used for prognostic studies on mortality among patients with AAD-including prediction models or prognostic factor studies-were suboptimal, and the model performance highly varied. Substantial efforts are warranted to improve the use of the methods in this population.

A scoping review of causal methods enabling predictions under hypothetical interventions

BACKGROUND

The methods with which prediction models are usually developed mean that neither the parameters nor the predictions should be interpreted causally. For many applications, this is perfectly acceptable. However, when prediction models are used to support decision making, there is often a need for predicting outcomes under hypothetical interventions.

AIMS

We aimed to identify published methods for developing and validating prediction models that enable risk estimation of outcomes under hypothetical interventions, utilizing causal inference. We aimed to identify the main methodological approaches, their underlying assumptions, targeted estimands, and potential pitfalls and challenges with using the method. Finally, we aimed to highlight unresolved methodological challenges.

METHODS

We systematically reviewed literature published by December 2019, considering papers in the health domain that used causal considerations to enable prediction models to be used for predictions under hypothetical interventions. We included both methodologies proposed in statistical/machine learning literature and methodologies used in applied studies.

RESULTS

We identified 4919 papers through database searches and a further 115 papers through manual searches. Of these, 87 papers were retained for full-text screening, of which 13 were selected for inclusion. We found papers from both the statistical and the machine learning literature. Most of the identified methods for causal inference from observational data were based on marginal structural models and g-estimation.

CONCLUSIONS

There exist two broad methodological approaches for allowing prediction under hypothetical intervention into clinical prediction models: (1) enriching prediction models derived from observational studies with estimated causal effects from clinical trials and meta-analyses and (2) estimating prediction models and causal effects directly from observational data. These methods require extending to dynamic treatment regimes, and consideration of multiple interventions to operationalise a clinical decision support system. Techniques for validating 'causal prediction models' are still in their infancy.