Chapter 12: systematic review of prognostic tests

Clinical prognostic factors for older people: A systematic review and meta-analysis

OBJECTIVE

To explore the accuracy and precision of prognostic tools used in older people in predicting mortality, hospitalization, and nursing home admission across different settings and timings.

DESIGN

Systematic review and meta-analysis of prospective and retrospective studies.

DATA SOURCES

A systematic search from database inception until 01st February 2023 was run in Medline, Embase, Cinhal, Cochrane Library.

ELIGIBILITY CRITERIA

Studies were eligible if they reported accuracy (area under the curve [AUC]) and/or precision (C-index) for the prognostic index in relation to any of the following outcomes: mortality, hospitalization, and nursing home admission.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers extracted data. Data were pooled using a random effects model. The risk of bias was assessed with the Quality in Prognosis Studies (QUIPS) tool. If more than three studies for the same setting and time were available, a meta-analysis was performed and evaluated using the GRADE tool; other data were reported descriptively.

RESULTS

Among 16,082 studies initially considered, 159 studies with a total of 2398856 older people (mean age: 78 years) were included. The majority of the studies was carried out in hospital or medical wards. In the community setting, only two tools (Health Assessment Tool and the Multidimensional Prognostic Index, MPI) had good precision for long-term mortality. In emergency department setting, Barthel Index had an excellent accuracy in predicting short-term mortality. In medical wards, the MPI had a moderate certainty of the evidence in predicting short-term mortality (13 studies; 11,787 patients; AUC=0.79 and 4 studies; 3915 patients; C-index=0.82). Similar findings were available for MPI when considering longer follow-up periods. When considering nursing home and surgical wards, the literature was limited. The risk of bias was generally acceptable; observed bias was mainly owing to attrition and confounding.

CONCLUSIONS

Several tools are used to predict poor prognosis in geriatric patients, but only those derived from a multidimensional evaluation have the characteristics of precision and accuracy.

The Prognostic Utility of Anticholinergic Burden Scales: An Integrative Review and Gap Analysis

BACKGROUND

Anticholinergic drugs are commonly prescribed, especially to older adults. Anticholinergic burden scales (ABS) have been used to evaluate the cumulative effects of multiple anticholinergics. However, studies have shown inconsistent results regarding the association between anticholinergic burden assessed with ABS and adverse clinical outcomes such as cognitive impairment, functional decline, and frailty. This review aims to identify gaps in research on the development, validation, and evaluation of ABS, and provide recommendations for future studies.

METHOD

A comprehensive search of five databases (MEDLINE, Embase, PsychInfo, CINAHL, CENTRAL) was conducted for relevant studies published from inception until 25 May 2023. Two reviewers screened for eligibility and assessed the quality of studies using different tools based on the study design and stage of the review framework. Research evidence was evaluated, and gaps were identified and grouped into evidence, knowledge, and methodological gaps, using evidence tables to summarize data.

RESULTS

Several evidence, knowledge, and methodological gaps in existing development, validation, and evaluation studies of ABS were identified. There is no universally accepted scale, and there is a need to define a clinically relevant threshold for measuring total anticholinergic burden. The current evidence has limitations, underrepresenting low- and middle-income countries, younger individuals, and populations with cognitive disabilities. The impact of anticholinergic burden on frailty is also understudied. Existing evaluation studies provide limited evidence on the benefit of reducing anticholinergic burden on clinical outcomes or the safety of anticholinergic deprescribing. There is also uncertainty regarding optimal reduction, clinically significant anticholinergic burden thresholds, and cost effectiveness.

CONCLUSIONS

Future research recommendations to bridge knowledge gaps include developing a risk assessment framework, refining ABS scales, establishing a standardized consensus scale, and creating a longitudinal measure of cumulative anticholinergic risk. Strategies to minimize bias, consider frailty, and promote multidisciplinary and multinational collaborations are also necessary to improve patient outcomes.

From Computation to Clinic

Theory-driven and data-driven computational approaches to psychiatry have enormous potential for elucidating mechanism of disease and providing translational linkages between basic science findings and the clinic. These approaches have already demonstrated utility in providing clinically relevant understanding, primarily via back translation from clinic to computation, revealing how specific disorders or symptoms map onto specific computational processes. Nonetheless, forward translation, from computation to clinic, remains rare. In addition, consensus regarding specific barriers to forward translation-and on the best strategies to overcome these barriers-is limited. This perspective review brings together expert basic and computationally trained researchers and clinicians to 1) identify challenges specific to preclinical model systems and clinical translation of computational models of cognition and affect, and 2) discuss practical approaches to overcoming these challenges. In doing so, we highlight recent evidence for the ability of computational approaches to predict treatment responses in psychiatric disorders and discuss considerations for maximizing the clinical relevance of such models (e.g., via longitudinal testing) and the likelihood of stakeholder adoption (e.g., via cost-effectiveness analyses).

Practical and validated tool to assess falls risk in the primary care setting: a systematic review

OBJECTIVE

Although several falls risk assessment tools are available, it is unclear which have been validated and which would be most suitable for primary care practices. This systematic review aims to identify the most suitable falls risk assessment tool for the primary care setting (ie, requires limited time, no expensive equipment and no additional space) and that has good predictive performance in the assessment of falls risk among older people living independently.

DESIGN

A systematic review based on prospective studies.

METHODS

An extensive search was conducted in the following databases: PubMed, Embase, CINAHL, Cochrane and PsycINFO. Tools were excluded if they required expensive and/or advanced software that is not usually available in primary care units and if they had not been validated in at least three different studies. Of 2492 articles published between January 2000 and July 2020, 27 were included.

RESULTS

Six falls risk assessment tools were identified: Timed Up and Go (TUG) test, Gait Speed test, Berg Balance Scale, Performance Oriented Mobility Assessment, Functional Reach test and falls history. Most articles reported area under the curve (AUC) values ranging from 0.5 to 0.7 for these tools. Sensitivity and specificity varied substantially across studies (eg, TUG, sensitivity:10%-83.3%, specificity:28.4%-96.6%).

CONCLUSIONS

Given that none of the falls risk assessment tools had sufficient predictive performance (AUC <0.7), other ways of assessing high falls risk among independently living older people in primary care should be investigated. For now, the most suitable way to assess falls risk in the primary care setting appears to involve asking patients about their falls history. Compared with the other five tools, the falls history requires the least amount of time, no expensive equipment, no training and no spatial adjustments. The clinical judgement of healthcare professionals continues to be most important, as it enables the identification of high falls risk even for patients with no falls history.

TRIAL REGISTRAION NUMBER

The Netherlands Trial Register, NL7917; Pre-results.

Effect of estimating equations for glomerular filtration rate on novel surrogate markers for renal outcome

Backgrounds

Recently, alternative surrogate endpoints such as a 30% or 40% decline in estimated glomerular filtration rate (eGFR) or eGFR slope over 2 to 3 years have been proposed for predicting renal outcomes. However, the impact of GFR estimation methods on the accuracy and effectiveness of surrogate markers is unknown.

Methods

We retrospectively enrolled participants in health screening programs at three hospitals from 1995 to 2009. We defined two different participant groups as YR1 and YR3, which had available 1-year or 3-year eGFR values along with their baseline eGFR levels. We compared the effectiveness of eGFR percentage change or slope to estimate end-stage renal disease (ESRD) risk according to two estimating equations (modified Modification of Diet in Renal Disease equation [eGFRm] and Chronic Kidney Disease-Epidemiology Collaboration (CKD-EPI) equation [eGFRc]) for GFR.

Results

In the YR1 and YR3 groups, 9,971 and 10,171 candidates were enrolled and ESRD incidence during follow-up was 0.26% and 0.19%, respectively. The eGFR percentage change was more effective than eGFR slope in estimating ESRD risk, regardless of the method of estimation. A 40% of decline in eGFR was better than 30%, and a 3-year baseline period was better than a 1-year period for prediction accuracy. Although some diagnostic indices from the CKD-EPI equation were better, we found no significant differences in the discriminative ability and hazard ratios for incident ESRD between eGFRc and eGFRm in either eGFR percentage change or eGFR slope.

Conclusion

There were no significant differences in the prediction accuracy of GFR percentage change or eGFR slope between eGFRc and eGFRm in the general population.

Computational approaches and machine learning for individual-level treatment predictions

RATIONALE

The impact of neuroscience-based approaches for psychiatry on pragmatic clinical decision-making has been limited. Although neuroscience has provided insights into basic mechanisms of neural function, these insights have not improved the ability to generate better assessments, prognoses, diagnoses, or treatment of psychiatric conditions.

OBJECTIVES

To integrate the emerging findings in machine learning and computational psychiatry to address the question: what measures that are not derived from the patient's self-assessment or the assessment by a trained professional can be used to make more precise predictions about the individual's current state, the individual's future disease trajectory, or the probability to respond to a particular intervention?

RESULTS

Currently, the ability to use individual differences to predict differential outcomes is very modest possibly related to the fact that the effect sizes of interventions are small. There is emerging evidence of genetic and neuroimaging-based heterogeneity of psychiatric disorders, which contributes to imprecise predictions. Although the use of machine learning tools to generate clinically actionable predictions is still in its infancy, these approaches may identify subgroups enabling more precise predictions. In addition, computational psychiatry might provide explanatory disease models based on faulty updating of internal values or beliefs.

CONCLUSIONS

There is a need for larger studies, clinical trials using machine learning, or computational psychiatry model parameters predictions as actionable outcomes, comparing alternative explanatory computational models, and using translational approaches that apply similar paradigms and models in humans and animals.

Subjective Motoric Complaints and New Onset Slow Gait

BACKGROUND

While reports of mobility problems are common with aging, their relationship to new onset of slow gait is unknown. Our objective was to examine the validity of subjective motoric complaints for predicting the incidence of slow gait.

METHODS

Ambulatory community-residing participants (mean age 76.6, 55% women) with gait speeds in the normal range enrolled in an aging cohort. Five subjective motoric complaints were assessed. Incident slow gait (walking speed 1 SD below age and sex means) was the primary outcome.

RESULTS

Of the 548 participants at baseline, 90 had prevalent slow gait and 253 participants (73.7%) reported one or more subjective motoric complaints. Subjective motoric complaints were more common in women than men (1.78 vs 1.23). Over a median follow-up of 3.34 years, 68 participants developed new onset slow gait. All 5 questions predicted incident slow gait (adjusted hazard ratios varying from 2.26 to 4.44). More subjective motoric complaints were associated with increased risk of developing incident slow gait (hazard ratio per complaint 1.81). Predictive validity of subjective motoric complaints for incident slow gait was unchanged when using alternate outcome definitions, accounting for diagnostic misclassification, recall bias, or adjusting for multiple confounders.

CONCLUSIONS

Subjective motoric complaints are a harbinger of mobility disability, and can help improve clinical risk assessments and identify high-risk individuals for interventions to prevent onset of slow gait.

Performance of Gene Expression Profile Tests for Prognosis in Patients With Localized Cutaneous Melanoma: A Systematic Review and Meta-analysis

Importance

The performance of prognostic gene expression profile (GEP) tests for cutaneous melanoma is poorly characterized.

Objective

To systematically assess the performance of commercially available GEP tests in patients with American Joint Committee on Cancer (AJCC) stage I or stage II disease.

Data Sources

For this systematic review and meta-analysis, comprehensive searches of PubMed/MEDLINE, Embase, and Web of Science were conducted on December 12, 2019, for English-language studies of humans without date restrictions.

Study Selection

Two reviewers identified GEP external validation studies of patients with localized melanoma. After exclusion criteria were applied, 7 studies (8%; 5 assessing DecisionDx-Melanoma and 2 assessing MelaGenix) were included.

Data Extraction and Synthesis

Data were extracted using an adaptation of the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS-PF). When feasible, meta-analysis using random-effects models was performed. Risk of bias and level of evidence were assessed with the Quality in Prognosis Studies tool and an adaptation of Grading of Recommendations Assessment, Development, and Evaluation.

Main Outcomes and Measures

Proportion of patients with or without melanoma recurrence correctly classified by the GEP test as being at high or low risk.

Results

In the 7 included studies, a total of 1450 study participants contributed data (age and sex unknown). The performance of both GEP tests varied by AJCC stage. Of patients tested with DecisionDx-Melanoma, 623 had stage I disease (6 true-positive [TP], 15 false-negative, 61 false-positive, and 541 true-negative [TN] results) and 212 had stage II disease (59 TP, 13 FN, 78 FP, and 62 TN results). Among patients with recurrence, DecisionDx-Melanoma correctly classified 29% with stage I disease and 82% with stage II disease. Among patients without recurrence, the test correctly classified 90% with stage I disease and 44% with stage II disease. Of patients tested with MelaGenix, 88 had stage I disease (7 TP, 15 FN, 15 FP, and 51 TN results) and 245 had stage II disease (59 TP, 19 FN, 95 FP, and 72 TN results). Among patients with recurrence, MelaGenix correctly classified 32% with stage I disease and 76% with stage II disease. Among patients without recurrence, the test correctly classified 77% with stage I disease and 43% with stage II disease.

Conclusions and Relevance

The prognostic ability of GEP tests among patients with localized melanoma varied by AJCC stage and appeared to be poor at correctly identifying recurrence in patients with stage I disease, suggesting limited potential for clinical utility in these patients.

Prognostic Gene Expression Profiling in Cutaneous Melanoma: Identifying the Knowledge Gaps and Assessing the Clinical Benefit

Importance

Use of prognostic gene expression profile (GEP) testing in cutaneous melanoma (CM) is rising despite a lack of endorsement as standard of care.

Objective

To develop guidelines within the national Melanoma Prevention Working Group (MPWG) on integration of GEP testing into the management of patients with CM, including (1) review of published data using GEP tests, (2) definition of acceptable performance criteria, (3) current recommendations for use of GEP testing in clinical practice, and (4) considerations for future studies.

Evidence Review

The MPWG members and other international melanoma specialists participated in 2 online surveys and then convened a summit meeting. Published data and meeting abstracts from 2015 to 2019 were reviewed.

Findings

The MPWG members are optimistic about the future use of prognostic GEP testing to improve risk stratification and enhance clinical decision-making but acknowledge that current utility is limited by test performance in patients with stage I disease. Published studies of GEP testing have not evaluated results in the context of all relevant clinicopathologic factors or as predictors of regional nodal metastasis to replace sentinel lymph node biopsy (SLNB). The performance of GEP tests has generally been reported for small groups of patients representing particular tumor stages or in aggregate form, such that stage-specific performance cannot be ascertained, and without survival outcomes compared with data from the American Joint Committee on Cancer 8th edition melanoma staging system international database. There are significant challenges to performing clinical trials incorporating GEP testing with SLNB and adjuvant therapy. The MPWG members favor conducting retrospective studies that evaluate multiple GEP testing platforms on fully annotated archived samples before embarking on costly prospective studies and recommend avoiding routine use of GEP testing to direct patient management until prospective studies support their clinical utility.

Conclusions and Relevance

More evidence is needed to support using GEP testing to inform recommendations regarding SLNB, intensity of follow-up or imaging surveillance, and postoperative adjuvant therapy. The MPWG recommends further research to assess the validity and clinical applicability of existing and emerging GEP tests. Decisions on performing GEP testing and patient management based on these results should only be made in the context of discussion of testing limitations with the patient or within a multidisciplinary group.

World Workshop on Oral Medicine VII: Biomarkers predicting lymphoma in the salivary glands of patients with Sjögren's syndrome-A systematic review

OBJECTIVE

To conduct a systematic review of studies exploring potential biomarkers for development, course, and efficacy of treatment of lymphomas in salivary glands of patients with Sjögren's syndrome.

MATERIAL AND METHODS

Eligible studies were identified through a comprehensive search of two databases, that is, PubMed and EMBASE. Quality of included articles was assessed with the "Quality In Prognosis Studies" (QUIPS) tool. The "CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies" (CHARMS) was used to facilitate data extraction.

RESULTS

Fifty-eight studies met the inclusion criteria. Only one study assessed the progression of lymphoma. Moderate risk of bias was detected in "outcome measurement," "study participation," and "study confounding" domains. Parotid gland enlargement, mixed monoclonal cryoglobulins, and low C4 levels represented strongest predictors of lymphoma development. The role of histological biomarkers, and specifically germinal centers, remains controversial. Clinical and methodological heterogeneity across studies precluded conduct of a meta-analysis.

CONCLUSIONS

Specific biomarkers in combination with clinical manifestations represent potential candidates for advancing precision medicine approaches to lymphoma prediction in patients with Sjögren's syndrome. Current focus has increasingly been on genetic and epigenetic markers as candidate predictors. Predictive accuracy of key biomarker candidates remains to be tested in well-designed prospectively followed Sjögren's syndrome cohorts.

Timing and methods of frailty assessments in geriatric trauma patients: A systematic review

INTRODUCTION

The trauma population is aging and better prognostic measures for geriatric trauma patients are required. Frailty rather than age appears to be associated with poor outcomes. This systematic review aimed to identify the optimum frailty assessment instrument and timing of assessment in patients aged over 65 years admitted to hospital after traumatic injury. The secondary aim was to evaluate outcomes associated with frailty in elderly trauma populations.

METHODS

This systematic review was registered with the PROSPERO International Prospective Register of Systematic Reviews (CRD42018090620). A MEDLINE and EMBASE literature search was conducted from inception to June 2019 combining the concepts of injury, geriatric, frailty, assessment and prognosis. Included studies were in patients 65 years or older hospitalised after injury and exposed to an instrument meeting consensus definition for frailty assessment. Study quality was assessed using criteria for review of prognostic studies combined with a GRADE approach.

RESULTS

Twenty-eight papers met inclusion criteria. Twenty-eight frailty or component instruments were reported, and assessments of pre-injury frailty were made up to 1-year post injury. Pre-injury frailty prevalence varied from 13% (13/100) to 94% (17/18), with in-hospital mortality rates from 2% (5/250) to 33% (6/18). Eleven studies found an association between frailty and mortality. Eleven studies reported an association between frailty and a composite outcome of mortality and adverse discharge destination. Generalisability and assessment of strength of associations was limited by single centre studies with inconsistent findings and overlapping cohorts.

CONCLUSIONS

Associations between frailty and adverse outcomes including mortality in geriatric trauma patients were demonstrated despite a range of frailty instruments, administering clinicians, time of assessment and data sources. Although evidence gaps remain, incorporating frailty assessment into trauma systems is likely to identify geriatric patients at risk of adverse outcomes. Consistency in frailty instruments and long-term geriatric specific outcome measures will improve research relevance.

LEVEL OF EVIDENCE

Level III prognostic.

Comparative Test Evaluation: Methods and Challenges

The present paper has three objectives. First, methods for comparing alternative tests for the purpose of replacement of one test with a second presumably superior test are described. Second, problems in the interpretation of the relevance of different diagnostic thresholds (thresholds of positivity) that define who is and who is not a disordered gambler are examined and a potential solution offered in the form of a common quantitative measure of the risk of being a disordered gambler. Third, alternative methodologies are described as potential solutions to the lack of a gold or reference standard in the evaluation of new tests.

What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences

BACKGROUND

Systematic reviews have been considered as the pillar on which evidence-based healthcare rests. Systematic review methodology has evolved and been modified over the years to accommodate the range of questions that may arise in the health and medical sciences. This paper explores a concept still rarely considered by novice authors and in the literature: determining the type of systematic review to undertake based on a research question or priority.

RESULTS

Within the framework of the evidence-based healthcare paradigm, defining the question and type of systematic review to conduct is a pivotal first step that will guide the rest of the process and has the potential to impact on other aspects of the evidence-based healthcare cycle (evidence generation, transfer and implementation). It is something that novice reviewers (and others not familiar with the range of review types available) need to take account of but frequently overlook. Our aim is to provide a typology of review types and describe key elements that need to be addressed during question development for each type.

CONCLUSIONS

In this paper a typology is proposed of various systematic review methodologies. The review types are defined and situated with regard to establishing corresponding questions and inclusion criteria. The ultimate objective is to provide clarified guidance for both novice and experienced reviewers and a unified typology with respect to review types.

Predicting late-onset sepsis by routine neonatal screening for colonisation by gram-negative bacteria in neonates at intensive care units: a protocol for a systematic review

INTRODUCTION

Hospitals conduct extensive screening procedures to assess colonisation of the body surface of neonates by gram-negative bacteria to avoid complications like late-onset sepsis. However, the benefits of these procedures are controversially discussed. Until now, no systematic review has investigated the value of routine screening for colonisation by gram-negative bacteria in neonates for late-onset sepsis prediction.

METHODS AND ANALYSIS

We will conduct a systematic review, considering studies of any design that include infants up to an age of 12 months. We will search MEDLINE and EMBASE (inception to 2016), reference lists and grey literature. Screening of titles, abstracts and full texts will be conducted by two independent reviewers. We will extract data on study characteristics and study results. Risk of bias will be assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Quality in Prognosis Studies (QUIPS) tools. Subgroup analyses are planned according to characteristics of studies, participants, index tests and outcome. For quantitative data synthesis on prognostic accuracy, sensitivity and specificity of screening to detect late-onset sepsis will be calculated. If sufficient data are available, we will calculate summary estimates using hierarchical summary receiver operating characteristics and bivariate models. Applying a risk factor approach, pooled summary estimates will be calculated as relative risk or OR, using fixed-effects and random-effects models. I-squared will be used to assess heterogeneity. All calculations will be performed in Stata V14.1 (College Station, Texas, USA). The results will be used to calculate positive and negative predictive value and number needed to be screened to prevent one case of sepsis. Grading of Recommendations Assessment, Development and Evaluation (GRADE) will be used to assess certainty in the evidence. The protocol follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guideline.

ETHICS AND DISSEMINATION

This study will not require ethical approval since it is not carried out in humans. The systematic review will be published in an open-access peer-reviewed journal.

TRIAL REGISTRATION NUMBER

CRD42016036664.

A systematic review of prognostic factors at the end of life for people with a hematological malignancy

Background

Accurate prognosticating is needed when patients are nearing the end of life to ensure appropriate treatment decisions, and facilitate palliative care provision and transitioning to terminal care. People with a hematological malignancy characteristically experience a fluctuating illness trajectory leading to difficulties with prognosticating. The aim of this review was to identify current knowledge regarding ‘bedside’ prognostic factors in the final 3 months of life for people with a hematological malignancy associated with increased risk of mortality.

Methods

A systematic review of the literature was performed across: PubMed; CINAHL; PsycINFO; and Cochrane with set inclusion criteria: 1) prognostic cohort studies; 2) published 2004–2014; 3) sample ≥ 18 years; 4) >50% sample had a hematological malignancy; 5) reported ‘bedside’ prognostic factors; 6) median survival of <3 months; and 7) English language. Quality appraisal was performed using the Quality In Prognostic Studies (QUIPS) tool. Results are reported in line with PRISMA guidelines.

Results

The search returned 4860 studies of which 28 met inclusion criteria. Twenty-four studies were rated moderate quality, three were high quality and one study was deemed to be of low quality. Most studies were set in the ICU (n = 24/28) and were retrospective (n = 25/28). Forty ‘bedside’ prognostic factors were identified as associated with increased risk of mortality encompassing the following broad categories: 1) demographics; 2) physiological complications or conditions; 3) disease characteristics; 4) laboratory blood values; and 5) interventions.

Conclusions

The literature on prognosticating in the final months of life was predominantly focused on people who had experienced acute physiological deterioration and were being treated aggressively in the in-patient setting. A significant gap in the literature exists for people who are treated less aggressively or are on a palliative trajectory. Findings did not report on, or confirm the significance of, many of the key prognostic factors associated with increased risk of mortality at the end of life in the solid tumour population, demonstrating key differences in the two populations.

Trial registration

This systematic review was not registered.

Electronic supplementary material

The online version of this article (doi:10.1186/s12885-017-3207-7) contains supplementary material, which is available to authorized users.

Prognostic Value of Pre-operative Renal Insufficiency in Urothelial Carcinoma: A Systematic Review and Meta-Analysis

The effect of pre-operative renal insufficiency on urothelial carcinoma (UC) prognosis has been investigated by numerous studies. While the majority report worse UC outcomes in patients with renal insufficiency, the results between the studies differed wildly. To enable us to better estimate the prognostic value of renal insufficiency on UC, we performed a systematic review and meta-analysis based on the published literature. A total of 16 studies which involved 5,232 patients with UC, investigated the relationship between pre-operative renal insufficiency and disease prognosis. Estimates of combined hazard ratio (HR) for bladder urothelial carcinoma recurrence, cancer-specific survival (CSS) and overall survival (OS) were 1.65 (95% CI, 1.11–2.19), 1.59 (95% CI, 1.14–2.05) and 1.45 (95% CI, 1.19–1.71), respectively; and for upper urinary tract urothelial carcinoma recurrence, CSS and OS were 2.27 (95% CI, 1.42–3.12), 1.02 (95% CI, 0.47–1.57) and 1.52 (95% CI, 1.05–1.99), respectively. Our results indicate that UC patients with pre-operative renal insufficiency tend to have higher recurrence rates and poorer survival compared to those with clinically normal renal function, thus renal function should be closely monitored in these patients. The impact of intervention for renal insufficiency on the prognosis of UC needs to be further studied.

A systematic review and individual patient data meta-analysis of prognostic factors for foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS)

BACKGROUND

Annual foot risk assessment of people with diabetes is recommended in national and international clinical guidelines. At present, these are consensus based and use only a proportion of the available evidence.

OBJECTIVES

We undertook a systematic review of individual patient data (IPD) to identify the most highly prognostic factors for foot ulceration (i.e. symptoms, signs, diagnostic tests) in people with diabetes.

DATA SOURCES

Studies were identified from searches of MEDLINE and EMBASE.

REVIEW METHODS

The electronic search strategies for MEDLINE and EMBASE databases created during an aggregate systematic review of predictive factors for foot ulceration in diabetes were updated and rerun to January 2013. One reviewer applied the IPD review eligibility criteria to the full-text articles of the studies identified in our literature search and also to all studies excluded from our aggregate systematic review to ensure that we did not miss eligible IPD. A second reviewer applied the eligibility criteria to a 10% random sample of the abstract search yield to check that no relevant material was missed. This review includes exposure variables (risk factors) only from individuals who were free of foot ulceration at the time of study entry and who had a diagnosis of diabetes mellitus (either type 1 or type 2). The outcome variable was incident ulceration.

RESULTS

Our search identified 16 cohort studies and we obtained anonymised IPD for 10. These data were collected from more than 16,000 people with diabetes worldwide and reanalysed by us. One data set was kept for independent validation. The data sets contributing IPD covered a range of temporal, geographical and clinical settings. We therefore selected random-effects meta-analysis, which assumes not that all the estimates from each study are estimates of the same underlying true value, but rather that the estimates belong to the same distribution. We selected candidate variables for meta-analysis using specific criteria. After univariate meta-analyses, the most clinically important predictors were identified by an international steering committee for inclusion in the primary, multivariable meta-analysis. Age, sex, duration of diabetes, monofilaments and pulses were considered most prognostically important. Meta-analyses based on data from the entire IPD population found that an inability to feel a 10-g monofilament [odds ratio (OR) 3.184, 95% confidence interval (CI) 2.654 to 3.82], at least one absent pedal pulse (OR 1.968, 95% CI 1.624 to 2.386), a longer duration of a diagnosis of diabetes (OR 1.024, 95% CI 1.011 to 1.036) and a previous history of ulceration (OR 6.589, 95% CI 2.488 to 17.45) were all predictive of risk. Female sex was protective (OR 0.743, 95% CI 0.598 to 0.922).

LIMITATIONS

It was not possible to perform a meta-analysis using a one-step approach because we were unable to procure copies of one of the data sets and instead accessed data via Safe Haven.

CONCLUSIONS

The findings from this review identify risk assessment procedures that can reliably inform national and international diabetes clinical guideline foot risk assessment procedures. The evidence from a large sample of patients in worldwide settings show that the use of a 10-g monofilament or one absent pedal pulse will identify those at moderate or intermediate risk of foot ulceration, and a history of foot ulcers or lower-extremity amputation is sufficient to identify those at high risk. We propose the development of a clinical prediction rule (CPR) from our existing model using the following predictor variables: insensitivity to a 10-g monofilament, absent pedal pulses and a history of ulceration or lower-extremities amputations. This CPR could replace the many tests, signs and symptoms that patients currently have measured using equipment that is either costly or difficult to use.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42011001841.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

A 'test and treat' strategy for elevated wound protease activity for healing in venous leg ulcers

BACKGROUND

Venous leg ulcers are a common and recurring type of complex wound. They can be painful, malodorous, prone to infection and slow to heal. Standard treatment includes compression therapy and a dressing. The use of protease-modulating treatments for venous leg ulcers is increasing. These treatments are based on some evidence that a proportion of slow to heal ulcers have elevated protease activity in the wound. Point-of-care tests which aim to detect elevated protease activity are now available. A 'test and treat' strategy involves testing for elevated proteases and then using protease-modulating treatments in ulcers which show elevated protease levels.

OBJECTIVES

To determine the effects on venous leg ulcer healing of a 'test and treat' strategy involving detection of high levels of wound protease activity and treatment with protease-modulating therapies, compared with alternative treatment strategies such as using the same treatment for all participants or using a different method of treatment selection.

SEARCH METHODS

We searched the following electronic databases to identify reports of relevant randomised clinical trials: The Cochrane Wounds Group Specialised Register (January 2016), the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library) Issue 12, 2015); Ovid MEDLINE (1946 to January 2016); Ovid MEDLINE (In-Process & Other Non-Indexed Citations January 2016); Ovid EMBASE (1974 to January 2016); EBSCO CINAHL (1937 to January 2016). We also searched three clinical trials registers, reference lists and the websites of regulatory agencies. There were no restrictions with respect to language, date of publication or study setting.

SELECTION CRITERIA

Published or unpublished RCTs which assessed a test and treat strategy for elevated protease activity in venous leg ulcers in adults compared with an alternative treatment strategy. The test and treat strategy needed to be the only systematic difference between the groups.

DATA COLLECTION AND ANALYSIS

Two review authors independently performed study selection; we planned that two authors would also assess risk of bias and extract data.

MAIN RESULTS

We did not identify any studies which met the inclusion criteria for this review. We identified one ongoing study; it was unclear whether this would be eligible for inclusion.

AUTHORS' CONCLUSIONS

Currently there is no randomised evidence on the impact of a test and treat policy for protease levels on outcomes in people with venous leg ulcers.

Methodological issues and recommendations for systematic reviews of prognostic studies: an example from cardiovascular disease

BACKGROUND

Prognostic factors are associated with the risk of future health outcomes in individuals with a particular health condition. The prognostic ability of such factors is increasingly being assessed in both primary research and systematic reviews. Systematic review methodology in this area is continuing to evolve, reflected in variable approaches to key methodological aspects. The aim of this article was to (i) explore and compare the methodology of systematic reviews of prognostic factors undertaken for the same clinical question, (ii) to discuss implications for review findings, and (iii) to present recommendations on what might be considered to be 'good practice' approaches.

METHODS

The sample was comprised of eight systematic reviews addressing the same clinical question, namely whether 'aspirin resistance' (a potential prognostic factor) has prognostic utility relative to future vascular events in patients on aspirin therapy for secondary prevention. A detailed comparison of methods around study identification, study selection, quality assessment, approaches to analysis, and reporting of findings was undertaken and the implications discussed. These were summarised into key considerations that may be transferable to future systematic reviews of prognostic factors.

RESULTS

Across systematic reviews addressing the same clinical question, there were considerable differences in the numbers of studies identified and overlap between included studies, which could only partially be explained by different study eligibility criteria. Incomplete reporting and differences in terminology within primary studies hampered study identification and selection process across reviews. Quality assessment was highly variable and only one systematic review considered a checklist for studies of prognostic questions. There was inconsistency between reviews in approaches towards analysis, synthesis, addressing heterogeneity and reporting of results.

CONCLUSIONS

Different methodological approaches may ultimately affect the findings and interpretation of systematic reviews of prognostic research, with implications for clinical decision-making.

Prediction of gastric cancer development by serum pepsinogen test and Helicobacter pylori seropositivity in Eastern Asians: a systematic review and meta-analysis

BACKGROUND

To identify high-risk groups for gastric cancer in presumptively healthy populations, several studies have investigated the predictive ability of the pepsinogen test, H. Pylori antibodies, and a risk-prediction model based on these two tests. To investigate whether these tests accurately predict gastric cancer development, we conducted a systematic review and meta-analysis.

METHODS

PubMed and other electronic databases were searched for cohort studies published in English or Japanese from January 1985 through December 2013. Six reviewers identified eligible studies, and at least two investigators extracted data on population and study-design characteristics, quality items, and outcomes of interest. Meta-analyses were performed on non-overlapping studies.

RESULTS

Nine prospective cohorts from Eastern Asia reported in 12 publications, including 33,741 asymptomatic middle-aged participants of gastric cancer screening, were eligible. For discriminating between asymptomatic adults at high and low risk of gastric cancer, the pepsinogen test (summary hazard ratio [HR], 3.5; 95% confidence interval [CI], 2.7-4.7; I2 = 0%) and H. pylori antibodies (summary HR, 3.2; 95% CI, 2.0-5.2; I2 = 0%) were statistically significant predictors as standalone tests. Although the risk-prediction model was in general moderately accurate in separating asymptomatic adults into four risk groups (summary c-statistic, 0.71; 95% CI: 0.68-0.73; I2 = 7%), calibration seemed to be poor. The study validity was generally limited.

CONCLUSIONS

The serum pepsinogen test, H. pylori antibodies, and the four-risk-group model for predicting gastric cancer development seem to have the potential to stratify middle-aged presumptively healthy adults. Future research needs to focus on comparative studies to evaluate the impact of screening programs adopting these tests. Also, validation, preferably with model updating, is necessary to see whether the current model performance is transferable to different populations.

Protocol for a systematic review and individual patient data meta-analysis of prognostic factors of foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS)

Background

Diabetes–related lower limb amputations are associated with considerable morbidity and mortality and are usually preceded by foot ulceration. The available systematic reviews of aggregate data are compromised because the primary studies report both adjusted and unadjusted estimates. As adjusted meta-analyses of aggregate data can be challenging, the best way to standardise the analytical approach is to conduct a meta-analysis based on individual patient data (IPD).

There are however many challenges and fundamental methodological omissions are common; protocols are rare and the assessment of the risk of bias arising from the conduct of individual studies is frequently not performed, largely because of the absence of widely agreed criteria for assessing the risk of bias in this type of review. In this protocol we propose key methodological approaches to underpin our IPD systematic review of prognostic factors of foot ulceration in diabetes.

Review questions;

1. What are the most highly prognostic factors for foot ulceration (i.e. symptoms, signs, diagnostic tests) in people with diabetes?

2. Can the data from each study be adjusted for a consistent set of adjustment factors?

3. Does the model accuracy change when patient populations are stratified according to demographic and/or clinical characteristics?

Methods

MEDLINE and EMBASE databases from their inception until early 2012 were searched and the corresponding authors of all eligible primary studies invited to contribute their raw data. We developed relevant quality assurance items likely to identify occasions when study validity may have been compromised from several sources. A confidentiality agreement, arrangements for communication and reporting as well as ethical and governance considerations are explained.

We have agreement from the corresponding authors of all studies which meet the eligibility criteria and they collectively possess data from more than 17000 patients. We propose, as a provisional analysis plan, to use a multi-level mixed model, using “study” as one of the levels. Such a model can also allow for the within-patient clustering that occurs if a patient contributes data from both feet, although to aid interpretation, we prefer to use patients rather than feet as the unit of analysis. We intend to only attempt this analysis if the results of the investigation of heterogeneity do not rule it out and the model diagnostics are acceptable.

Discussion

This review is central to the development of a global evidence-based strategy for the risk assessment of the foot in patients with diabetes, ensuring future recommendations are valid and can reliably inform international clinical guidelines.

From differences in means between cases and controls to risk stratification: a business plan for biomarker development

Researchers developing biomarkers for early detection can determine the potential for clinical benefit at early stages of development. We provide the theoretical background showing the quantitative connection between biomarker levels in cases and controls and clinically meaningful risk measures, as well as a spreadsheet for researchers to use in their own research. We provide researchers with tools to decide whether a test is useful, whether it needs technical improvement, whether it may work only in specific populations, or whether any further development is futile. The methods described here apply to any method that aims to estimate risk of disease based on biomarkers, clinical tests, genetics, environment, or behavior.

SIGNIFICANCE

Many efforts go into futile biomarker development and premature clinical testing. In many instances, predictions for translational success or failure can be made early, simply based on critical analysis of case–control data. Our article presents well-established theory in a form that can be appreciated by biomarker researchers. Furthermore, we provide an interactive spreadsheet that links biomarker performance with specific disease characteristics to evaluate the promise of biomarker candidates at an early stage.