Individual participant data meta-analysis of prognostic factor studies: state of the art?

Two-stage or not two-stage? That is the question for IPD meta-analysis projects

Individual participant data meta-analysis (IPDMA) projects obtain, check, harmonise and synthesise raw data from multiple studies. When undertaking the meta-analysis, researchers must decide between a two-stage or a one-stage approach. In a two-stage approach, the IPD are first analysed separately within each study to obtain aggregate data (e.g., treatment effect estimates and standard errors); then, in the second stage, these aggregate data are combined in a standard meta-analysis model (e.g., common-effect or random-effects). In a one-stage approach, the IPD from all studies are analysed in a single step using an appropriate model that accounts for clustering of participants within studies and, potentially, between-study heterogeneity (e.g., a general or generalised linear mixed model). The best approach to take is debated in the literature, and so here we provide clearer guidance for a broad audience. Both approaches are important tools for IPDMA researchers and neither are a panacea. If most studies in the IPDMA are small (few participants or events), a one-stage approach is recommended due to using a more exact likelihood. However, in other situations, researchers can choose either approach, carefully following best practice. Some previous claims recommending to always use a one-stage approach are misleading, and the two-stage approach will often suffice for most researchers. When differences do arise between the two approaches, often it is caused by researchers using different modelling assumptions or estimation methods, rather than using one or two stages per se.

Perioperative Factors Associated With Postoperative Delirium in Patients Undergoing Noncardiac Surgery: An Individual Patient Data Meta-Analysis

Importance

Postoperative delirium (POD) is a common and serious complication after surgery. Various predisposing factors are associated with POD, but their magnitude and importance using an individual patient data (IPD) meta-analysis have not been assessed.

Objective

To identify perioperative factors associated with POD and assess their relative prognostic value among adults undergoing noncardiac surgery.

Data Sources

MEDLINE, EMBASE, and CINAHL from inception to May 2020.

Study Selection

Studies were included that (1) enrolled adult patients undergoing noncardiac surgery, (2) assessed perioperative risk factors for POD, and (3) measured the incidence of delirium (measured using a validated approach). Data were analyzed in 2020.

Data Extraction and Synthesis

Individual patient data were pooled from 21 studies and 1-stage meta-analysis was performed using multilevel mixed-effects logistic regression after a multivariable imputation via chained equations model to impute missing data.

Main Outcomes and Measures

The end point of interest was POD diagnosed up to 10 days after a procedure. A wide range of perioperative risk factors was considered as potentially associated with POD.

Results

A total of 192 studies met the eligibility criteria, and IPD were acquired from 21 studies that enrolled 8382 patients. Almost 1 in 5 patients developed POD (18%), and an increased risk of POD was associated with American Society of Anesthesiologists (ASA) status 4 (odds ratio [OR], 2.43; 95% CI, 1.42-4.14), older age (OR for 65-85 years, 2.67; 95% CI, 2.16-3.29; OR for >85 years, 6.24; 95% CI, 4.65-8.37), low body mass index (OR for body mass index <18.5, 2.25; 95% CI, 1.64-3.09), history of delirium (OR, 3.9; 95% CI, 2.69-5.66), preoperative cognitive impairment (OR, 3.99; 95% CI, 2.94-5.43), and preoperative C-reactive protein levels (OR for 5-10 mg/dL, 2.35; 95% CI, 1.59-3.50; OR for >10 mg/dL, 3.56; 95% CI, 2.46-5.17). Completing a college degree or higher was associated with a decreased likelihood of developing POD (OR 0.45; 95% CI, 0.28-0.72).

Conclusions and Relevance

In this systematic review and meta-analysis of individual patient data, several important factors associated with POD were found that may help identify patients at high risk and may have utility in clinical practice to inform patients and caregivers about the expected risk of developing delirium after surgery. Future studies should explore strategies to reduce delirium after surgery.

Role of factor VIII, IX, and XI in venous thrombosis recurrence risk in adults and children: A systematic review

Background

Predicting recurrent venous thromboembolic events (VTEs) is challenging in clinical practice for both adults and children, but it is relevant for clinical management. Identifying laboratory risk factors for VTE recurrence may aid in clinical decision-making.

Objective

The goal of this systematic review is to investigate the predictive role of FVIII, IX, or XI in recurrent VTE in adult and pediatric patients with a first VTE.

Methods

A systematic review of the published literature was conducted in databases MEDLINE In-Process, Other Nonindexed Citations, MEDLINE Epub Ahead of Print, EMBASE Classic + EMBASE (OvidSP), and Cochrane (Wiley). We included observational and interventional studies that comprised adults or children with a first VTE, FVIII, FIX, and/or FXI and objectively confirmed VTE recurrence. The quality in prognosis studies tool was used to assess the risk of bias.

Results

We identified 2177 unique studies, of which 19 were included (18 for adults and 1 for children). The risk of bias was overall low to moderate. The studies were heterogenous with regards to population (provoked/unprovoked primary VTE), exposure (type of assay and cut-off values), and statistical analysis results (measures of association and modeling strategy). In adults, contradictory evidence was found for FVIII and FXI as outcome predictors, while no research could establish if FIX predicts VTE recurrence. Data in pediatrics were limited. Given the extensive heterogeneity of the literature, a meta-analysis was not performed.

Conclusions

Overall, there is contradictory evidence that FVIII, FIX, or FXI predict recurrent VTE in adults and children. Addressing heterogeneity is a relevant aspect to consider in future studies investigating prognostic factors for VTE recurrence.

Identifying effect modifiers of CAR-T cell therapeutic efficacy: a systematic review and individual patient data meta-analysis protocol

BACKGROUND

Chimeric antigen receptor T cell therapy (CAR-T) represents a promising and exciting new therapy for hematologic malignancies, where prognosis for relapsed/refractory patients remains poor. Encouraging results from clinical trials have often been tempered by heterogeneity in response to treatment among patients, as well as safety concerns including cytokine release syndrome. The identification of specific patient or treatment-specific factors underlying this heterogeneity may provide the key to the long-term sustainability of this complex and expensive therapy. An individual patient data meta-analysis (IPMDA) may provide potential explanations for the high degree of heterogeneity. Therefore, our objective is to perform a systematic review and IPDMA of CAR-T cell therapy in patients with hematologic malignancies to explore potential effect modifiers of CAR-T cell therapy.

METHODS AND ANALYSIS

We will search MEDLINE, Embase, and the Cochrane Central Register of Controlled Clinical Trials. Studies will be screened in duplicate at the abstract level, then at the full-text level by two independent reviewers. We will include any prospective clinical trial of CAR-T cell therapy in patients with hematologic malignancies. Our primary outcome is complete response, while secondary outcomes of interest include overall response, progression-free survival, overall survival, and safety. IPD will be collected from each included trial and, in the case of missing data, corresponding authors/study sponsors will be contacted. Standard aggregate meta-analyses will be performed, followed by the IPD meta-analysis using a one-stage approach. A modified Institute of Health Economics tool will be used to evaluate the risk of bias of included studies.

ETHICS AND DISSEMINATION

Identifying characteristics that may act as modifiers of CAR-T cell efficacy is of paramount importance and can help shape future clinical trials in the field. Results from this study will be submitted for publication in a peer-reviewed scientific journal, presented at relevant conferences and shared with relevant stakeholders.

Structured reporting to improve transparency of analyses in prognostic marker studies

BACKGROUND

Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted.

METHODS

A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted.

RESULTS

Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance.

CONCLUSIONS

A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research.

A systematic review of blood biomarkers with individual participant data meta-analysis of matrix metalloproteinase-7 in idiopathic pulmonary fibrosis

BACKGROUND

Blood-derived biomarkers have been described extensively as potential prognostic markers in idiopathic pulmonary fibrosis (IPF), but studies have been limited by analyses using data-dependent thresholds, inconsistent adjustment for confounders and an array of end-points, thus often yielding ungeneralisable results. Meta-analysis of individual participant data (IPD) is a powerful tool to overcome these limitations. Through systematic review of blood-derived biomarkers, sufficient studies with measurements of matrix metalloproteinase (MMP)-7 were identified to facilitate standardised analyses of the prognostic potential of this biomarker in IPF.

METHODS

Electronic databases were searched on 12 November 2020 to identify prospective studies reporting outcomes in patients with untreated IPF, stratified according to at least one pre-specified biomarker, measured at either baseline, or change over 3 months. IPD were sought for studies investigating MMP-7 as a prognostic factor. The primary outcome was overall mortality according to standardised MMP-7 z-scores, with a secondary outcome of disease progression in 12 months, all adjusted for age, gender, smoking and baseline forced vital capacity.

RESULTS

IPD was available for nine studies out of 12 identified, reporting outcomes from 1664 participants. Baseline MMP-7 levels were associated with increased mortality risk (adjusted hazard ratio 1.23, 95% CI 1.03-1.48; I2=64.3%) and disease progression (adjusted OR 1.27, 95% CI 1.11-1.46; I2=5.9%). In limited studies, 3-month change in MMP-7 was not associated with outcomes.

CONCLUSION

IPD meta-analysis demonstrated that greater baseline MMP-7 levels were independently associated with an increased risk of poor outcomes in patients with untreated IPF, while short-term changes did not reflect disease progression.

Behavior Problems Following Childhood TBI: The Role of Sex, Age, and Time Since Injury

OBJECTIVE

To examine the frequency of behavioral problems after childhood traumatic brain injury (TBI) and their associations with injury severity, sex, and social environmental factors.

SETTING

Children's hospitals in the Midwestern/Western United States.

PARTICIPANTS

381 boys and 210 girls with moderate (n = 359) and severe (n = 227) TBI, with an average age at injury of 11.7 years (range 0.3-18) who were injured ≤3 years ago.

DESIGN

Secondary data analysis of a multistudy cohort.

MAIN MEASURES

Child Behavior Checklist (CBCL) administered pretreatment.

RESULTS

Thirty-seven percent had borderline/clinical elevations on the CBCL Total Problem Scale, with comparable rates of Internalizing and Externalizing problems (33% and 31%, respectively). Less parental education was associated with higher rates of internalizing, externalizing, and total problems. Time since injury had a linear association with internalizing symptoms, with greater symptoms at longer postinjury intervals. Younger boys had significantly higher levels of oppositional defiant symptoms than girls, whereas older girls had significantly greater attention-deficit hyperactivity disorder symptoms than boys.

CONCLUSIONS

Pediatric TBI is associated with high rates of behavior problems, with lower socioeconomic status predicting substantially elevated risk. Associations of higher levels of internalizing symptoms with greater time since injury highlight the importance of tracking children over time.

Natural history of Lafora disease: a prognostic systematic review and individual participant data meta-analysis

Background

Lafora disease (LD) is a rare fatal autosomal recessive form of progressive myoclonus epilepsy. It affects previously healthy children or adolescents, causing pharmacoresistant epilepsy, myoclonus and severe psychomotor deterioration. This work aims to describe the clinical course of LD and identify predictors of outcome by means of a prognostic systematic review with individual participant data meta-analysis.

Methods

A search was conducted on MEDLINE and Embase with no restrictions on publication date. Only studies reporting genetically confirmed LD cases were included. Kaplan–Meier estimate was used to assess probability of death and loss of autonomy. Univariable and multivariable Cox regression models with mixed effects (clustered survival data) were performed to evaluate prognostic factors.

Results

Seventy-three papers describing 298 genetically confirmed LD cases were selected. Mean age at disease onset was 13.4 years (SD 3.7), with 9.1% aged ≥ 18 years. Overall survival rates in 272 cases were 93% [95% CI 89–96] at 5 years, 62% [95% CI 54–69] at 10 years and 57% [95% CI 49–65] at 15 years. Median survival time was 11 years. The probability of loss of autonomy in 110 cases was 45% [95% CI 36–55] at 5 years, 75% [95% CI 66–84] at 10 years, and 83% [95% CI 74–90] at 15 years. Median loss of autonomy time was 6 years. Asian origin and age at onset < 18 years emerged as negative prognostic factors, while type of mutated gene and symptoms at onset were not related to survival or disability.

Conclusions

This study documented that half of patients survived at least 11 years. The notion of actual survival rate and prognostic factors is crucial to design studies on the effectiveness of upcoming new disease-modifying therapies.

PRIME-IPD SERIES Part 1. The PRIME-IPD tool promoted verification and standardization of study datasets retrieved for IPD meta-analysis

OBJECTIVES

We describe a systematic approach to preparing data in the conduct of Individual Participant Data (IPD) analysis.

STUDY DESIGN AND SETTING

A guidance paper proposing methods for preparing individual participant data for meta-analysis from multiple study sources, developed by consultation of relevant guidance and experts in IPD. We present an example of how these steps were applied in checking data for our own IPD meta analysis (IPD-MA).

RESULTS

We propose five steps of Processing, Replication, Imputation, Merging, and Evaluation to prepare individual participant data for meta-analysis (PRIME-IPD). Using our own IPD-MA as an exemplar, we found that this approach identified missing variables and potential inconsistencies in the data, facilitated the standardization of indicators across studies, confirmed that the correct data were received from investigators, and resulted in a single, verified dataset for IPD-MA.

CONCLUSION

The PRIME-IPD approach can assist researchers to systematically prepare, manage and conduct important quality checks on IPD from multiple studies for meta-analyses. Further testing of this framework in IPD-MA would be useful to refine these steps.

Prognostic factors for adolescent knee pain: an individual participant data meta-analysis of 1281 patients

ABSTRACT

Adolescent knee pain has a propensity for chronicity, impacting physical activity and health into adulthood. The aim of this study is to investigate prognostic factors in adolescents with knee pain using individual participant data (IPD) meta-analysis. Studies were identified through a systematic search and a collaborative group. We included IPD from prospective studies of adolescents (age 10-19 years) with nontraumatic knee pain (13 studies and 1516 adolescents with 1281 unique participants). Primary outcomes were pain intensity and function (Knee Injury and Osteoarthritis Outcome Score "Sport/Rec" subscale). Primary endpoint was 12 months. Risk of bias was appraised with Quality in Prognosis Studies tool. Harmonised IPD was analysed by multilevel modelling. Fifty-one percent reported knee pain after 12-months. Lower baseline pain frequency was associated with lower pain intensity at 12 months ("less than weekly"; 12 (95% confidence interval [CI] 7-17) and "monthly"; 15 (95% CI 9-22) points lower on a 100-point pain scale, compared with "almost daily pain"). Other factors most strongly associated with 1-year pain prognosis were lower quality of life (30, 95% CI 19-42 points per unit change in the EQ5D index score), female sex (8 points, 95% CI 4-12 higher compared with males), and bilateral pain (7, 95% CI 1-13 points higher pain). Similar factors were associated with function. Body mass index, pain sensitivity, and knee strength were not associated with prognosis of pain or function. Adolescent knee pain is associated with clinically relevant long-term pain and functional deficits. Self-reported characteristics may help identify those at risk of poor prognosis.

Patient- and Tumour-related Prognostic Factors for Urinary Incontinence After Radical Prostatectomy for Nonmetastatic Prostate Cancer: A Systematic Review and Meta-analysis

CONTEXT

While urinary incontinence (UI) commonly occurs after radical prostatectomy (RP), it is unclear what factors increase the risk of UI development.

OBJECTIVE

To perform a systematic review of patient- and tumour-related prognostic factors for post-RP UI. The primary outcome was UI within 3 mo after RP. Secondary outcomes included UI at 3-12 mo and ≥12 mo after RP.

EVIDENCE ACQUISITION

Databases including Medline, EMBASE, and CENTRAL were searched between January 1990 and May 2020. All studies reporting patient- and tumour-related prognostic factors in univariable or multivariable analyses were included. Surgical factors were excluded. Risk of bias (RoB) and confounding assessments were performed using the Quality In Prognosis Studies (QUIPS) tool. Random-effects meta-analyses were performed for all prognostic factor, where possible.

EVIDENCE SYNTHESIS

A total of 119 studies (5 randomised controlled trials, 24 prospective, 88 retrospective, and 2 case-control studies) with 131 379 patients were included. RoB was high for study participation and confounding; moderate to high for statistical analysis, study attrition, and prognostic factor measurement; and low for outcome measurements. Significant prognostic factors for postoperative UI within 3 mo after RP were age (odds ratio [OR] per yearly increase 1.04, 95% confidence interval [CI] 1.03-1.05), membranous urethral length (MUL; OR per 1-mm increase 0.81, 95% CI 0.74-0.88), prostate volume (PV; OR per 1-ml increase 1.005, 95% CI 1.000-1.011), and Charlson comorbidity index (CCI; OR 1.28, 95% CI 1.09-1.50).

CONCLUSIONS

Increasing age, shorter MUL, greater PV, and higher CCI are independent prognostic factors for UI within 3 mo after RP, with all except CCI remaining prognostic at 3-12 mo.

PATIENT SUMMARY

We reviewed the literature to identify patient and disease factors associated with urinary incontinence after surgery for prostate cancer. We found increasing age, larger prostate volume, shorter length of a section of the urethra (membranous urethra), and lower fitness were associated with worse urinary incontinence for the first 3 mo after surgery, with all except lower fitness remaining predictive at 3-12 mo.

The methodological quality of individual participant data meta-analysis on intervention effects: systematic review

OBJECTIVE

To assess the methodological quality of individual participant data (IPD) meta-analysis and to identify areas for improvement.

DESIGN

Systematic review.

DATA SOURCES

Medline, Embase, and Cochrane Database of Systematic Reviews.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Systematic reviews with IPD meta-analyses of randomised controlled trials on intervention effects published in English.

RESULTS

323 IPD meta-analyses covering 21 clinical areas and published between 1991 and 2019 were included: 270 (84%) were non-Cochrane reviews and 269 (84%) were published in journals with a high impact factor (top quarter). The IPD meta-analyses showed low compliance in using a satisfactory technique to assess the risk of bias of the included randomised controlled trials (43%, 95% confidence interval 38% to 48%), accounting for risk of bias when interpreting results (40%, 34% to 45%), providing a list of excluded studies with justifications (32%, 27% to 37%), establishing an a priori protocol (31%, 26% to 36%), prespecifying methods for assessing both the overall effects (44%, 39% to 50%) and the participant-intervention interactions (31%, 26% to 36%), assessing and considering the potential of publication bias (31%, 26% to 36%), and conducting a comprehensive literature search (19%, 15% to 23%). Up to 126 (39%) IPD meta-analyses failed to obtain IPD from 90% or more of eligible participants or trials, among which only 60 (48%) provided reasons and 21 (17%) undertook certain strategies to account for the unavailable IPD.

CONCLUSIONS

The methodological quality of IPD meta-analyses is unsatisfactory. Future IPD meta-analyses need to establish an a priori protocol with prespecified data syntheses plan, comprehensively search the literature, critically appraise included randomised controlled trials with appropriate technique, account for risk of bias during data analyses and interpretation, and account for unavailable IPD.

Arm Based on LEg blood pressures (ABLE-BP): can systolic leg blood pressure measurements predict systolic brachial blood pressure? Protocol for an individual participant data meta-analysis from the INTERPRESS-IPD Collaboration

INTRODUCTION

Blood pressure (BP) is normally measured on the upper arm, and guidelines for the diagnosis and treatment of high BP are based on such measurements. Leg BP measurement can be an alternative when brachial BP measurement is impractical, due to injury or disability. Limited data exist to guide interpretation of leg BP values for hypertension management; study-level systematic review findings suggest that systolic BP (SBP) is 17 mm Hg higher in the leg than the arm. However, uncertainty remains about the applicability of this figure in clinical practice due to substantial heterogeneity.

AIMS

To examine the relationship between arm and leg SBP, develop and validate a multivariable model predicting arm SBP from leg SBP and investigate the prognostic association between leg SBP and cardiovascular disease and mortality.

METHODS AND ANALYSIS

Individual participant data (IPD) meta-analyses using arm and leg SBP measurements for 33 710 individuals from 14 studies within the Inter-arm blood pressure difference IPD (INTERPRESS-IPD) Collaboration. We will explore cross-sectional relationships between arm and leg SBP using hierarchical linear regression with participants nested by study, in multivariable models. Prognostic models will be derived for all-cause and cardiovascular mortality and cardiovascular events.

ETHICS AND DISSEMINATION

Data originate from studies with prior ethical approval and consent, and data sharing agreements are in place-no further approvals are required to undertake the secondary analyses proposed in this protocol. Findings will be published in peer-reviewed journal articles and presented at conferences. A comprehensive dissemination strategy is in place, integrated with patient and public involvement.

PROSPERO REGISTRATION NUMBER

CRD42015031227.

Preoperative prognostic factors associated with postoperative delirium in older people undergoing surgery: protocol for a systematic review and individual patient data meta-analysis

BACKGROUND

Early identification of patients at risk for postoperative delirium is essential because adequate well-timed interventions could reduce the occurrence of delirium and the related detrimental outcomes.

METHODS

We will conduct a systematic review and individual patient data (IPD) meta-analysis of prognostic studies evaluating the predictive value of risk factors associated with an increased risk of postoperative delirium in elderly patients undergoing elective surgery. We will identify eligible studies through systematic search of MEDLINE, EMBASE, and CINAHL from their inception to May 2020. Eligible studies will enroll older adults (≥ 50 years) undergoing elective surgery and assess pre-operative prognostic risk factors for delirium and incidence of delirium measured by a trained individual using a validated delirium assessment tool. Pairs of reviewers will, independently and in duplicate, screen titles and abstracts of identified citations, review the full texts of potentially eligible studies. We will contact chief investigators of eligible studies requesting to share the IPD to a secured repository. We will use one-stage approach for IPD meta-analysis and will assess certainty of evidence using the GRADE approach.

DISCUSSION

Since we are using existing anonymized data, ethical approval is not required for this study. Our results can be used to guide clinical decisions about the most efficient way to prevent postoperative delirium in elderly patients.

SYSTEMATIC REVIEW REGISTRATION

CRD42020171366 .

A non-linear optimisation method to extract summary statistics from Kaplan-Meier survival plots using the published P value

Background

Meta-analyses of studies evaluating survival (time-to-event) outcomes are a powerful technique to assess the strength of evidence for a given disease or treatment. However, these studies rely on the adequate reporting of summary statistics in the source articles to facilitate further analysis. Unfortunately, many studies, especially within the field of prognostic research do not report such statistics, making secondary analyses challenging. Consequently, methods have been developed to infer missing statistics from the commonly published Kaplan-Meier (KM) plots but are liable to error especially when the published number at risk is not included.

Methods

We therefore developed a method using non-linear optimisation (nlopt) that only requires the KM plot and the commonly published P value to better estimate the underlying censoring pattern. We use this information to then calculate the natural logarithm of the hazard ratio (ln (HR)) and its variance (var) ln (HR), statistics important for meta-analyses.

Results

We compared this method to the Parmar method which also does not require the number at risk to be published. In a validation set consisting of 13 KM studies, a statistically significant improvement in calculating ln (HR) when using an exact P value was obtained (mean absolute error 0.014 vs 0.077, P = 0.003). Thus, when the true HR has a value of 1.5, inference of the HR using the proposed method would set limits between 1.49/1.52, an improvement of the 1.39/1.62 limits obtained using the Parmar method. We also used Monte Carlo simulations to establish recommendations for the number and positioning of points required for the method.

Conclusion

The proposed non-linear optimisation method is an improvement on the existing method when only a KM plot and P value are included and as such will enhance the accuracy of meta-analyses performed for studies analysing time-to-event outcomes. The nlopt source code is available, as is a simple-to-use web implementation of the method.

Prediction of stillbirth: an umbrella review of evaluation of prognostic variables

BACKGROUND

Stillbirth accounts for over 2 million deaths a year worldwide and rates remains stubbornly high. Multivariable prediction models may be key to individualised monitoring, intervention or early birth in pregnancy to prevent stillbirth.

OBJECTIVES

To collate and evaluate systematic reviews of factors associated with stillbirth in order to identify variables relevant to prediction model development.

SEARCH STRATEGY

MEDLINE, Embase, DARE and Cochrane Library databases and reference lists were searched up to November 2019.

SELECTION CRITERIA

We included systematic reviews of association of individual variables with stillbirth without language restriction.

DATA COLLECTION AND ANALYSIS

Abstract screening and data extraction were conducted in duplicate. Methodological quality was assessed using AMSTAR and QUIPS criteria. The evidence supporting association with each variable was graded.

RESULTS

The search identified 1198 citations. Sixty-nine systematic reviews reporting 64 variables were included. The most frequently reported were maternal age (n = 5), body mass index (n = 6) and maternal diabetes (n = 5). Uterine artery Doppler appeared to have the best performance of any single test for stillbirth. The strongest evidence of association was for nulliparity and pre-existing hypertension.

CONCLUSION

We have identified variables relevant to the development of prediction models for stillbirth. Age, parity and prior adverse pregnancy outcomes had a more convincing association than the best performing tests, which were PAPP-A, PlGF and UtAD. The evidence was limited by high heterogeneity and lack of data on intervention bias.

TWEETABLE ABSTRACT

Review shows key predictors for use in developing models predicting stillbirth include age, prior pregnancy outcome and PAPP-A, PLGF and Uterine artery Doppler.

Development of a Prognostic Factor Prediction Model in Patients with Musculoskeletal Pain Treated with Homeopathy: An Individual Patient Data Meta-Analysis of Three Randomized Clinical Trials

BACKGROUND

Prognostic factor research methodology has not yet been applied to randomized clinical trial data of homeopathic medicines.

OBJECTIVES

To investigate the principle of individualization in homeopathy by developing a prognostic factor prediction model.

METHOD

A pooled, in-dividual patient data meta-analysis of 3 randomized trials -investigating the efficacy of a homeopathic gel (Spiroflor SRL®) containing Rhus toxicodendron as a key ingredient in osteoarthritis of the knee and acute low back pain. The prognostic value of a predefined set of 5 typical R. toxicodendron symptoms was investigated by assessing treatment-by-symptom interactions on pain as an outcome measure in a regression model.

RESULTS

The pooled dataset consisted of 284 patients in the Spiroflor SRL group and 275 patients in the control group. Adjusted for pain at baseline, a statistically significant effect modification for the symptoms "numbness or tingling of the affected part" (+2.0 mm VAS; p = 0.02), "amelioration by movement" (-5.6 mm VAS; p = 0.01), and "amelioration of pain by local heat" (+7.0 mm VAS; p = 0.02) was found.

CONCLUSIONS

Investigating aspects of treatment individualization in homeopathy using randomized trial data and standard meta-analytical techniques is possible. The symptom amelioration by local heat is of possible value as a homeopathic symptom (prognostic factor) predicting an increased likelihood of pain relief following treatment with the homeopathic product.

Obtaining and managing data sets for individual participant data meta-analysis: scoping review and practical guide

BACKGROUND

Shifts in data sharing policy have increased researchers' access to individual participant data (IPD) from clinical studies. Simultaneously the number of IPD meta-analyses (IPDMAs) is increasing. However, rates of data retrieval have not improved. Our goal was to describe the challenges of retrieving IPD for an IPDMA and provide practical guidance on obtaining and managing datasets based on a review of the literature and practical examples and observations.

METHODS

We systematically searched MEDLINE, Embase, and the Cochrane Library, until January 2019, to identify publications focused on strategies to obtain IPD. In addition, we searched pharmaceutical websites and contacted industry organizations for supplemental information pertaining to recent advances in industry policy and practice. Finally, we documented setbacks and solutions encountered while completing a comprehensive IPDMA and drew on previous experiences related to seeking and using IPD.

RESULTS

Our scoping review identified 16 articles directly relevant for the conduct of IPDMAs. We present short descriptions of these articles alongside overviews of IPD sharing policies and procedures of pharmaceutical companies which display certification of Principles for Responsible Clinical Trial Data Sharing via Pharmaceutical Research and Manufacturers of America or European Federation of Pharmaceutical Industries and Associations websites. Advances in data sharing policy and practice affected the way in which data is requested, obtained, stored and analyzed. For our IPDMA it took 6.5 years to collect and analyze relevant IPD and navigate additional administrative barriers. Delays in obtaining data were largely due to challenges in communication with study sponsors, frequent changes in data sharing policies of study sponsors, and the requirement for a diverse skillset related to research, administrative, statistical and legal issues.

CONCLUSIONS

Knowledge of current data sharing practices and platforms as well as anticipation of necessary tasks and potential obstacles may reduce time and resources required for obtaining and managing data for an IPDMA. Sufficient project funding and timeline flexibility are pre-requisites for successful collection and analysis of IPD. IPDMA researchers must acknowledge the additional and unexpected responsibility they are placing on corresponding study authors or data sharing administrators and should offer assistance in readying data for sharing.

One-stage individual participant data meta-analysis models for continuous and binary outcomes: Comparison of treatment coding options and estimation methods

A one-stage individual participant data (IPD) meta-analysis synthesizes IPD from multiple studies using a general or generalized linear mixed model. This produces summary results (eg, about treatment effect) in a single step, whilst accounting for clustering of participants within studies (via a stratified study intercept, or random study intercepts) and between-study heterogeneity (via random treatment effects). We use simulation to evaluate the performance of restricted maximum likelihood (REML) and maximum likelihood (ML) estimation of one-stage IPD meta-analysis models for synthesizing randomized trials with continuous or binary outcomes. Three key findings are identified. First, for ML or REML estimation of stratified intercept or random intercepts models, a t-distribution based approach generally improves coverage of confidence intervals for the summary treatment effect, compared with a z-based approach. Second, when using ML estimation of a one-stage model with a stratified intercept, the treatment variable should be coded using "study-specific centering" (ie, 1/0 minus the study-specific proportion of participants in the treatment group), as this reduces the bias in the between-study variance estimate (compared with 1/0 and other coding options). Third, REML estimation reduces downward bias in between-study variance estimates compared with ML estimation, and does not depend on the treatment variable coding; for binary outcomes, this requires REML estimation of the pseudo-likelihood, although this may not be stable in some situations (eg, when data are sparse). Two applied examples are used to illustrate the findings.

RELEASE: a protocol for a systematic review based, individual participant data, meta- and network meta-analysis, of complex speech-language therapy interventions for stroke-related aphasia

BACKGROUND

Speech and language therapy (SLT) benefits people with aphasia following stroke. Group level summary statistics from randomised controlled trials hinder exploration of highly complex SLT interventions and a clinically relevant heterogeneous population. Creating a database of individual participant data (IPD) for people with aphasia aims to allow exploration of individual and therapy-related predictors of recovery and prognosis.

AIM

To explore the contribution that individual participant characteristics (including stroke and aphasia profiles) and SLT intervention components make to language recovery following stroke.

METHODS AND PROCEDURES

We will identify eligible IPD datasets (including randomised controlled trials, non-randomised comparison studies, observational studies and registries) and invite their contribution to the database. Where possible, we will use meta- and network meta-analysis to explore language performance after stroke and predictors of recovery as it relates to participants who had no SLT, historical SLT or SLT in the primary research study. We will also examine the components of effective SLT interventions.

OUTCOMES AND RESULTS

Outcomes include changes in measures of functional communication, overall severity of language impairment, auditory comprehension, spoken language (including naming), reading and writing from baseline. Data captured on assessment tools will be collated and transformed to a standardised measure for each of the outcome domains.

CONCLUSION

Our planned systematic-review-based IPD meta- and network meta-analysis is a large scale, international, multidisciplinary and methodologically complex endeavour. It will enable hypotheses to be generated and tested to optimise and inform development of interventions for people with aphasia after stroke.

SYSTEMATIC REVIEW REGISTRATION

The protocol has been registered at the International Prospective Register of Systematic Reviews (PROSPERO; registration number: CRD42018110947).

Pragmatic methods for reviewing exceptionally large bodies of evidence: systematic mapping review and overview of systematic reviews using lung cancer survival as an exemplar

INTRODUCTION

Lung cancer (LC) is the most common cause of cancer death in the world and associated with significant economic burden. We conducted a review of published literature to identify prognostic factors associated with LC survival and determine which may be modifiable and could be targeted to improve outcomes.

METHODS

The exceptionally large volume of LC prognostic research required a new staged approach to reviewing the literature. This comprised an initial mapping review of existing reviews or meta-analyses, based on titles and abstracts, followed by an overview of systematic reviews evaluating factors that independently contribute to lung cancer survival. The overview of reviews was based on full text papers and incorporated a more in-depth assessment of reviews evaluating modifiable factors.

RESULTS

A large volume of published systematic reviews and meta-analyses were identified, but very few focused on modifiable factors for LC survival. Several modifiable factors were identified, which are potential candidates for targeted interventions aiming to improve cancer outcomes. The mapping review included 398 reviews, of which 207 investigated the independent effect of prognostic factors on lung cancer survival. The most frequently evaluated factors were novel biomarkers (86 biomarkers in 138 reviews). Only 15 modifiable factors were investigated in 20 reviews. Those associated with significant survival improvement included normal BMI/less weight loss, good performance status, not smoking/quitting after diagnosis, good pre-treatment quality of life, small gross volume tumour, early-stage tumour, lung resection undertaken by a thoracic/cardiothoracic surgeon, care being discussed by a multidisciplinary team, and timeliness of care.

CONCLUSIONS

The study utilised a novel approach for reviewing an extensive and complicated body of research evidence. It enabled us to address a broad research question and focus on a specific area of priority. The staged approach ensured the review remained relevant to the stakeholders throughout, whilst maintaining the use of objective and transparent methods. It also provided important information on the needs of future research. However, it required extensive planning, management, and ongoing reviewer training.

Evidence synthesis in prognosis research

Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence. We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods.

Prediction of pre-eclampsia: review of reviews

OBJECTIVE

Primary studies and systematic reviews provide estimates of varying accuracy for different factors in the prediction of pre-eclampsia. The aim of this study was to review published systematic reviews to collate evidence on the ability of available tests to predict pre-eclampsia, to identify high-value avenues for future research and to minimize future research waste in this field.

METHODS

MEDLINE, EMBASE and The Cochrane Library including DARE (Database of Abstracts of Reviews of Effects) databases, from database inception to March 2017, and bibliographies of relevant articles were searched, without language restrictions, for systematic reviews and meta-analyses on the prediction of pre-eclampsia. The quality of the included reviews was assessed using the AMSTAR tool and a modified version of the QUIPS tool. We evaluated the comprehensiveness of search, sample size, tests and outcomes evaluated, data synthesis methods, predictive ability estimates, risk of bias related to the population studied, measurement of predictors and outcomes, study attrition and adjustment for confounding.

RESULTS

From 2444 citations identified, 126 reviews were included, reporting on over 90 predictors and 52 prediction models for pre-eclampsia. Around a third (n = 37 (29.4%)) of all reviews investigated solely biochemical markers for predicting pre-eclampsia, 31 (24.6%) investigated genetic associations with pre-eclampsia, 46 (36.5%) reported on clinical characteristics, four (3.2%) evaluated only ultrasound markers and six (4.8%) studied a combination of tests; two (1.6%) additional reviews evaluated primary studies investigating any screening test for pre-eclampsia. Reviews included between two and 265 primary studies, including up to 25 356 688 women in the largest review. Only approximately half (n = 67 (53.2%)) of the reviews assessed the quality of the included studies. There was a high risk of bias in many of the included reviews, particularly in relation to population representativeness and study attrition. Over 80% (n = 106 (84.1%)) summarized the findings using meta-analysis. Thirty-two (25.4%) studies lacked a formal statement on funding. The predictors with the best test performance were body mass index (BMI) > 35 kg/m² , with a specificity of 92% (95% CI, 89-95%) and a sensitivity of 21% (95% CI, 12-31%); BMI > 25 kg/m² , with a specificity of 73% (95% CI, 64-83%) and a sensitivity of 47% (95% CI, 33-61%); first-trimester uterine artery pulsatility index or resistance index > 90^th centile (specificity 93% (95% CI, 90-96%) and sensitivity 26% (95% CI, 23-31%)); placental growth factor (specificity 89% (95% CI, 89-89%) and sensitivity 65% (95% CI, 63-67%)); and placental protein 13 (specificity 88% (95% CI, 87-89%) and sensitivity 37% (95% CI, 33-41%)). No single marker had a test performance suitable for routine clinical use. Models combining markers showed promise, but none had undergone external validation.

CONCLUSIONS

This review of reviews calls into question the need for further aggregate meta-analysis in this area given the large number of published reviews subject to the common limitations of primary predictive studies. Prospective, well-designed studies of predictive markers, preferably randomized intervention studies, and combined through individual-patient data meta-analysis are needed to develop and validate new prediction models to facilitate the prediction of pre-eclampsia and minimize further research waste in this field. Copyright © 2018 ISUOG. Published by John Wiley & Sons Ltd.

Towards more Accessible Precision Medicine: Building a more Transferable Machine Learning Model to Support Prognostic Decisions for Micro- and Macrovascular Complications of Type 2 Diabetes Mellitus

Although machine learning models are increasingly being developed for clinical decision support for patients with type 2 diabetes, the adoption of these models into clinical practice remains limited. Currently, machine learning (ML) models are being constructed on local healthcare systems and are validated internally with no expectation that they would validate externally and thus, are rarely transferrable to a different healthcare system. In this work, we aim to demonstrate that (1) even a complex ML model built on a national cohort can be transferred to two local healthcare systems, (2) while a model constructed on a local healthcare system's cohort is difficult to transfer; (3) we examine the impact of training cohort size on the transferability; and (4) we discuss criteria for external validity. We built a model using our previously published Multi-Task Learning-based methodology on a national cohort extracted from OptumLabs® Data Warehouse and transferred the model to two local healthcare systems (i.e., University of Minnesota Medical Center and Mayo Clinic) for external evaluation. The model remained valid when applied to the local patient populations and performed as well as locally constructed models (concordance: .73-.92), demonstrating transferability. The performance of the locally constructed models reduced substantially when applied to each other's healthcare system (concordance: .62-.90). We believe that our modeling approach, in which a model is learned from a national cohort and is externally validated, produces a transferable model, allowing patients at smaller healthcare systems to benefit from precision medicine.

Online Family Problem-solving Treatment for Pediatric Traumatic Brain Injury

BACKGROUND AND OBJECTIVES

To determine whether online family problem-solving treatment (OFPST) is more effective in improving behavioral outcomes after pediatric traumatic brain injury with increasing time since injury.

METHODS

This was an individual participant data meta-analysis of outcome data from 5 randomized controlled trials of OFPST conducted between 2003 and 2016. We included 359 children ages 5 to 18 years who were hospitalized for moderate-to-severe traumatic brain injury 1 to 24 months earlier. Outcomes, assessed pre- and posttreatment, included parent-reported measures of externalizing, internalizing, and executive function behaviors and social competence.

RESULTS

Participants included 231 boys and 128 girls with an average age at injury of 13.6 years. Time since injury and age at injury moderated OFPST efficacy. For earlier ages and short time since injury, control participants demonstrated better externalizing problem scores than those receiving OFPST (Cohen's d = 0.44; P = .008; n = 295), whereas at older ages and longer time since injury, children receiving OFPST had better scores (Cohen's d = -0.60; P = .002). Children receiving OFPST were rated as having better executive functioning relative to control participants at a later age at injury, with greater effects seen at longer (Cohen's d = -0.66; P = .009; n = 298) than shorter (Cohen's d = -0. 28; P = .028) time since injury.

CONCLUSIONS

OFPST may be more beneficial for older children and when begun after the initial months postinjury. With these findings, we shed light on the optimal application of family problem-solving treatments within the first 2 years after injury.

Examination of Injury, Host, and Social-Environmental Moderators of Online Family Problem Solving Treatment Efficacy for Pediatric Traumatic Brain Injury Using an Individual Participant Data Meta-Analytic Approach

A series of five randomized controlled clinical trials (RCTs) conducted between 2002 and 2015 support the potential efficacy of online family problem-solving treatment (OFPST) in improving both child and parent/family outcomes after pediatric traumatic brain injury (TBI). However, small sample sizes and heterogeneity across individual studies have precluded examination of potentially important moderators. We jointly analyzed individual participant data (IPD) from these five RCTs, involving 359 children and adolescents between the ages of 5 and 18 years, to confirm the role of previously identified moderators (child's age and pre-treatment symptom levels, parental education) and to examine other potential moderators (race, sex, IQ), using IPD meta-analysis. This reanalysis revealed statistically strong evidence that parental education, child age at baseline, IQ, sex, and parental depression level pre-treatment moderated the effect of OFPST on various outcomes. In particular, children of parents with a less than high school education exhibited fewer internalizing problems and better social competence. Children injured at an older age exhibited fewer externalizing behaviors and less executive dysfunction following OFPST. Child IQ moderated the effect of OFPST on social competence, with significantly better competence for children with lower IQ who received OFPST. Lower levels of parental depression followed OFPST among subgroups with lower IQ, boys, and higher parental depression scores at baseline. Our findings indicate that the optimal application of OFPST is likely to involve older children, those with lower IQ scores, or those from families with lower socioeconomic status (SES).

Simulation-based power calculations for planning a two-stage individual participant data meta-analysis

Background

Researchers and funders should consider the statistical power of planned Individual Participant Data (IPD) meta-analysis projects, as they are often time-consuming and costly. We propose simulation-based power calculations utilising a two-stage framework, and illustrate the approach for a planned IPD meta-analysis of randomised trials with continuous outcomes where the aim is to identify treatment-covariate interactions.

Methods

The simulation approach has four steps: (i) specify an underlying (data generating) statistical model for trials in the IPD meta-analysis; (ii) use readily available information (e.g. from publications) and prior knowledge (e.g. number of studies promising IPD) to specify model parameter values (e.g. control group mean, intervention effect, treatment-covariate interaction); (iii) simulate an IPD meta-analysis dataset of a particular size from the model, and apply a two-stage IPD meta-analysis to obtain the summary estimate of interest (e.g. interaction effect) and its associated p-value; (iv) repeat the previous step (e.g. thousands of times), then estimate the power to detect a genuine effect by the proportion of summary estimates with a significant p-value.

Results

In a planned IPD meta-analysis of lifestyle interventions to reduce weight gain in pregnancy, 14 trials (1183 patients) promised their IPD to examine a treatment-BMI interaction (i.e. whether baseline BMI modifies intervention effect on weight gain). Using our simulation-based approach, a two-stage IPD meta-analysis has < 60% power to detect a reduction of 1 kg weight gain for a 10-unit increase in BMI. Additional IPD from ten other published trials (containing 1761 patients) would improve power to over 80%, but only if a fixed-effect meta-analysis was appropriate. Pre-specified adjustment for prognostic factors would increase power further. Incorrect dichotomisation of BMI would reduce power by over 20%, similar to immediately throwing away IPD from ten trials.

Conclusions

Simulation-based power calculations could inform the planning and funding of IPD projects, and should be used routinely.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0492-z) contains supplementary material, which is available to authorized users.

Depression during first episode psychosis and subsequent suicide risk: A systematic review and meta-analysis of longitudinal studies

BACKGROUND

Evidence suggests first episode psychosis (FEP) is associated with suicide, and the influence of depression on suicidal behaviour in cross sectional studies is clear. However the influence of depression during FEP on longer-term mortality is not certain. Existing evidence was synthesised to understand the influence of depressive symptoms during FEP on subsequent suicidal behaviour.

METHODS

Medline, Embase, PsycINFO, Cochrane Library, Web of Science, OpenGrey, and NICE Evidence were searched from inception to Jan 25, 2017. Longitudinal observational studies assessing the relationship between depressive symptoms during FEP with a measure of s at a specified follow-up time were included. Summary estimates were extracted. The Downs and Black Instrument was used to appraise study quality. Odds ratio (OR) of suicidal behaviour were calculated using random effects meta-analyses. The study protocol was registered with PROSPERO (CRD42017055881).

RESULTS

Of 4210 articles found, 23 fulfilled eligibility criteria. 13 were included in meta-analysis (n=3002). 428 participants demonstrated suicidal behaviour in the study periods. Odds of suicidal behaviour during follow-up were significantly higher among patients with depressive symptoms during FEP compared to those without (OR=1.59, 95% CI 1.14-2.21; I2=50.0%, p=0.02). Meta-regression demonstrated no evidence of influence of length of follow-up on results.

CONCLUSIONS

Depressive symptoms during FEP are associated with increased longer-term risk of suicidal behaviour. This association should be acknowledged during early management planning. Large-scale clinical trials are needed to identify efficacious management of depression during FEP.

Prognostic Value of S-100β Protein for Prediction of Post-Concussion Symptoms after a Mild Traumatic Brain Injury: Systematic Review and Meta-Analysis

This systematic review and meta-analysis aimed to determine the prognostic value of S-100β protein to identify patients with post-concussion symptoms after a mild traumatic brain injury (mTBI). A search strategy was submitted to seven databases from their inception to October 2016. Individual patient data were requested. Cohort studies evaluating the association between S-100β protein level and post-concussion symptoms assessed at least seven days after the mTBI were considered. Outcomes were dichotomized as persistent (≥3 months) or early (≥7 days <3 months). Our search strategy yielded 23,298 citations of which 29 studies including between seven and 223 patients (n = 2505) were included. Post-concussion syndrome (PCS) (16 studies) and neuropsychological symptoms (9 studies) were the most frequently assessed outcomes. The odds of having persistent PCS (odds ratio [OR] 0.62, 95% confidence interval [CI]: 0.34-1.12, p = 0.11, I² 0% [n = five studies]) in patients with an elevated S-100β protein serum level were not significantly different from those of patients with normal values while the odds of having early PCS (OR 1.67, 95% CI: 0.98-2.85, p = 0.06, I² 38% [n = five studies]) were close to statistical significance. Similarly, having an elevated S-100β protein serum level was not associated with the odds of returning to work at six months (OR 2.31, 95% CI: 0.50-10.64, p = 0.28, I² 22% [n = two studies]). Overall risk of bias was considered moderate. Results suggest that the prognostic biomarker S-100β protein has a low clinical value to identify patients at risk of persistent post-concussion symptoms. Variability in injury to S-100ß protein sample time, mTBI populations, and outcomes assessed could potentially explain the lack of association and needs further evaluation.

External validation, update and development of prediction models for pre-eclampsia using an Individual Participant Data (IPD) meta-analysis: the International Prediction of Pregnancy Complication Network (IPPIC pre-eclampsia) protocol

BACKGROUND

Pre-eclampsia, a condition with raised blood pressure and proteinuria is associated with an increased risk of maternal and offspring mortality and morbidity. Early identification of mothers at risk is needed to target management.

METHODS/DESIGN

We aim to systematically review the existing literature to identify prediction models for pre-eclampsia. We have established the International Prediction of Pregnancy Complication Network (IPPIC), made up of 72 researchers from 21 countries who have carried out relevant primary studies or have access to existing registry databases, and collectively possess data from more than two million patients. We will use the individual participant data (IPD) from these studies to externally validate these existing prediction models and summarise model performance across studies using random-effects meta-analysis for any, late (after 34 weeks) and early (before 34 weeks) onset pre-eclampsia. If none of the models perform well, we will recalibrate (update), or develop and validate new prediction models using the IPD. We will assess the differential accuracy of the models in various settings and subgroups according to the risk status. We will also validate or develop prediction models based on clinical characteristics only; clinical and biochemical markers; clinical and ultrasound parameters; and clinical, biochemical and ultrasound tests.

DISCUSSION

Numerous systematic reviews with aggregate data meta-analysis have evaluated various risk factors separately or in combination for predicting pre-eclampsia, but these are affected by many limitations. Our large-scale collaborative IPD approach encourages consensus towards well developed, and validated prognostic models, rather than a number of competing non-validated ones. The large sample size from our IPD will also allow development and validation of multivariable prediction model for the relatively rare outcome of early onset pre-eclampsia.

TRIAL REGISTRATION

The project was registered on Prospero on the 27 November 2015 with ID: CRD42015029349.

The 6-Minute-Walk Distance Test as a Chronic Obstructive Pulmonary Disease Stratification Tool. Insights from the COPD Biomarker Qualification Consortium

RATIONALE

The 6-minute-walk distance (6MWD) test predicts mortality in chronic obstructive pulmonary disease (COPD). Whether variability in study type (observational vs. interventional) or region performed limits use of the test as a stratification tool or outcome measure for therapeutic trials is unclear.

OBJECTIVES

To analyze the original data from several large observational studies and from randomized clinical trials with bronchodilators to support the qualification of the 6MWD test as a drug development tool in COPD.

METHODS

Original data from 14,497 patients with COPD from six observational (n = 9,641) and five interventional (n = 4,856) studies larger than 100 patients and longer than 6 months in duration were included. The geographical, anthropometrics, FEV1, dyspnea, comorbidities, and health status scores were measured. Associations between 6MWD and mortality, hospitalizations, and exacerbations adjusted by study type, age, and sex were evaluated. Thresholds for outcome prediction were calculated using receiver operating curves. The change in 6MWD after inhaled bronchodilator treatment and surgical lung volume reduction were analyzed to evaluate the responsiveness of the test as an outcome measure.

MEASUREMENTS AND MAIN RESULTS

The 6MWD was significantly lower in nonsurvivors, those hospitalized, or who exacerbated compared with those without events at 6, 12, and greater than 12 months. At these time points, the 6MWD receiver operating characteristic curve-area under the curve to predict mortality was 0.71, 0.70, and 0.68 and for hospitalizations was 0.61, 0.60, and 0.59, respectively. After treatment, the 6MWD was not different between placebo and bronchodilators but increased after surgical lung volume reduction compared with medical therapy. Variation across study types (observational or therapeutic) or regions did not confound the ability of 6MWD to predict outcome.

CONCLUSIONS

The 6MWD test can be used to stratify patients with COPD for clinical trials and interventions aimed at modifying exacerbations, hospitalizations, or death.

Somatic POLE proofreading domain mutation, immune response, and prognosis in colorectal cancer: a retrospective, pooled biomarker study

BACKGROUND

Precision cancer medicine depends on defining distinct tumour subgroups using biomarkers that may occur at very modest frequencies. One such subgroup comprises patients with exceptionally mutated (ultramutated) cancers caused by mutations that impair DNA polymerase epsilon (POLE) proofreading.

METHODS

We examined the association of POLE proofreading domain mutation with clinicopathological variables and immune response in colorectal cancers from clinical trials (VICTOR, QUASAR2, and PETACC-3) and colorectal cancer cohorts (Leiden University Medical Centre 1 and 2, Oslo 1 and 2, Bern, AMC-AJCC-II, and Epicolon-1). We subsequently investigated its association with prognosis in stage II/III colorectal cancer by Cox regression of pooled individual patient data from more than 4500 cases from these studies.

FINDINGS

Pathogenic somatic POLE mutations were detected in 66 (1·0%) of 6517 colorectal cancers, and were mutually exclusive with mismatch repair deficiency (MMR-D) in the 6277 cases for whom both markers were determined (none of 66 vs 833 [13·4%] of 6211; p<0·0001). Compared with cases with wild-type POLE, cases with POLE mutations were younger at diagnosis (median 54·5 years vs 67·2 years; p<0·0001), were more frequently male (50 [75·8%] of 66 vs 3577 [55·5%] of 6445; p=0·0010), more frequently had right-sided tumour location (44 [68·8%] of 64 vs 2463 [39·8%] of 6193; p<0·0001), and were diagnosed at an earlier disease stage (p=0·006, χ2 test for trend). Compared with mismatch repair proficient (MMR-P) POLE wild-type tumours, POLE-mutant colorectal cancers displayed increased CD8+ lymphocyte infiltration and expression of cytotoxic T-cell markers and effector cytokines, similar in extent to that observed in immunogenic MMR-D cancers. Both POLE mutation and MMR-D were associated with significantly reduced risk of recurrence compared with MMR-P colorectal cancers in multivariable analysis (HR 0·34 [95% CI 0·11-0·76]; p=0·0060 and 0·72 [0·60-0·87]; p=0·00035), although the difference between the groups was not significant.

INTERPRETATION

POLE proofreading domain mutations identify a subset of immunogenic colorectal cancers with excellent prognosis. This association underscores the importance of rare biomarkers in precision cancer medicine, but also raises important questions about how to identify and implement them in practice.

FUNDING

Cancer Research UK, Academy of Medical Sciences, Health Foundation, EU, ERC, NIHR, Wellcome Trust, Dutch Cancer Society, Dutch Digestive Foundation.

What factors predict length of stay in a neonatal unit: a systematic review

OBJECTIVE

In the UK, 1 in 10 babies require specialist neonatal care. This care can last from hours to months depending on the need of the baby. The increasing survival of very preterm babies has increased neonatal care resource use. Evidence from multiple studies is crucial to identify factors which may be important for predicting length of stay (LOS). The ability to predict LOS is vital for resource planning, decision-making and parent counselling. The objective of this review was to identify which factors are important to consider when predicting LOS in the neonatal unit.

DESIGN

A systematic review was undertaken which searched MEDLINE, EMBASE and Scopus for papers from 1994 to 2016 (May) for research investigating prediction of neonatal LOS. Strict inclusion and exclusion criteria were applied. Quality of each study was discussed, but not used as a reason for exclusion from the review.

MAIN OUTCOME MEASURE

Prediction of LOS in the neonatal unit.

RESULTS

9 studies were identified which investigated the prediction of neonatal LOS indicating a lack of evidence in the area. Inherent factors, particularly birth weight, sex and gestational age allow for a simple and objective prediction of LOS, which can be calculated on the first day of life. However, other early occurring factors may well also be important and estimates may need revising throughout the baby's stay in hospital.

CONCLUSIONS

Predicting LOS is vital to aid the commissioning of services and to help clinicians in their counselling of parents. The lack of evidence in this area indicates a need for larger studies to investigate methods of accurately predicting LOS.

Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ

Meta‐analysis using individual participant data (IPD) obtains and synthesises the raw, participant‐level data from a set of relevant studies. The IPD approach is becoming an increasingly popular tool as an alternative to traditional aggregate data meta‐analysis, especially as it avoids reliance on published results and provides an opportunity to investigate individual‐level interactions, such as treatment‐effect modifiers. There are two statistical approaches for conducting an IPD meta‐analysis: one‐stage and two‐stage. The one‐stage approach analyses the IPD from all studies simultaneously, for example, in a hierarchical regression model with random effects. The two‐stage approach derives aggregate data (such as effect estimates) in each study separately and then combines these in a traditional meta‐analysis model. There have been numerous comparisons of the one‐stage and two‐stage approaches via theoretical consideration, simulation and empirical examples, yet there remains confusion regarding when each approach should be adopted, and indeed why they may differ.

In this tutorial paper, we outline the key statistical methods for one‐stage and two‐stage IPD meta‐analyses, and provide 10 key reasons why they may produce different summary results. We explain that most differences arise because of different modelling assumptions, rather than the choice of one‐stage or two‐stage itself. We illustrate the concepts with recently published IPD meta‐analyses, summarise key statistical software and provide recommendations for future IPD meta‐analyses. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Meta-STEPP: subpopulation treatment effect pattern plot for individual patient data meta-analysis

We have developed a method, called Meta-STEPP (subpopulation treatment effect pattern plot for meta-analysis), to explore treatment effect heterogeneity across covariate values in the meta-analysis setting for time-to-event data when the covariate of interest is continuous. Meta-STEPP forms overlapping subpopulations from individual patient data containing similar numbers of events with increasing covariate values, estimates subpopulation treatment effects using standard fixed-effects meta-analysis methodology, displays the estimated subpopulation treatment effect as a function of the covariate values, and provides a statistical test to detect possibly complex treatment-covariate interactions. Simulation studies show that this test has adequate type-I error rate recovery as well as power when reasonable window sizes are chosen. When applied to eight breast cancer trials, Meta-STEPP suggests that chemotherapy is less effective for tumors with high estrogen receptor expression compared with those with low expression. Copyright © 2016 John Wiley & Sons, Ltd.

Mental Health Implications of Traumatic Brain Injury (TBI) in Children and Youth

OBJECTIVE

Traumatic brain injury (TBI) is the most common cause of death and disability in children and adolescents. Psychopathology is an established risk factor for, and a frequent consequence of, TBI. This paper reviews the literature relating psychopathology and TBI.

METHOD

Selective literature review.

RESULTS

The risk of sustaining a TBI is increased by pre-existing psychopathology (particularly ADHD and aggression) and psychosocial adversity. Even among individuals with no psychopathology prior to the injury, TBI is frequently followed by mental illness especially ADHD, personality change, conduct disorder and, less frequently, by post-traumatic stress and anxiety disorders. The outcome of TBI can be partially predicted by pre-injury adjustment and injury severity, but less well by age at injury. Few individuals receive treatment for mental illness following TBI.

CONCLUSION

TBI has substantial relevance to mental health professionals and their clinical practice. Available evidence, while limited, indicates that the risk for TBI in children and adolescents is increased in the presence of several, potentially treatable mental health conditions and that the outcome of TBI involves a range of mental health problems, many of which are treatable. Prevention and management efforts targeting psychiatric risks and outcomes are an urgent priority. Child and adolescent mental health professionals can play a critical role in the prevention and treatment of TBI through advocacy, education, policy development and clinical practice.

Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use

Jayne Tierney and colleagues offer guidance on how to spot a well-designed and well-conducted individual participant data meta-analysis.

Neurodevelopmental outcomes after cardiac surgery in infancy

BACKGROUND

Neurodevelopmental disability is the most common complication for survivors of surgery for congenital heart disease (CHD).

METHODS

We analyzed individual participant data from studies of children evaluated with the Bayley Scales of Infant Development, second edition, after cardiac surgery between 1996 and 2009. The primary outcome was Psychomotor Development Index (PDI), and the secondary outcome was Mental Development Index (MDI).

RESULTS

Among 1770 subjects from 22 institutions, assessed at age 14.5 ± 3.7 months, PDIs and MDIs (77.6 ± 18.8 and 88.2 ± 16.7, respectively) were lower than normative means (each P < .001). Later calendar year of birth was associated with an increased proportion of high-risk infants (complexity of CHD and prevalence of genetic/extracardiac anomalies). After adjustment for center and type of CHD, later year of birth was not significantly associated with better PDI or MDI. Risk factors for lower PDI were lower birth weight, white race, and presence of a genetic/extracardiac anomaly (all P ≤ .01). After adjustment for these factors, PDIs improved over time (0.39 points/year, 95% confidence interval 0.01 to 0.78; P = .045). Risk factors for lower MDI were lower birth weight, male gender, less maternal education, and presence of a genetic/extracardiac anomaly (all P < .001). After adjustment for these factors, MDIs improved over time (0.38 points/year, 95% confidence interval 0.05 to 0.71; P = .02).

CONCLUSIONS

Early neurodevelopmental outcomes for survivors of cardiac surgery in infancy have improved modestly over time, but only after adjustment for innate patient risk factors. As more high-risk CHD infants undergo cardiac surgery and survive, a growing population will require significant societal resources.

Multivariate meta-analysis of prognostic factor studies with multiple cut-points and/or methods of measurement

A prognostic factor is any measure that is associated with the risk of future health outcomes in those with existing disease. Often, the prognostic ability of a factor is evaluated in multiple studies. However, meta-analysis is difficult because primary studies often use different methods of measurement and/or different cut-points to dichotomise continuous factors into 'high' and 'low' groups; selective reporting is also common. We illustrate how multivariate random effects meta-analysis models can accommodate multiple prognostic effect estimates from the same study, relating to multiple cut-points and/or methods of measurement. The models account for within-study and between-study correlations, which utilises more information and reduces the impact of unreported cut-points and/or measurement methods in some studies. The applicability of the approach is improved with individual participant data and by assuming a functional relationship between prognostic effect and cut-point to reduce the number of unknown parameters. The models provide important inferential results for each cut-point and method of measurement, including the summary prognostic effect, the between-study variance and a 95% prediction interval for the prognostic effect in new populations. Two applications are presented. The first reveals that, in a multivariate meta-analysis using published results, the Apgar score is prognostic of neonatal mortality but effect sizes are smaller at most cut-points than previously thought. In the second, a multivariate meta-analysis of two methods of measurement provides weak evidence that microvessel density is prognostic of mortality in lung cancer, even when individual participant data are available so that a continuous prognostic trend is examined (rather than cut-points).

Long working hours and alcohol use: systematic review and meta-analysis of published studies and unpublished individual participant data

OBJECTIVE

To quantify the association between long working hours and alcohol use.

DESIGN

Systematic review and meta-analysis of published studies and unpublished individual participant data.

DATA SOURCES

A systematic search of PubMed and Embase databases in April 2014 for published studies, supplemented with manual searches. Unpublished individual participant data were obtained from 27 additional studies.

REVIEW METHODS

The search strategy was designed to retrieve cross sectional and prospective studies of the association between long working hours and alcohol use. Summary estimates were obtained with random effects meta-analysis. Sources of heterogeneity were examined with meta-regression.

RESULTS

Cross sectional analysis was based on 61 studies representing 333,693 participants from 14 countries. Prospective analysis was based on 20 studies representing 100,602 participants from nine countries. The pooled maximum adjusted odds ratio for the association between long working hours and alcohol use was 1.11 (95% confidence interval 1.05 to 1.18) in the cross sectional analysis of published and unpublished data. Odds ratio of new onset risky alcohol use was 1.12 (1.04 to 1.20) in the analysis of prospective published and unpublished data. In the 18 studies with individual participant data it was possible to assess the European Union Working Time Directive, which recommends an upper limit of 48 hours a week. Odds ratios of new onset risky alcohol use for those working 49-54 hours and ≥ 55 hours a week were 1.13 (1.02 to 1.26; adjusted difference in incidence 0.8 percentage points) and 1.12 (1.01 to 1.25; adjusted difference in incidence 0.7 percentage points), respectively, compared with working standard 35-40 hours (incidence of new onset risky alcohol use 6.2%). There was no difference in these associations between men and women or by age or socioeconomic groups, geographical regions, sample type (population based v occupational cohort), prevalence of risky alcohol use in the cohort, or sample attrition rate.

CONCLUSIONS

Individuals whose working hours exceed standard recommendations are more likely to increase their alcohol use to levels that pose a health risk.

Methodological issues and recommendations for systematic reviews of prognostic studies: an example from cardiovascular disease

BACKGROUND

Prognostic factors are associated with the risk of future health outcomes in individuals with a particular health condition. The prognostic ability of such factors is increasingly being assessed in both primary research and systematic reviews. Systematic review methodology in this area is continuing to evolve, reflected in variable approaches to key methodological aspects. The aim of this article was to (i) explore and compare the methodology of systematic reviews of prognostic factors undertaken for the same clinical question, (ii) to discuss implications for review findings, and (iii) to present recommendations on what might be considered to be 'good practice' approaches.

METHODS

The sample was comprised of eight systematic reviews addressing the same clinical question, namely whether 'aspirin resistance' (a potential prognostic factor) has prognostic utility relative to future vascular events in patients on aspirin therapy for secondary prevention. A detailed comparison of methods around study identification, study selection, quality assessment, approaches to analysis, and reporting of findings was undertaken and the implications discussed. These were summarised into key considerations that may be transferable to future systematic reviews of prognostic factors.

RESULTS

Across systematic reviews addressing the same clinical question, there were considerable differences in the numbers of studies identified and overlap between included studies, which could only partially be explained by different study eligibility criteria. Incomplete reporting and differences in terminology within primary studies hampered study identification and selection process across reviews. Quality assessment was highly variable and only one systematic review considered a checklist for studies of prognostic questions. There was inconsistency between reviews in approaches towards analysis, synthesis, addressing heterogeneity and reporting of results.

CONCLUSIONS

Different methodological approaches may ultimately affect the findings and interpretation of systematic reviews of prognostic research, with implications for clinical decision-making.

Primary squamous cell carcinomas in the thyroid gland: an individual participant data meta-analysis

Primary squamous cell carcinomas arising from the thyroid gland (SCCTh) is extremely rare diseases, which have never been fully studied. Thus, we performed a systematic review and individual participant data meta-analysis of published SCCTh cases, to understand the clinical characteristics and to identify the prognostic factors of primary SCCTh. A literature search was conducted within Medline, EMBASE, Cochrane library databases and KoreaMed using the following Medical Subject Headings (MeSH) keywords: “primary,” “squamous,” “carcinoma,” “cancer,” and “thyroid.” Eighty-four patients' individual data from 39 articles and five patients' data in our institute were selected for analysis (N = 89). The mean age at diagnosis was 63.0 years (range, 24–90) and female preponderance (M:F = 1:2) was noted. The commonest complaint was the anterior neck mass, followed by dyspnea or dysphagia, and extension to the adjacent structure was found in 72%. The median survival was 9.0 months (95% CI, 6.0–23.0) and 3-year survival rate (3YSR) was 37.6% by Kaplan–Meier method, but only 20.1% by a shared frailty model for adjusting heterogeneity. Complete resection (R0) of tumors was the only significant prognostic factor in multivariable analysis, and the benefit of adjuvant treatment was not proved. The prognosis of patients with SCCTh is very poor (20% in 3YSR), but complete resection of disease is correlated with improved survival. To achieve complete surgical eradication of tumors, early detection and accurate diagnosis should be emphasized.