Strategies to improve the credibility of meta-analyses in spine surgery: a systematic survey

Interventions to Optimize Spinal Cord Perfusion in Patients With Acute Traumatic Spinal Cord Injury: An Updated Systematic Review

STUDY DESIGN

Systematic review update.

OBJECTIVES

Interventions that aim to optimize spinal cord perfusion are thought to play an important role in minimizing secondary ischemic damage and improving outcomes in patients with acute traumatic spinal cord injuries (SCIs). However, exactly how to optimize spinal cord perfusion and enhance neurologic recovery remains controversial. We performed an update of a recent systematic review (Evaniew et al, J. Neurotrauma 2020) to evaluate the effects of Mean Arterial Pressure (MAP) support or Spinal Cord Perfusion Pressure (SCPP) support on neurological recovery and rates of adverse events among patients with acute traumatic SCI.

METHODS

We searched PubMed/MEDLINE, EMBASE and ClinicalTrials.gov for new published reports. Two reviewers independently screened articles, extracted data, and evaluated risk of bias. We implemented the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach to rate confidence in the quality of the evidence.

RESULTS

From 569 potentially relevant new citations since 2019, we identified 9 new studies for inclusion, which were combined with 19 studies from a prior review to give a total of 28 studies. According to low or very low quality evidence, the effect of MAP support on neurological recovery is uncertain, and increased SCPP may be associated with improved neurological recovery. Both approaches may involve risks for specific adverse events, but the importance of these adverse events to patients remains unclear. Very low quality evidence failed to yield reliable guidance about particular monitoring techniques, perfusion ranges, pharmacological agents, or durations of treatment.

CONCLUSIONS

This update provides an evidence base to support the development of a new clinical practice guideline for the hemodynamic management of patients with acute traumatic SCI. While avoidance of hypotension and maintenance of spinal cord perfusion are important principles in the management of an acute SCI, the literature does not provide high quality evidence in support of a particular protocol. Further prospective, controlled research studies with objective validated outcome assessments are required to examine interventions to optimize spinal cord perfusion in this setting.

Incidence of sacroiliac joint pain after lumbosacral spine fusion: A systematic review

OBJECTIVES

To determine the incidence of sacroiliac joint (SIJ) pain after lumbosacral spinal fusion.

BACKGROUND

Persistent low back pain is a potential source of disability and poor outcomes following lumbar spine fusion. The SIJ has been described as a potential source. However, there is a paucity of data concerning its importance.

METHODS

This is a PROSPERO registered systematic review. A systematic search of the English literature was performed in Medline, Embase and Cochrane Library databases. MeSH terms such as Lumbar vertebrae, Sacrum, Spinal Fusion, Pain, Sacrum, Ligaments, Sacroiliac Joint were utilized for the search. Key words such as "sacroiliac dysfunction.mp." and "sacroiliac complex.mp." were utilized for the search. Two independent reviewers reviewed articles to determine eligibility for final review and analysis. The Newcastle-Ottawa Scale was used to appraise the quality of all nonrandomized observational studies. Inverse variance weighting with random effects was used to pool data. The GRADE approach, PRISMA workflow and checklists was performed.

RESULTS

Twelve studies were included. All studies were observational and of moderate to low quality. The pooled incidence of sacroiliac joint pain was 15.8%. The pooled incidence of SIJ pain for patients without fusion extending to the sacrum was 15.8%. The pooled incidence of SIJ pain for patients with fusion extending to the sacrum was 32.9%. There was high heterogeneity.

CONCLUSION

SIJ pain is a potential cause of persistent pain after lumbar spine surgery. The current literature of poor quality. Patients presenting with pain after lumbosacral spine fusion should be evaluated for SIJ related pain.

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

BACKGROUND

Dietary recommendations and policies should be guided by rigorous systematic reviews. Reviews that are of poor methodological quality may be ineffective or misleading. Most of the evidence in nutrition comes from nonrandomized studies of nutritional exposures (usually referred to as nutritional epidemiology studies), but to date methodological evaluations of the quality of systematic reviews of such studies have been sparse and inconsistent.

OBJECTIVES

We aimed to investigate the quality of recently published systematic reviews and meta-analyses of nutritional epidemiology studies and to propose guidance addressing major limitations.

METHODS

We searched MEDLINE (January 2018-August 2019), EMBASE (January 2018-August 2019), and the Cochrane Database of Systematic Reviews (January 2018-February 2019) for systematic reviews of nutritional epidemiology studies. We included a random sample of 150 reviews.

RESULTS

Most reviews were published by authors from Asia (n = 49; 32.7%) or Europe (n = 43; 28.7%) and investigated foods or beverages (n = 60; 40.0%) and cancer morbidity and mortality (n = 54; 36%). Reviews often had important limitations: less than one-quarter (n = 30; 20.0%) reported preregistration of a protocol and almost one-third (n = 42; 28.0%) did not report a replicable search strategy. Suboptimal practices and errors in the synthesis of results were common: one-quarter of meta-analyses (n = 30; 26.1%) selected the meta-analytic model based on statistical indicators of heterogeneity and almost half of meta-analyses (n = 50; 43.5%) did not consider dose-response associations even when it was appropriate to do so. Only 16 (10.7%) reviews used an established system to evaluate the certainty of evidence.

CONCLUSIONS

Systematic reviews of nutritional epidemiology studies often have serious limitations. Authors can improve future reviews by involving statisticians, methodologists, and researchers with substantive knowledge in the specific area of nutrition being studied and using a rigorous and transparent system to evaluate the certainty of evidence.

Critically Low Confidence in the Results Produced by Spine Surgery Systematic Reviews: An AMSTAR-2 Evaluation From 4 Spine Journals

STUDY DESIGN

A systematic cross-sectional survey of systematic reviews (SRs).

OBJECTIVE

To evaluate the methodological quality of spine surgery SRs published in 2018 using the updated AMSTAR 2 critical appraisal instrument.

METHODS

We identified the PubMed indexed journals devoted to spine surgery research in 2018. All SRs of spine surgical interventions from those journals were critically appraised for quality independently by 2 reviewers using the AMSTAR 2 instrument. We calculated the percentage of SRs achieving a positive response for each AMSTAR 2 domain item and assessed the levels of confidence in the results of each SR.

RESULTS

We identified 28 SRs from 4 journals that met our criteria for inclusion. Only 49.5% of the AMSTAR 2 domain items satisfied the AMSTAR 2 criteria. Critical domain items were satisfied less often (39.1%) compared with noncritical domain items (57.3%). Domain items most poorly reported include accounting for individual study risk of bias when interpreting results (14%), list and justification of excluded articles (18%), and an a priori establishment of methods prior to the review or registered protocol (18%). The overall confidence in the results was rated "low" in 2 SRs and "critically low" in 26.

CONCLUSIONS

The credibility of a SR and its value to clinicians and policy makers are dependent on its methodological quality. This appraisal found significant methodological limitations in several critical domains, such that the confidence in the findings of these reviews is "critically low."

Clinical predictors of achieving the minimal clinically important difference after surgery for cervical spondylotic myelopathy: an external validation study from the Canadian Spine Outcomes and Research Network

OBJECTIVE

Recently identified prognostic variables among patients undergoing surgery for cervical spondylotic myelopathy (CSM) are limited to two large international data sets. To optimally inform shared clinical decision-making, the authors evaluated which preoperative clinical factors are significantly associated with improvement on the modified Japanese Orthopaedic Association (mJOA) scale by at least the minimum clinically important difference (MCID) 12 months after surgery, among patients from the Canadian Spine Outcomes and Research Network (CSORN).

METHODS

The authors performed an observational cohort study with data that were prospectively collected from CSM patients at 7 centers between 2015 and 2017. Candidate variables were tested using univariable and multiple binomial logistic regression, and multiple sensitivity analyses were performed to test assumptions about the nature of the statistical models. Validated mJOA MCIDs were implemented that varied according to baseline CSM severity.

RESULTS

Among 205 patients with CSM, there were 64 (31%) classified as mild, 86 (42%) as moderate, and 55 (27%) as severe. Overall, 52% of patients achieved MCID and the mean change in mJOA score at 12 months after surgery was 1.7 ± 2.6 points (p < 0.01), but the subgroup of patients with mild CSM did not significantly improve (mean change 0.1 ± 1.9 points, p = 0.8). Univariate analyses failed to identify significant associations between achieving MCID and sex, BMI, living status, education, smoking, disability claims, or number of comorbidities. After adjustment for potential confounders, the odds of achieving MCID were significantly reduced with older age (OR 0.7 per decade, 95% CI 0.5-0.9, p < 0.01) and higher baseline mJOA score (OR 0.8 per point, 95% CI 0.7-0.9, p < 0.01). The effects of symptom duration (OR 1.0 per additional month, 95% CI 0.9-1.0, p = 0.2) and smoking (OR 0.4, 95% CI 0.2-1.0, p = 0.06) were not statistically significant.

CONCLUSIONS

Surgery is effective at halting the progression of functional decline with CSM, and approximately half of all patients achieve the MCID. Data from the CSORN confirmed that older age is independently associated with poorer outcomes, but novel findings include that patients with milder CSM did not experience meaningful improvement, and that symptom duration and smoking were not important. These findings support a nuanced approach to shared decision-making that acknowledges some prognostic uncertainty when weighing the various risks, benefits, and alternatives to surgical treatment.

Interventions to Optimize Spinal Cord Perfusion in Patients with Acute Traumatic Spinal Cord Injuries: A Systematic Review

Interventions to optimize spinal cord perfusion via support of mean arterial pressure (MAP) or spinal cord perfusion pressure (SCPP) are thought to play a critical role in the management of patients with acute traumatic spinal cord injuries, but there is ongoing controversy about efficacy and safety. We aimed to determine the effects of optimizing spinal cord perfusion on neurological recovery and risks for adverse events. We searched multiple databases for published and unpublished reports. Two reviewers independently screened articles, extracted data, and evaluated risk of bias. We synthesized data and evaluated confidence in anticipated treatment effects according to the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach. We identified 20 eligible observational studies and 1 eligible randomized controlled trial. According to low or very low quality evidence, the effect of MAP support on neurological recovery after acute traumatic spinal cord injury is uncertain, and the use of vasopressors to support MAP may be associated with increased rates of predominantly cardiac adverse events. Increased SCPP appears likely to be associated with improved neurological recovery, but SCPP monitoring via intradural catheters at the anatomical site of injury may involve increased risks of cerebrospinal fluid leakage requiring revision surgery or pseudomeningocele. No study directly compared the effects of specific MAP goal ranges, SCPP ranges, SCPP monitoring techniques, or durations of treatment. Very low quality evidence suggests that norepinephrine may have less risk of adverse events than dopamine. The current literature is insufficient to make strong recommendations about interventions to support spinal cord perfusion via MAP or SCPP goals in patients with acute traumatic spinal cord injuries. Data are compatible with a variety of treatment decisions, and individualized approaches may be optimal. Further investigation to clarify the risks, benefits, and alternatives to MAP or SCPP support in this population is warranted.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Bias in cervical total disc replacement trials

PURPOSE OF REVIEW

Cervical disc replacement (CDR) has emerged as a motion-preserving alternative to anterior cervical discectomy and fusion in selected cases. Despite favorable literature, CDR is not universally accepted because of concerns regarding bias in the existing literature. The purpose of this review is to identify the possible biases in the disc replacement literature.

RECENT FINDINGS

Recent studies that compare CDR and ACDF have demonstrated equivalent or superior outcomes, lower rates of secondary surgery, and equivalent safety at medium- and long-term follow-up. In our review, we identified four types of bias that may affect the CDR literature: publication bias, external validity, confounding bias, and financial conflicts of interest. Bias, whether intentional or unintentional, can impact the interpretation and outcome of CDR studies. Recognition of this issue is critical when utilizing the existing literature to determine the efficacy of CDR and designing future studies.

Credibility and quality of meta-analyses addressing graft choice in anterior cruciate ligament reconstruction: a systematic review

PURPOSE

This systematic review examined the methodological credibility and quality of reporting of all meta-analyses which have compared bone-patellar tendon-bone (BPTB) versus hamstring tendon (HT) for anterior cruciate ligament (ACL) reconstruction (ACLR).

METHODS

EMBASE, MEDLINE, and The Cochrane Library were systematically searched, and two reviewers independently assessed eligibility, credibility according to the Users' Guide to medical literature, and completeness of reporting according to the preferred reporting items for systematic review and meta-analyses (PRISMA) checklist. Inter-rater agreement was quantified using Kappa, and we used Pearson's correlation coefficient to evaluate potential associations.

RESULTS

Seventeen meta-analyses were identified comparing BPTB versus HT for ACLR. The majority of meta-analyses were published in 2011 (5; 29 %), and North America was the most common continent of publication (6; 35 %). The three most commonly reported outcomes were stability (82 %), complications (76 %), and function (return to sport, IKDC score) (71 %). The median number of satisfactorily reported items in the Users' Guide was three out of seven (IQR 2-4). The median number of satisfactorily reported items in PRISMA for the meta-analyses was 20 out of 27 (IQR 19-22).

CONCLUSION

The credibility of the meta-analyses comparing BPTB versus HT autograft for ACLR although limited is improving with time. Earlier studies had limited methodological rigour; however, the more recent studies have shown promise in improved methodology. The study findings suggest that decisions should be made on a case-to-case basis with coordination of patient factors and preferences as well as surgeon experience on the background of the best available evidence.

LEVEL OF EVIDENCE

IV.

Reporting of planned statistical methods in published surgical randomised trial protocols: a protocol for a methodological systematic review

INTRODUCTION

Poor reporting can lead to inadequate presentation of data, confusion regarding research methodology used, selective reporting of results, and other misinformation regarding health research. One of the most recent attempts to improve quality of reporting comes from the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Group, which makes recommendations for the reporting of protocols. In this report, we present a protocol for a systematic review of published surgical randomised controlled trial (RCT) protocols, with the purpose of assessing the reporting quality and completeness of the statistical aspects.

METHODS

We will include all published protocols of randomised trials that investigate surgical interventions. We will search MEDLINE, EMBASE, and CENTRAL for relevant studies. Author pairs will independently review all titles, abstracts, and full texts identified by the literature search, and extract data using a structured data extraction form. We will extract the following: year of publication, country, sample size, description of study population, description of intervention and control, primary outcome, important methodological qualities, and quality of reporting of planned statistical methods based on the SPIRIT guidelines.

ETHICS AND DISSEMINATION

The results of this review will demonstrate the quality of statistical reporting of published surgical RCT protocols. This knowledge will inform recommendations to surgeons, researchers, journal editors and peer reviewers, and other knowledge users that focus on common deficiencies in reporting and how to rectify them. Ethics approval for this study is not required. We will disseminate the results of this review in peer-reviewed publications and conference presentations, and at a doctoral independent study of oral defence.

Methylprednisolone for the Treatment of Patients with Acute Spinal Cord Injuries: A Systematic Review and Meta-Analysis

Previous meta-analyses of methylprednisolone (MPS) for patients with acute traumatic spinal cord injuries (TSCIs) have not addressed confidence in the quality of evidence used for pooled effect estimates, and new primary studies have been recently published. We aimed to determine whether MPS improves motor recovery and is associated with increased risks for adverse events. We searched MEDLINE, EMBASE, and The Cochrane Library, and two reviewers independently screened articles, extracted data, and evaluated risk of bias. We pooled outcomes from randomized, controlled trials (RCTs) and controlled observational studies separately and used the Grades of Recommendation, Assessment, Development, and Evaluation approach to evaluate confidence. We included four RCTs and 17 observational studies. MPS was not associated with an increase in long-term motor score recovery (two RCTs: 335 participants; mean difference [MD], −1.11; 95% confidence interval [CI], −4.75 to 2.53; p = 0.55, low confidence; two observational studies: 528 participants; MD, 1.37; 95% CI, −3.08 to 5.83; p = 0.55, very low confidence) or improvement by at least one motor grade (three observational studies: 383 participants; risk ratio [RR], 0.84; 95% CI, 0.53–1.33; p = 0.46, very low confidence). Evidence from two RCTs demonstrated superior short-term motor score improvement if MPS was administered within 8 h of injury (two RCTs: 250 participants; MD, 4.46; 95% CI, 0.97–7.94; p = 0.01, low confidence), but risk of bias and imprecision limit confidence in these findings. Observational studies demonstrated a significantly increased risk for gastrointestinal bleeding (nine studies: 2857 participants; RR, 2.18; 95% CI, 1.13–4.19; p = 0.02, very low confidence), but RCTs did not. Pooled evidence does not demonstrate a significant long-term benefit for MPS in patients with acute TSCIs and suggests it may be associated with increased gastrointestinal bleeding. These findings support current guidelines against routine use, but strong recommendations are not warranted because confidence in the effect estimates is limited.