1
|
Lee KS, Prevedello DM. Systematic reviews and meta-analyses in neurosurgery Part II: a guide to designing the protocol. Neurosurg Rev 2024; 47:360. [PMID: 39060698 DOI: 10.1007/s10143-024-02555-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Revised: 06/22/2024] [Accepted: 07/07/2024] [Indexed: 07/28/2024]
Abstract
Despite clearly established guidelines, recent audits have found the conduct and reporting of systematic reviews and meta-analyses (SRMAs) within neurosurgery to be relatively lackluster in methodological rigor and compliance. Protocols of SRMAs allow for planning and documentation of review methods, guard against arbitrary decision-making during the review process, and enable readers to assess for the presence of selective reporting. To aid transparency, authors should provide sufficient detail in their protocol so that the readers could reproduce the study themselves. Development of our guideline drew heavily from the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols (PRISMA-P) initiative. The objective of this article is not to enumerate every detail of this checklist, but to provide guidance to authors preparing their protocol, with examples, for a systematic review in neurosurgery. Particularly, we emphasize on the PICO framework - population (P), interventions (I), comparators (C), outcomes (O) - which is central to constructing a clinical question, defining the scope of the systematic review, defining and prioritizing the primary outcome, to specifying the eligibility criteria, designing the search strategy, and identifying potential sources of heterogeneity. We encourage our readers to make use of this guideline alongside the PRISMA-P 2015 statement, when drafting and appraising systematic review protocols.
Collapse
Affiliation(s)
- Keng Siang Lee
- Department of Neurosurgery, King's College Hospital, London, UK.
- Department of Basic and Clinical Neurosciences, Maurice Wohl Clinical Neuroscience Institute, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK.
| | - Daniel M Prevedello
- Department of Neurosurgery, Wexner Medical Center, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
2
|
Lee KS, Higgins JP, Prevedello DM. Systematic reviews and meta-analyses in neurosurgery part I: interpreting and critically appraising as a guide for clinical practice. Neurosurg Rev 2024; 47:339. [PMID: 39023639 DOI: 10.1007/s10143-024-02560-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2024] [Revised: 06/26/2024] [Accepted: 07/07/2024] [Indexed: 07/20/2024]
Abstract
Neurosurgeons are inundated with the Herculean task to keep abreast with the rapid pace at which clinical research is proliferating. Systematic reviews and meta-analyses (SRMAs) have consequently surged in popularity because when executed properly, they constitute the highest level of evidence, and may save busy neurosurgeons many hours of combing the literature. Well-executed SRMAs may prove instructive for clinical practice, but poorly conducted reviews sow confusion and may potentially cause harm. Unfortunately, many SRMAs within neurosurgery are relatively lackluster in methodological rigor. When neurosurgeons apply the results of an SRMA to patient care, they should start by evaluating the extent to which the employed methods have likely protected against misleading results. The present article aims to educate the reader about how to interpret an SRMA, to assess the potential relevance of its results in the special context of the neurosurgical patient population.
Collapse
Affiliation(s)
- Keng Siang Lee
- Department of Neurosurgery, King's College Hospital, London, UK.
- Department of Basic and Clinical Neurosciences, Maurice Wohl Clinical Neuroscience Institute, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK.
| | - Julian Pt Higgins
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Daniel M Prevedello
- Department of Neurosurgery, Wexner Medical Center, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
3
|
George AM, Gupta S, Keshwara SM, Mustafa MA, Gillespie CS, Richardson GE, Steele AC, Zamanipoor Najafabadi AH, Dirven L, Marson AG, Islim AI, Jenkinson MD, Millward CP. Meningioma systematic reviews and meta-analyses: an assessment of reporting and methodological quality. Br J Neurosurg 2022; 36:678-685. [PMID: 36263847 DOI: 10.1080/02688697.2022.2115008] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
INTRODUCTION Systematic reviews (SR) and systematic reviews with meta-analysis (SRMA) can constitute the highest level of research evidence. Such evidence syntheses are relied upon heavily to inform the clinical knowledge base and to guide clinical practice for meningioma. This review evaluates the reporting and methodological quality of published meningioma evidence syntheses to date. METHODS Eight electronic databases/registries were searched to identify eligible meningioma SRs with and without meta-analysis published between January 1990 and December 2020. Articles concerning spinal meningioma were excluded. Reporting and methodological quality were assessed against the following tools: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2), and Risk Of Bias in Systematic reviews (ROBIS). RESULTS 116 SRs were identified, of which 57 were SRMAs (49.1%). The mean PRISMA score for SRMA was 20.9 out of 27 (SD 3.9, 77.0% PRISMA adherence) and for SR without meta-analysis was 13.8 out of 22 (SD 3.4, 63% PRISMA adherence). Thirty-eight studies (32.8%) achieved greater than 80% adherence to PRISMA. Methodological quality assessment against AMSTAR 2 revealed that 110 (94.8%) studies were of critically low quality. Only 21 studies (18.1%) were judged to have a low risk of bias against ROBIS. CONCLUSION The reporting and methodological quality of meningioma evidence syntheses was poor. Established guidelines and critical appraisal tools may be used as an adjunct to aid methodological conduct and reporting of such reviews, in order to improve the validity and transparency of research which may influence clinical practice.
Collapse
Affiliation(s)
- Alan M George
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Shubhi Gupta
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Sumirat M Keshwara
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Mohammad A Mustafa
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Conor S Gillespie
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - George E Richardson
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Amy C Steele
- School of Life Sciences, University of Liverpool, Liverpool, UK
| | - Amir H Zamanipoor Najafabadi
- University Neurosurgical Center Holland, Leiden University Medical Centre, Haaglanden Medical Center, Haga Teaching Hospitals, Leiden, The Netherlands
| | - Linda Dirven
- Department of Neurology, Leiden University Medical Center, Leiden, The Netherlands
| | - Anthony G Marson
- Department of Neurology, The Walton Centre NHS Foundation Trust, Liverpool, UK.,Institute of Systems, Molecular, & Integrative Biology, University of Liverpool, Liverpool, UK
| | - Abdurrahman I Islim
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK.,Institute of Systems, Molecular, & Integrative Biology, University of Liverpool, Liverpool, UK
| | - Michael D Jenkinson
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK.,Institute of Systems, Molecular, & Integrative Biology, University of Liverpool, Liverpool, UK
| | - Christopher P Millward
- Department of Neurosurgery, The Walton Centre NHS Foundation Trust, Liverpool, UK.,Institute of Systems, Molecular, & Integrative Biology, University of Liverpool, Liverpool, UK
| |
Collapse
|
4
|
Patel S, Singh R, Shlobin NA, Quiñones-Hinojosa A, Bydon M. Letter to the Editor. Excessive neurosurgical research involvement among interested neurosurgical trainees: strengths and shortcomings. J Neurosurg 2022; 137:316-317. [PMID: 35276653 DOI: 10.3171/2022.1.jns227] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Affiliation(s)
| | | | - Nathan A Shlobin
- Northwestern University Feinberg School of Medicine, Chicago, IL
| | | | | |
Collapse
|
5
|
Kondapavulur S, Silva AB, Wang DD. Ventral Intermediate Nucleus of the Thalamus versus Posterior Subthalamic Area: Network Meta-Analysis of DBS Target Site Efficacy for Essential Tremor. Stereotact Funct Neurosurg 2022; 100:224-235. [PMID: 35350022 DOI: 10.1159/000522573] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 02/03/2022] [Indexed: 01/11/2023]
Abstract
BACKGROUND Deep brain stimulation (DBS) targeting the ventral intermediate nucleus (Vim) of the thalamus or the posterior subthalamic area (PSA) are effective treatments for essential tremor (ET). However, their relative efficacy is unknown. OBJECTIVE Here, we present the first systematic review and network meta-analysis, examining the efficacy of Vim versus PSA DBS for treating medically refractory ET. METHODS We included all primary studies that reported validated Fahn-Tolosa-Marin Tremor Rating Scale (FTM-TRS) scores pre-/postimplantation or on-/off-stimulation postimplantation, for patients receiving either Vim or PSA DBS. The primary outcome was FTM-TRS score reduction; the secondary outcome was percent reduction in score. We categorized all outcomes as short-term (≤12 months) or long-term (>12 months). RESULTS For pre-/postimplantation comparisons, 19 and 11 studies met inclusion criteria for short- and long-term follow-ups, respectively. For on-/off-stimulation tremor score comparisons, 8 studies met inclusion criteria for short-term follow-up. Network meta-analysis of pre-/postimplantation tremor scores showed greater tremor reduction with PSA implantation short-term (absolute tremor reduction: PSA: -30.94 [95% confidence interval (CI): -34.93, -26.95]; Vim: -26.26 [95% CI: -33.39, -19.12]; relative tremor reduction: PSA: 63.3% [95% CI: 61.8%-64.8%]; Vim: 57.8% [95% CI: 56.5%-59.0%]). However, there was no difference in efficacy between PSA and Vim DBS when comparing tremor on-versus off-stimulation at short-term follow-up or pre- versus postimplantation tremor reduction long-term. CONCLUSION Our systematic review highlighted both heterogeneity in scoring systems used and lack of transparency in reporting total scores, limiting direct comparison across studies. We found a modestly superior efficacy with PSA stimulation in the short term, but no difference in tremor reduction long-term.
Collapse
Affiliation(s)
- Sravani Kondapavulur
- Department of Neurological Surgery, UCSF, San Francisco, California, USA.,Medical Scientist Training Program, UCSF, San Francisco, California, USA
| | - Alexander B Silva
- Department of Neurological Surgery, UCSF, San Francisco, California, USA.,Medical Scientist Training Program, UCSF, San Francisco, California, USA
| | - Doris D Wang
- Department of Neurological Surgery, UCSF, San Francisco, California, USA
| |
Collapse
|
6
|
Phua QS, Lu L, Harding M, Poonnoose SI, Jukes A, To MS. Systematic Analysis of Publication Bias in Neurosurgery Meta-Analyses. Neurosurgery 2022; 90:262-269. [DOI: 10.1227/neu.0000000000001788] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 09/19/2021] [Indexed: 12/11/2022] Open
|
7
|
Kondapavulur S, Burke J, Volz M, Wang DD, Starr PA. Use of Topical Vancomycin Powder to Reduce Surgical Site Infections after Deep Brain Stimulation Surgery: UCSF Experience and Meta-Analysis. Stereotact Funct Neurosurg 2022; 100:130-139. [PMID: 34839296 PMCID: PMC8917085 DOI: 10.1159/000520197] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 09/17/2021] [Indexed: 01/03/2023]
Abstract
OBJECTIVE Surgical site infection (SSI) is the most common serious complication of deep brain stimulation (DBS) implantation surgery. Here, we report a single-surgeon experience on the efficacy of topical, intrawound vancomycin powder (VP) in reducing SSI for DBS surgery and present the first systematic review and meta-analysis examining the effect of topical vancomycin on SSI in patients after DBS surgery. METHODS For the retrospective review, all unique patients undergoing DBS surgery at UCSF for new hardware implantation or internal pulse generator (IPG) replacement by a single surgeon from September 2013 to March 2019, with at least 1 year of follow-up data, were included. For the meta-analysis, we included all primary studies that compared SSIs with and without application of topical vancomycin in DBS surgeries. RESULTS 368 unique patients met inclusion criteria; 195 patients received topical VP (VP group) and 173 did not (control). 99/195 patients in the VP group underwent new DBS implantation and 96/195 had IPG replacement. 71/173 patients in the control group had new DBS implantation and 102/173 had IPG replacement. There were 10 total cases of SSI: 4 patients from the VP group (3 new implants and 1 IPG replacement) and 6 patients from the control group (3 new implants and 3 IPG replacements), resulting in SSI rates of 2.1 and 3.5%, respectively (p value = 0.337). Including our retrospective analysis, 6 studies met inclusion criteria for the systematic review and meta-analysis. In the 4 studies that examined primary DBS implants, 479 total patients received topical VP and 436 did not; mean odds ratio for SSI with topical vancomycin was 0.802 (95% confidence interval [CI] 0.175-3.678). Across the 5 studies that examined IPG implantations or replacements, 606 total patients received topical VP while 1,173 patients did not; mean odds ratio for SSI with topical vancomycin was 0.492 (95% CI 0.164-1.475). In either case, topical VP application did not significantly decrease risk of SSI. CONCLUSION Surgical infections after DBS surgery are uncommon events, with studies demonstrating mixed results on whether topical vancomycin reduces this risk. Our single-institution retrospective analysis and systematic review of prior studies both demonstrated no significant SSI rate reduction with topical VP. This is likely due to low baseline SSI rates, resulting in a small effect size for prevention. Given the cost-effectiveness, simplicity, and low risk, topical, intrawound VP remains a treatment option to further reduce risk of SSI, particularly in settings with higher baseline infection rates.
Collapse
Affiliation(s)
| | - John Burke
- Department of Neurological Surgery, UCSF, San Francisco, CA, USA
| | - Monica Volz
- Department of Neurological Surgery, UCSF, San Francisco, CA, USA
| | - Doris D. Wang
- Department of Neurological Surgery, UCSF, San Francisco, CA, USA
| | - Philip A. Starr
- Department of Neurological Surgery, UCSF, San Francisco, CA, USA
| |
Collapse
|
8
|
Shlobin NA, Moher D. Commentary: Reporting Guidelines for Studies on Artificial Intelligence: What Neurosurgeons Should Know. Neurosurgery 2021; 89:E316-E317. [PMID: 34432029 DOI: 10.1093/neuros/nyab331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Accepted: 07/28/2021] [Indexed: 11/14/2022] Open
Affiliation(s)
- Nathan A Shlobin
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA.,Division of Pediatric Neurosurgery, Lurie Children's Hospital, Chicago, Illinois, USA
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada.,School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| |
Collapse
|
9
|
Shlobin NA, Moher D. Commentary: Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 Statement: What Neurosurgeons Should Know. Neurosurgery 2021; 89:E267-E268. [PMID: 34352888 DOI: 10.1093/neuros/nyab289] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Accepted: 06/27/2021] [Indexed: 11/13/2022] Open
Affiliation(s)
- Nathan A Shlobin
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA.,Division of Pediatric Neurosurgery, Lurie Children's Hospital, Chicago, Illinois, USA
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada.,School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| |
Collapse
|
10
|
Blanco-Teherán C, Quintana-Pájaro L, Narvaez-Rojas A, Martínez-Pérez R, García-Ballestas E, Moscote Salazar L, Prada-Soto S, Lozada-Martínez I. Evidence-based medicine in neurosurgery: why and how? J Neurosurg Sci 2021; 66:49-53. [PMID: 34342191 DOI: 10.23736/s0390-5616.21.05331-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Since its inception, medical training and practice are based primarily on the clinical experience provided by the mentors of each of the medical schools. In the 1990s, the first steps were taken for the construction of what we now know as evidence-based medicine. Evidence-based medicine is the set of studies, methods, and principles for the prevention of medical diseases, management guidelines, and algorithms used in sources of evidence. Neurosurgery based on evidence has emerged thanks to advances in neuroscience and information technology that allows the globalization of current scientific information. The results of important reviews on the levels of evidence in neurosurgery are low in percentages of high-quality evidence in this field of medicine. Based on the above, the objective of this manuscript is to describe the application of evidence in neurosurgery, the current state of the art in evidence-based medicine, and the steps needed to create evidence of the best quality in neurosurgery.
Collapse
Affiliation(s)
| | | | - Alexis Narvaez-Rojas
- Department of General Surgery, Carlos Roberto Huembes Hospital, National Autonomous University of Nicaragua, Managua, Nicaragua
| | | | | | - Luis Moscote Salazar
- Center for Biomedical Research, University of Cartagena, Cartagena, Colombia.,Medical and Surgical Research Center, University of Cartagena, Cartagena, Colombia.,Colombian Clinical Research Group in Neurocritical Care, University of Cartagena, Cartagena, Colombia.,Latin American Council of Neurocritical Care, Cartagena, Colombia
| | - Silvia Prada-Soto
- Medical and Surgical Research Center, University of Cartagena, Cartagena, Colombia
| | - Ivan Lozada-Martínez
- Medical and Surgical Research Center, University of Cartagena, Cartagena, Colombia - .,Colombian Clinical Research Group in Neurocritical Care, University of Cartagena, Cartagena, Colombia.,Latin American Council of Neurocritical Care, Cartagena, Colombia
| |
Collapse
|
11
|
O'Donohoe TJ, Bridson TL, Shafik CG, Wynne D, Dhillon RS, Tee JW. Quality of Literature Searches Published in Leading Neurosurgical Journals: A Review of Reviews. Neurosurgery 2021; 88:891-899. [PMID: 33503659 DOI: 10.1093/neuros/nyaa573] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2020] [Accepted: 11/10/2020] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND There is mounting evidence that the search strategies upon which systematic reviews (SRs) are based frequently contain errors are incompletely reported or insensitive. OBJECTIVE To appraise the quality of search strategies in the 10 leading specialty neurosurgical journals and identify factors associated with superior searches. METHODS This research-on-research study systematically surveyed SRs published in the 10 leading neurosurgical journals between 01/10/2017 and 31/10/2019. All SRs were eligible for assessment using a predefined coding manual that was adapted from the preferred reporting items for systematic reviews and meta-analyses (PRISMA), a measurement tool to assess systematic reviews (AMSTAR), and Cochrane Collaboration guidelines. The PubMed interface was used to search the MEDLINE database, which was supplemented by individual journal searches. Descriptive statistics were utilized to identify factors associated with improved search strategies. RESULTS A total of 633 articles were included and contained a median of 19.00 (2.00-1654.00) studies. Less than half (45.97%) of included search strategies were considered to be reproducible. Aggregated reporting score was positively associated with in-text reference to reporting guideline adherence (τb = 0.156, P < .01). The number of articles retrieved by a search (τb = 0.11, P < .01) was also associated with the reporting of a reproducible search strategy. CONCLUSION This study demonstrates that the search strategies used in neurosurgical SRs require improvement. In addition to increasing awareness of reporting standards, we propose that this be achieved by the incorporation of PRISMA and other guidelines into article submission and peer-review processes. This may lead to the conduct of more informative SRs, which may result in improved clinician decision-making and patient outcomes.
Collapse
Affiliation(s)
- Tom J O'Donohoe
- Department of Neurosurgery, St. Vincent's Hospital, Fitzroy, Australia
- National Trauma Research Institute, Prahran, Australia
| | - Tahnee L Bridson
- College of Medicine and Dentistry, James Cook University, Townsville, Australia
| | | | - David Wynne
- Department of Neurosurgery, St. Vincent's Hospital, Fitzroy, Australia
| | - Rana S Dhillon
- Department of Neurosurgery, St. Vincent's Hospital, Fitzroy, Australia
| | - Jin W Tee
- National Trauma Research Institute, Prahran, Australia
- Department of Neurosurgery, Alfred Health, Melbourne, Australia
| |
Collapse
|
12
|
Esene IN. Commentary: Pearls for Interpreting Neurosurgical Systematic Reviews and Meta-Analyses: Lessons From a Collaborative Effort. Neurosurgery 2020; 87:E594-E595. [DOI: 10.1093/neuros/nyaa376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Accepted: 06/23/2020] [Indexed: 11/12/2022] Open
|
13
|
Dettori JR, Skelly AC, Brodt ED. Critically Low Confidence in the Results Produced by Spine Surgery Systematic Reviews: An AMSTAR-2 Evaluation From 4 Spine Journals. Global Spine J 2020; 10:667-673. [PMID: 32677574 PMCID: PMC7359690 DOI: 10.1177/2192568220917926] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
STUDY DESIGN A systematic cross-sectional survey of systematic reviews (SRs). OBJECTIVE To evaluate the methodological quality of spine surgery SRs published in 2018 using the updated AMSTAR 2 critical appraisal instrument. METHODS We identified the PubMed indexed journals devoted to spine surgery research in 2018. All SRs of spine surgical interventions from those journals were critically appraised for quality independently by 2 reviewers using the AMSTAR 2 instrument. We calculated the percentage of SRs achieving a positive response for each AMSTAR 2 domain item and assessed the levels of confidence in the results of each SR. RESULTS We identified 28 SRs from 4 journals that met our criteria for inclusion. Only 49.5% of the AMSTAR 2 domain items satisfied the AMSTAR 2 criteria. Critical domain items were satisfied less often (39.1%) compared with noncritical domain items (57.3%). Domain items most poorly reported include accounting for individual study risk of bias when interpreting results (14%), list and justification of excluded articles (18%), and an a priori establishment of methods prior to the review or registered protocol (18%). The overall confidence in the results was rated "low" in 2 SRs and "critically low" in 26. CONCLUSIONS The credibility of a SR and its value to clinicians and policy makers are dependent on its methodological quality. This appraisal found significant methodological limitations in several critical domains, such that the confidence in the findings of these reviews is "critically low."
Collapse
Affiliation(s)
- Joseph R. Dettori
- Spectrum Research, Inc, Steilacoom, WA, USA,Joseph R. Dettori, Spectrum Research, Inc, Steilacoom, WA, USA.
| | | | | |
Collapse
|
14
|
Lu VM, Graffeo CS, Perry A, Link MJ, Meyer FB, Dawood HY, Mekary RA, Smith TR. Pearls for Interpreting Neurosurgical Systematic Reviews and Meta-Analyses: Lessons From a Collaborative Effort. Neurosurgery 2020; 87:435-441. [DOI: 10.1093/neuros/nyaa027] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Accepted: 12/28/2019] [Indexed: 01/11/2023] Open
Abstract
Abstract
Systematic reviews and meta-analyses in the neurosurgical literature have surged in popularity over the last decade. It is our concern that, without a renewed effort to critically interpret and appraise these studies as high or low quality, we run the risk of the quality and value of evidence-based medicine in neurosurgery being misinterpreted. Correspondingly, we have outlined 4 major domains to target in interpreting neurosurgical systematic reviews and meta-analyses based on the lessons learned by a collaboration of clinicians and academics summarized as 4 pearls. The domains of (1) heterogeneity, (2) modeling, (3) certainty, and (4) bias in neurosurgical systematic reviews and meta-analyses were identified as aspects in which the authors’ approaches have changed over time to improve robustness and transparency. Examples of how and why these pearls were adapted were provided in areas of cranial neuralgia, spine, pediatric, and neuro-oncology to demonstrate how neurosurgical readers and writers may improve their interpretation of these domains. The incorporation of these pearls into practice will empower neurosurgical academics to effectively interpret systematic reviews and meta-analyses, enhancing the quality of our evidence-based medicine literature while maintaining a critical focus on the needs of the individual patients in neurosurgery.
Collapse
Affiliation(s)
- Victor M Lu
- Department of Neurosurgery, Mayo Clinic, Rochester, Minnesota
| | | | - Avital Perry
- Department of Neurosurgery, Mayo Clinic, Rochester, Minnesota
| | - Michael J Link
- Department of Neurosurgery, Mayo Clinic, Rochester, Minnesota
| | - Fredric B Meyer
- Department of Neurosurgery, Mayo Clinic, Rochester, Minnesota
| | - Hassan Y Dawood
- Computational Neuroscience Outcomes Center, Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Rania A Mekary
- Computational Neuroscience Outcomes Center, Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
- School of Pharmacy, MCPHS University, Boston, Massachusetts
| | - Timothy R Smith
- Computational Neuroscience Outcomes Center, Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
15
|
To MS, Jukes A. Reporting trends of p values in the neurosurgical literature. J Neurosurg 2020; 132:662-670. [PMID: 30738384 DOI: 10.3171/2018.8.jns172897] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2017] [Accepted: 08/28/2018] [Indexed: 11/06/2022]
Abstract
OBJECTIVE The objective of this study was to evaluate the trends in reporting of p values in the neurosurgical literature from 1990 through 2017. METHODS All abstracts from the Journal of Neurology, Neurosurgery, and Psychiatry (JNNP), Journal of Neurosurgery (JNS) collection (including Journal of Neurosurgery: Spine and Journal of Neurosurgery: Pediatrics), Neurosurgery (NS), and Journal of Neurotrauma (JNT) available on PubMed from 1990 through 2017 were retrieved. Automated text mining was performed to extract p values from relevant abstracts. Extracted p values were analyzed for temporal trends and characteristics. RESULTS The search yielded 47,889 relevant abstracts. A total of 34,324 p values were detected in 11,171 abstracts. Since 1990 there has been a steady, proportionate increase in the number of abstracts containing p values. There were average absolute year-on-year increases of 1.2% (95% CI 1.1%-1.3%; p < 0.001), 0.93% (95% CI 0.75%-1.1%; p < 0.001), 0.70% (95% CI 0.57%-0.83%; p < 0.001), and 0.35% (95% CI 0.095%-0.60%; p = 0.0091) of abstracts reporting p values in JNNP, JNS, NS, and JNT, respectively. There have also been average year-on-year increases of 0.045 (95% CI 0.031-0.059; p < 0.001), 0.052 (95% CI 0.037-0.066; p < 0.001), 0.042 (95% CI 0.030-0.054; p < 0.001), and 0.041 (95% CI 0.026-0.056; p < 0.001) p values reported per abstract for these respective journals. The distribution of p values showed a positive skew and strong clustering of values at rounded decimals (i.e., 0.01, 0.02, etc.). Between 83.2% and 89.8% of all reported p values were at or below the "significance" threshold of 0.05 (i.e., p ≤ 0.05). CONCLUSIONS Trends in reporting of p values and the distribution of p values suggest publication bias remains in the neurosurgical literature.
Collapse
Affiliation(s)
- Minh-Son To
- 1School of Medicine, Flinders University, Bedford Park
| | - Alistair Jukes
- 2Department of Neurosurgery, Royal Adelaide Hospital, Adelaide
- 3Department of Otolaryngology, Head and Neck Surgery, The Queen Elizabeth Hospital, Woodville; and
- 4School of Medicine, University of Adelaide, South Australia, Australia
| |
Collapse
|
16
|
Packer M. Methodological and Clinical Heterogeneity and Extraction Errors in Meta-Analyses of Catheter Ablation for Atrial Fibrillation in Heart Failure. J Am Heart Assoc 2019; 8:e013779. [PMID: 31625420 PMCID: PMC6898828 DOI: 10.1161/jaha.119.013779] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Background Meta‐analyses are expected to follow a standardized process, and thus, they have become highly formulaic, although there is little evidence that such regimentation yields high‐quality results. Methods and Results This article describes the results of a critical examination of 14 published meta‐analyses of catheter ablation for atrial fibrillation in heart failure that were based on a nearly identical core set of 4 to 6 primary trials. Methodological issues included (1) the neglect of primary data or the failure to report any primary data; (2) the inaccurate recording of the number of randomized patients; (3) the lack of attention to data missingness or baseline imbalances; (4) the failure to contact investigators of primary trials for additional data; (5) the incorrect extraction of data, the misidentification of events, and the assignment of events to the wrong treatment groups; (6) the calculation of summary estimates based on demonstrably heterogenous data, methods of differing reliability, or unrelated end points; and (7) the development of conclusions based on sparse numbers of events or overly reliant on the results of 1 dominant trial. Conclusions These findings reinforce existing concerns about the methodological validity of meta‐analyses and their current status in the hierarchy of medical evidence, and they raise new questions about the process by which meta‐analyses undergo peer review by medical journals.
Collapse
Affiliation(s)
- Milton Packer
- Baylor Heart and Vascular Institute Baylor University Medical Center Dallas TX.,Imperial College London United Kingdom
| |
Collapse
|
17
|
O'Donohoe TJ, Dhillon R, Bridson TL, Tee J. Reporting Quality of Systematic Review Abstracts Published in Leading Neurosurgical Journals: A Research on Research Study. Neurosurgery 2019; 85:1-10. [PMID: 30649511 DOI: 10.1093/neuros/nyy615] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2017] [Accepted: 11/21/2018] [Indexed: 08/09/2023] Open
Abstract
BACKGROUND Systematic review (SR) abstracts are frequently relied upon to guide clinical decision-making. However, there is mounting evidence that the quality of abstract reporting in the medical literature is suboptimal. OBJECTIVE To appraise SR abstract reporting quality in neurosurgical journals and identify factors associated with improved reporting. METHODS This study systematically surveyed SR abstracts published in 8 leading neurosurgical journals between 8 April 2007 and 21 August 2017. Abstracts were identified through a search of the MEDLINE database and their reporting quality was determined in duplicate using a tool derived from the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Abstracts (PRISMA-A) statement. All SR abstracts that provided comparison between treatment strategies were eligible for inclusion. Descriptive statistics were utilized to identify factors associated with improved reporting. RESULTS A total of 257 abstracts were included in the analysis, with a mean of 22.8 (±25.3) included studies. The overall quality of reporting in included abstracts was suboptimal, with a mean score of 53.05% (±11.18). Reporting scores were higher among abstracts published after the release of the PRISMA-A guidelines (M = 56.52; 21.74-73.91) compared with those published beforehand (M = 47.83; 8.70-69.57; U = 4346.00, z = -4.61, P < .001). Similarly, both word count (r = 0.338, P < .001) and journal impact factor (r = 0.199, P = .001) were associated with an improved reporting score. CONCLUSION This study demonstrates that the overall reporting quality of abstracts in leading neurosurgical journals requires improvement. Strengths include the large number abstracts assessed, and its weaknesses include the fact that only neurosurgery-specific journals were surveyed. We recommend that attention be turned toward strengthening abstract submission and peer-review processes.
Collapse
Affiliation(s)
- Tom J O'Donohoe
- Department of Neurosurgery, St. Vincent's Hospital, Fitzroy, Victoria, Australia
- National Trauma Research Institute, Prahran, Victoria, Australia
| | - Rana Dhillon
- Department of Neurosurgery, St. Vincent's Hospital, Fitzroy, Victoria, Australia
| | - Tahnee L Bridson
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Jin Tee
- National Trauma Research Institute, Prahran, Victoria, Australia
- Department of Neurosurgery, Alfred Health, Prahran, Victoria, Australia
| |
Collapse
|
18
|
Yan P, Yao L, Li H, Zhang M, Xun Y, Li M, Cai H, Lu C, Hu L, Guo T, Liu R, Yang K. The methodological quality of robotic surgical meta-analyses needed to be improved: a cross-sectional study. J Clin Epidemiol 2019; 109:20-29. [DOI: 10.1016/j.jclinepi.2018.12.013] [Citation(s) in RCA: 67] [Impact Index Per Article: 13.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Revised: 12/10/2018] [Accepted: 12/17/2018] [Indexed: 12/13/2022]
|
19
|
Joiner EF, Youngerman BE, Hudson TS, Yang J, Welch MR, McKhann GM, Neugut AI, Bruce JN. Effectiveness of perioperative antiepileptic drug prophylaxis for early and late seizures following oncologic neurosurgery: a meta-analysis. J Neurosurg 2019; 130:1274-1282. [PMID: 29701546 DOI: 10.3171/2017.10.jns172236] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2017] [Accepted: 10/30/2017] [Indexed: 01/08/2023]
Abstract
OBJECTIVE The purpose of this meta-analysis was to evaluate the impact of perioperative antiepileptic drug (AED) prophylaxis on short- and long-term seizure incidence among patients undergoing brain tumor surgery. It is the first meta-analysis to focus exclusively on perioperative AED prophylaxis among patients undergoing brain tumor surgery. METHODS The authors searched PubMed/MEDLINE, Embase, Cochrane Central Register of Controlled Trials, clinicaltrials.gov, and the System for Information on Gray Literature in Europe for records related to perioperative AED prophylaxis for patients with brain tumors. Risk of bias in the included studies was assessed using the Cochrane risk of bias tool. Incidence rates for early seizures (within the first postoperative week) and total seizures were estimated based on data from randomized controlled trials. A Mantel-Haenszel random-effects model was used to analyze pooled relative risk (RR) of early seizures (within the first postoperative week) and total seizures associated with perioperative AED prophylaxis versus control. RESULTS Four RCTs involving 352 patients met the criteria of inclusion. The results demonstrated that perioperative AED prophylaxis for patients undergoing brain tumor surgery provides a statistically significant reduction in risk of early postoperative seizures compared with control (RR = 0.352, 95% confidence interval 0.130-0.949, p = 0.039). AED prophylaxis had no statistically significant effect on the total (combined short- and long-term) incidence of seizures. CONCLUSIONS This meta-analysis demonstrates for the first time that perioperative AED prophylaxis for brain tumor surgery provides a statistically significant reduction in early postoperative seizure risk.
Collapse
Affiliation(s)
| | | | | | - Jingyan Yang
- 2Department of Epidemiology, Mailman School of Public Health, and
| | - Mary R Welch
- 3Department of Neurology, Columbia University
- 4Herbert Irving Comprehensive Cancer Center, and
| | - Guy M McKhann
- 1Department of Neurological Surgery
- 4Herbert Irving Comprehensive Cancer Center, and
| | - Alfred I Neugut
- 2Department of Epidemiology, Mailman School of Public Health, and
- 4Herbert Irving Comprehensive Cancer Center, and
- 5Department of Medicine, Columbia University College of Physicians and Surgeons, New York, New York
| | - Jeffrey N Bruce
- 1Department of Neurological Surgery
- 4Herbert Irving Comprehensive Cancer Center, and
| |
Collapse
|
20
|
Page MJ, Moher D. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev 2017; 6:263. [PMID: 29258593 PMCID: PMC5738221 DOI: 10.1186/s13643-017-0663-8] [Citation(s) in RCA: 344] [Impact Index Per Article: 49.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/02/2017] [Accepted: 12/08/2017] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations. METHODS We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated). RESULTS We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions. CONCLUSIONS Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.
Collapse
Affiliation(s)
- Matthew J Page
- School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, VIC, 3004, Australia.
| | - David Moher
- Centre for Journalology and Canadian EQUATOR Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, K1H 8L6, Canada.,School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1H 8M5, Canada
| |
Collapse
|
21
|
Gómez-García F, Ruano J, Aguilar-Luque M, Gay-Mimbrera J, Maestre-Lopez B, Sanz-Cabanillas J, Carmona-Fernández P, González-Padilla M, Vélez García-Nieto A, Isla-Tejera B. Systematic reviews and meta-analyses on psoriasis: role of funding sources, conflict of interest and bibliometric indices as predictors of methodological quality. Br J Dermatol 2017; 176:1633-1644. [DOI: 10.1111/bjd.15380] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/31/2017] [Indexed: 12/11/2022]
Affiliation(s)
- F. Gómez-García
- Department of Dermatology; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - J. Ruano
- Department of Dermatology; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - M. Aguilar-Luque
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - J. Gay-Mimbrera
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - B. Maestre-Lopez
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
- School of Medicine; Universidad de Córdoba; Córdoba Spain
| | - J.L. Sanz-Cabanillas
- Department of Dermatology; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - P.J. Carmona-Fernández
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - M. González-Padilla
- Department of Dermatology; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - A. Vélez García-Nieto
- Department of Dermatology; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
| | - B. Isla-Tejera
- Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)/Hospital Universitario Reina Sofía/Universidad de Córdoba; Córdoba Spain
- Department of Pharmacy; Hospital Universitario Reina Sofía; Menendez Pidal Ave 14004 Córdoba Spain
| |
Collapse
|
22
|
Oh JH, Shin WJ, Park S, Chung JS. Reporting and methodologic evaluation of meta-analyses published in the anesthesia literature according to AMSTAR and PRISMA checklists: a preliminary study. Korean J Anesthesiol 2017; 70:446-455. [PMID: 28794841 PMCID: PMC5548948 DOI: 10.4097/kjae.2017.70.4.446] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2016] [Revised: 03/03/2017] [Accepted: 03/04/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND There have been few recent reports on the methodological quality of meta-analysis, despite the enormous number of studies using meta-analytic techniques in the field of anesthesia. The purpose of this study was to evaluate the quality of meta-analyses and systematic reviews according to the Assessment of Multiple Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines in the anesthesia literature. METHODS A search was conducted to identify all meta-analyses ever been published in the British Journal of Anaesthesia (BJA), Anaesthesia, and Korean Journal of Anesthesiology (KJA) between Jan. 01, 2004 and Nov. 31, 2016. We aimed to apply the AMSTAR and PRISMA checklists to all published meta-analyses. RESULTS We identified 121 meta-analyses in the anesthesia literature from January 2004 through the end of November 2016 (BJA; 75, Anaesthesia; 43, KJA; 3). The number of studies published and percentage of 'Yes' responses for meta-analysis articles published after the year 2010 was significantly increased compared to that of studies published before the year 2009 (P = 0.014 for Anaesthesia). In the anesthesia literature as a whole, participation of statisticians as authors statistically improved average scores of PRISMA items (P = 0.004) especially in the BJA (P = 0.003). CONCLUSIONS Even though there is little variability in the reporting and methodology of meta-analysis in the anesthesia literature, significant quality improvement in the reporting was observed in the Anaesthesia by applying the PRISMA checklist. Participation of a statistician as an author improved the reporting quality of the meta-analysis.
Collapse
Affiliation(s)
- Jae Hoon Oh
- Department of Anesthesiology and Pain Medicine, Hanyang University College of Medicine, Seoul, Korea
| | - Woo Jong Shin
- Department of Anesthesiology and Pain Medicine, Hanyang University College of Medicine, Seoul, Korea
| | - Suin Park
- Department of Anesthesiology and Pain Medicine, Hanyang University College of Medicine, Seoul, Korea
| | - Jae Soon Chung
- Department of Anesthesiology and Pain Medicine, Hanyang University College of Medicine, Seoul, Korea
| |
Collapse
|
23
|
Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: Review of literature. J Clin Neurosci 2017; 39:45-48. [PMID: 28246008 DOI: 10.1016/j.jocn.2017.02.016] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2016] [Accepted: 02/08/2017] [Indexed: 11/28/2022]
Abstract
BACKGROUND The knowledge of reporting compliance of systematic reviews with PRISMA guidelines may assist in improving the quality of secondary research in brain AVM management and subsequently application to patient population and clinical practice. This may allow researchers and clinicians to be equipped to appraise existing literatures based on known deficit to look for or expect. The objective of this study was to assess the compliance of systematic reviews and meta-analysis in the management of brain AVM. METHODS Systematic reviews and meta-analyses articles published in medical journals between 1st of May 2011 and 30th April 2016 (five-year period) were examined. Exclusion criteria were articles that were not systematic reviews and not meta-analyses, narrative literature reviews, historical literature reviews, animal studies, unpublished articles, commentaries and letter to the editor. Electronic database search performed through Medline PubMed on 20th September 2016. RESULTS This systematic review examined seven systematic review articles on intracranial arteriovenous malformation compliance with PRISMA statement guidelines. The mean percentage of applicable PRISMA items across all studies was 74% (range 67-93%). Protocol registration and declaration, risk of bias and funding sources were the most poorly reported of the PRISMA items (14% each). A significant variance in the total percentages was evident between studies (67-93%). CONCLUSION Systematic review reporting in medical literature is excessively variable and overall poor. As these papers are being published with increasing frequency, need to fully adhere to PRISMA statement guide for systematic review to ensure high-quality publications. Complete reporting of PRISMA items within systematic reviews in cerebral arteriovenous malformation enhance quality assessment, robust critical appraisal, better judgement and ultimately sound application to practice thereby improving research standards and patients care.
Collapse
Affiliation(s)
- T Akhigbe
- Clincal Research Fellow, University Hospital Southampton, UK.
| | - A Zolnourian
- Clincal Research Fellow, University Hospital Southampton, UK.
| | - D Bulters
- Consultant Neurosurgeon, University Hospital Southampton, UK.
| |
Collapse
|
24
|
Brinjikji W, Rouchaud A. Reply. AJNR Am J Neuroradiol 2017; 38:E8. [PMID: 27737855 DOI: 10.3174/ajnr.a4987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- W Brinjikji
- Department of Radiology Mayo Clinic Rochester, Minnesota
| | - A Rouchaud
- Department of Interventional Neuroradiology Hôpital Bicêtre, APHP, Paris Sud Université Paris, France
| |
Collapse
|
25
|
Chapman SJ, Drake TM, Bolton WS, Barnard J, Bhangu A. Longitudinal analysis of reporting and quality of systematic reviews in high-impact surgical journals. Br J Surg 2016; 104:198-204. [PMID: 28001294 DOI: 10.1002/bjs.10423] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2016] [Revised: 08/02/2016] [Accepted: 10/06/2016] [Indexed: 12/17/2022]
Abstract
BACKGROUND The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement aims to optimize the reporting of systematic reviews. The performance of the PRISMA Statement in improving the reporting and quality of surgical systematic reviews remains unclear. METHODS Systematic reviews published in five high-impact surgical journals between 2007 and 2015 were identified from online archives. Manuscripts blinded to journal, publication year and authorship were assessed according to 27 reporting criteria described by the PRISMA Statement and scored using a validated quality appraisal tool (AMSTAR, Assessing the Methodological Quality of Systematic Reviews). Comparisons were made between studies published before (2007-2009) and after (2011-2015) its introduction. The relationship between reporting and study quality was measured using Spearman's rank test. RESULTS Of 281 eligible manuscripts, 80 were published before the PRISMA Statement and 201 afterwards. Most manuscripts (208) included a meta-analysis, with the remainder comprising a systematic review only. There was no meaningful change in median compliance with the PRISMA Statement (19 (i.q.r. 16-21) of 27 items before versus 19 (17-22) of 27 after introduction of PRISMA) despite achieving statistical significance (P = 0·042). Better reporting compliance was associated with higher methodological quality (rs = 0·70, P < 0·001). CONCLUSION The PRISMA Statement has had minimal impact on the reporting of surgical systematic reviews. Better compliance was associated with higher-quality methodology.
Collapse
Affiliation(s)
- S J Chapman
- Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK
| | - T M Drake
- University of Sheffield Medical School, Sheffield, UK
| | - W S Bolton
- Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK
| | - J Barnard
- University of Leeds Medical School, Leeds, UK
| | - A Bhangu
- Department of Colorectal Surgery, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| |
Collapse
|
26
|
Whaley P, Letcher RJ, Covaci A, Alcock R. Raising the standard of systematic reviews published in Environment International. ENVIRONMENT INTERNATIONAL 2016; 97:274-276. [PMID: 27567414 DOI: 10.1016/j.envint.2016.08.007] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2016] [Accepted: 08/16/2016] [Indexed: 05/26/2023]
Affiliation(s)
- Paul Whaley
- Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK.
| | - Robert J Letcher
- Environment and Climate Change Canada, National Wildlife Research Centre, 1125 Colonel By Drive, Carleton University, Ottawa, ON, Canada
| | - Adrian Covaci
- Toxicological Centre, University of Antwerp, Universiteitsplein 1, 2610 Wilrijk, Belgium
| | - Ruth Alcock
- Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK
| |
Collapse
|
27
|
Koffel JB, Rethlefsen ML. Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study. PLoS One 2016; 11:e0163309. [PMID: 27669416 PMCID: PMC5036875 DOI: 10.1371/journal.pone.0163309] [Citation(s) in RCA: 54] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2016] [Accepted: 09/07/2016] [Indexed: 02/06/2023] Open
Abstract
Background A high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines. Objectives To examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy. Methods We identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility. Results 272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy. Conclusions Despite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.
Collapse
Affiliation(s)
- Jonathan B. Koffel
- Bio-Medical Library, University of Minnesota, Minneapolis, Minnesota, United States of America
- * E-mail:
| | - Melissa L. Rethlefsen
- Spencer S. Eccles Health Sciences Library, University of Utah, Salt Lake City, Utah, United States of America
| |
Collapse
|
28
|
DeLong WB, Polissar NL, Neradilek MB, Laam LA. A Letter to the Editor regarding, "Time to Surgery and Outcomes in Cauda Equina Syndrome: An Analysis of 45 Cases." Critique of Bydon et al. World Neurosurg 2016; 93:479-82. [PMID: 27637700 DOI: 10.1016/j.wneu.2016.02.107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2016] [Accepted: 02/25/2016] [Indexed: 10/21/2022]
Affiliation(s)
- W Bradford DeLong
- Department of Neurosurgery, University of California, San Francisco, San Francisco, California, USA.
| | | | | | - Leslie A Laam
- Health Services Research and Policy, University of Rochester Medical Center, Rochester, New York, USA
| |
Collapse
|
29
|
Is the Best Evidence Good Enough: Quality Assessment and Factor Analysis of Meta-Analyses on Depression. PLoS One 2016; 11:e0157808. [PMID: 27336624 PMCID: PMC4919061 DOI: 10.1371/journal.pone.0157808] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2016] [Accepted: 06/06/2016] [Indexed: 01/08/2023] Open
Abstract
Background The quality of meta-analyses (MAs) on depression remains uninvestigated. Objective To assess the overall reporting and methodological qualities of MAs on depression and to explore potential factors influencing both qualities. Methods MAs investigating epidemiology and interventions for depression published in the most recent year (2014–2015) were selected from PubMed, EMBASE, PsycINFO and Cochrane Library. The characteristics of the included studies were collected and the total and per-item quality scores of the included studies were calculated based on the two checklists. Univariate and multivariate linear regression analyses were used to explore the potential factors influencing the quality of the articles. Results A total of 217 MAs from 74 peer-reviewed journals were included. The mean score of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was 23.0 of 27 and mean score of Assessment of Multiple Systematic Reviews (AMSTAR) was 8.3 of 11. Items assessing registration and protocol (14.2%, 37/217) in PRISMA and item requiring a full list of included and excluded studies (16.1%, 40/217) in AMSTAR had poorer adherences than other items. The MAs that included only RCTs, pre-registered, had five more authors or authors from Cochrane groups and the MAs found negative results had better reporting and methodological qualities. Conclusions The reporting and methodological qualities of MAs on depression remained to be improved. Design of included studies, characteristics of authors and pre-registration in PROSPERO database are important factors influencing quality of MAs in the field of depression.
Collapse
|
30
|
Tsivgoulis G, Katsanos AH, Mavridis D, Magoufis G, Arthur A, Alexandrov AV. Mechanical Thrombectomy Improves Functional Outcomes Independent of Pretreatment With Intravenous Thrombolysis. Stroke 2016; 47:1661-4. [DOI: 10.1161/strokeaha.116.013097] [Citation(s) in RCA: 46] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2016] [Accepted: 03/16/2016] [Indexed: 01/19/2023]
Affiliation(s)
- Georgios Tsivgoulis
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| | - Aristeidis H. Katsanos
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| | - Dimitris Mavridis
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| | - Georgios Magoufis
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| | - Adam Arthur
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| | - Andrei V. Alexandrov
- From the Department of Neurology (G.T., A.V.A.) and Department of Neurosurgery, Semmes-Murphey Neurologic and Spine Institute (A.A.), University of Tennessee Health Science Center, Memphis; Second Department of Neurology, “Attikon” Hospital, School of Medicine, University of Athens, Athens, Greece (G.T., A.H.K.); Department of Neurology, International Clinical Research Center, St. Anne’s University Hospital in Brno, Brno, Czech Republic (G.T.); and Department of Neurology, School of Medicine (A.H.K
| |
Collapse
|
31
|
Burda BU, Holmer HK, Norris SL. Limitations of A Measurement Tool to Assess Systematic Reviews (AMSTAR) and suggestions for improvement. Syst Rev 2016; 5:58. [PMID: 27072548 PMCID: PMC4830078 DOI: 10.1186/s13643-016-0237-1] [Citation(s) in RCA: 60] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/15/2015] [Accepted: 04/05/2016] [Indexed: 01/04/2023] Open
Abstract
A Measurement Tool to Assess Systematic Reviews (AMSTAR) is a commonly used tool to assess the quality of systematic reviews; however, modifications are needed to improve its usability, reliability, and validity. In this commentary, we summarize our experience and the experiences of others who have used AMSTAR and provide suggestions for its improvement. We propose that AMSTAR should modify a number of individual items and their instructions and responses to make them more congruent with an assessment of the methodologic quality of systematic reviews. We recommend adding new items and modifying existing items to assess the quality of the body of evidence and to address subgroup and sensitivity analyses. More detailed instructions are needed for scoring individual items across multiple reviewers, and we recommend that a total score should not be calculated. These suggestions need to be empirically tested prior to implementation.
Collapse
Affiliation(s)
- Brittany U Burda
- Kaiser Permanente Center for Health Research, 3800 N. Interstate Ave, Portland, OR, 97227, USA.
| | - Haley K Holmer
- Oregon Health and Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239, USA
| | - Susan L Norris
- World Health Organization, Av. Appia 20, CH-1211, Geneva, 27, Switzerland
| |
Collapse
|
32
|
Abstract
Rupture of an intracranial aneurysm is the most common cause of subarachnoid haemorrhage (SAH), which is a life-threatening acute cerebrovascular event that typically affects working-age people. The exact prevalence of unruptured intracranial aneurysms (UIAs) is unknown, but at least one in 20 to 30 adults is likely to carry an asymptomatic UIA. Approximately one quarter of these UIAs rupture in a lifetime. Complex methodological challenges in conducting studies of epidemiology and risk factors for UIAs and SAH might have led to conclusions being drawn on the basis of epidemiological data of variable quality. We believe that, as a result, misconceptions about UIAs and SAH may have arisen. In this Perspectives article, we discuss three possible misconceptions about the epidemiology of UIAs and SAH, and suggest how the quality of future research could be improved.
Collapse
Affiliation(s)
- Miikka Korja
- Department of Neurosurgery, University of Helsinki and Helsinki University Hospital, P.O. BOX 266, FI-00029 HUS, Finland
| | - Jaakko Kaprio
- Department of Public Health, University of Helsinki, P.O. BOX 41, FI-00014 Helsinki, Finland
| |
Collapse
|
33
|
Affiliation(s)
- Matthew J Shepard
- Department of Neurological Surgery, University of Virginia, Charlottesville, Virginia
| | - W Jeffrey Elias
- Department of Neurological Surgery, University of Virginia, Charlottesville, Virginia
| |
Collapse
|
34
|
Wasiak J, Shen AY, Tan HB, Mahar R, Kan G, Khoo WR, Faggion CM. Methodological quality assessment of paper-based systematic reviews published in oral health. Clin Oral Investig 2015; 20:399-431. [PMID: 26589200 DOI: 10.1007/s00784-015-1663-5] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2015] [Accepted: 11/11/2015] [Indexed: 12/18/2022]
Abstract
OBJECTIVES This study aimed to conduct a methodological assessment of paper-based systematic reviews (SR) published in oral health using a validated checklist. A secondary objective was to explore temporal trends on methodological quality. MATERIAL AND METHODS Two electronic databases (OVID Medline and OVID EMBASE) were searched for paper-based SR of interventions published in oral health from inception to October 2014. Manual searches of the reference lists of paper-based SR were also conducted. Methodological quality of included paper-based SR was assessed using an 11-item questionnaire, Assessment of Multiple Systematic Reviews (AMSTAR) checklist. Methodological quality was summarized using the median and inter-quartile range (IQR) of the AMSTAR score over different categories and time periods. RESULTS A total of 643 paper-based SR were included. The overall median AMSTAR score was 4 (IQR 2-6). The highest median score (5) was found in the pain dentistry and periodontology fields, while the lowest median score (3) was found in implant dentistry, restorative dentistry, oral medicine, and prosthodontics. The number of paper-based SR per year and the median AMSTAR score increased over time (median score in 1990s was 2 (IQR 2-3), 2000s was 4 (IQR 2-5), and 2010 onwards was 5 (IQR 3-6)). CONCLUSION Although the methodological quality of paper-based SR published in oral health has improved in the last few years, there is still scope for improving quality in most evaluated dental specialties. CLINICAL RELEVANCE Large-scale assessment of methodological quality of dental SR highlights areas of methodological strengths and weaknesses that can be targeted in future publications to encourage better quality review methodology.
Collapse
Affiliation(s)
- J Wasiak
- Epworth Healthcare, Richmond, VIC, Australia.,School of Public Health and Preventive Medicine, The Alfred Centre, Monash University, Melbourne, Australia.,Faculty of Medicine, Dentistry and Health Sciences, Melbourne Dental School, Melbourne, Australia
| | - A Y Shen
- Eastern Health, C/O - Box Hill Hospital, Box Hill, Melbourne, Australia.
| | - H B Tan
- Alfred Health, Melbourne, Australia
| | - R Mahar
- School of Population Health, University of Queensland, Brisbane, Australia
| | - G Kan
- Melbourne Health, Melbourne, Australia
| | - W R Khoo
- Southern Health, Melbourne, Australia
| | - C M Faggion
- Department of Periodontology, Faculty of Dentistry, University of Münster, Münster, Germany
| |
Collapse
|
35
|
Evaniew N, van der Watt L, Bhandari M, Ghert M, Aleem I, Drew B, Guyatt G. Strategies to improve the credibility of meta-analyses in spine surgery: a systematic survey. Spine J 2015; 15:2066-76. [PMID: 26002725 DOI: 10.1016/j.spinee.2015.05.018] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/23/2014] [Revised: 03/09/2015] [Accepted: 05/13/2015] [Indexed: 02/03/2023]
Abstract
BACKGROUND CONTEXT Meta-analyses are powerful tools that can synthesize existing research, inform clinical practice, and support evidence-based care. These studies have become increasingly popular in the spine surgery literature, but the rigor with which they are being conducted has not yet been evaluated. PURPOSE Our primary objectives were to evaluate the methodological quality (credibility) of spine surgery meta-analyses and to propose strategies to improve future research. Our secondary objectives were to evaluate completeness of reporting and identify factors associated with higher credibility and completeness of reporting. STUDY DESIGN This study is based on a systematic survey of meta-analyses. OUTCOME MEASURES We evaluated credibility according to the Users' Guide to the Medical Literature and completeness of reporting according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist. METHODS We systematically searched MEDLINE, EMBASE, and The Cochrane Library, and two reviewers independently assessed eligibility, credibility, and completeness of reporting. We used multivariable linear regression to evaluate potential associations. Interrater agreement was quantified using kappa and intraclass correlation (ICC) coefficients. RESULTS We identified 132 eligible meta-analyses of spine surgery interventions. The mean credibility score was 3 of 7 (standard deviation [SD], 1.4; ICC, 0.86), with agreement for each item ranging from 0.54 (moderate) to 0.83 (almost perfect). Clinical questions were judged as sensible in 125 (95%), searches were exhaustive in 102 (77%), and risk of bias assessments were undertaken in 91 (69%). Seven (5%) meta-analyses addressed possible explanations for heterogeneity using a priori subgroup hypotheses and 24 (18%) presented results that were immediately clinically applicable. Investigators undertook duplicate assessments of eligibility, risk of bias, and data extraction in 46 (35%) and rated overall confidence in the evidence in 24 (18%). Later publication year, increasing Journal Impact Factor, increasing number of databases, inclusion of Randomized Controlled Trials, and inclusion of non-English studies were significantly associated with higher credibility scores (p<.05). The mean score for reporting was 18 of 27 (SD, 4.4; ICC, 0.94). CONCLUSIONS The credibility of many current spine surgery meta-analyses is limited. Researchers can improve future meta-analyses by performing exhaustive literature searches, addressing possible explanations of heterogeneity, presenting results in a clinically useful manner, reproducibly selecting and assessing primary studies, addressing confidence in the pooled effect estimates, and adhering to guidelines for complete reporting.
Collapse
Affiliation(s)
- Nathan Evaniew
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7.
| | - Leon van der Watt
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| | - Mohit Bhandari
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| | - Michelle Ghert
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| | - Ilyas Aleem
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| | - Brian Drew
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| | - Gordon Guyatt
- McMaster University, Department of Surgery, Division of Orthopaedics, 293 Wellington St. N, Hamilton ON, Canada, L8L 8E7
| |
Collapse
|
36
|
Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One 2015; 10:e0125931. [PMID: 25938454 PMCID: PMC4418838 DOI: 10.1371/journal.pone.0125931] [Citation(s) in RCA: 63] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2015] [Accepted: 03/25/2015] [Indexed: 01/10/2023] Open
Abstract
Background Previous research looking at published systematic reviews has shown that their search strategies are often suboptimal and that librarian involvement, though recommended, is low. Confidence in the results, however, is limited due to poor reporting of search strategies the published articles. Objectives To more accurately measure the use of recommended search methods in systematic reviews, the levels of librarian involvement, and whether librarian involvement predicts the use of recommended methods. Methods A survey was sent to all authors of English-language systematic reviews indexed in the Database of Abstracts of Reviews of Effects (DARE) from January 2012 through January 2014. The survey asked about their use of search methods recommended by the Institute of Medicine, Cochrane Collaboration, and the Agency for Healthcare Research and Quality and if and how a librarian was involved in the systematic review. Rates of use of recommended methods and librarian involvement were summarized. The impact of librarian involvement on use of recommended methods was examined using a multivariate logistic regression. Results 1560 authors completed the survey. Use of recommended search methods ranged widely from 98% for use of keywords to 9% for registration in PROSPERO and were generally higher than in previous studies. 51% of studies involved a librarian, but only 64% acknowledge their assistance. Librarian involvement was significantly associated with the use of 65% of recommended search methods after controlling for other potential predictors. Odds ratios ranged from 1.36 (95% CI 1.06 to 1.75) for including multiple languages to 3.07 (95% CI 2.06 to 4.58) for using controlled vocabulary. Conclusions Use of recommended search strategies is higher than previously reported, but many methods are still under-utilized. Librarian involvement predicts the use of most methods, but their involvement is under-reported within the published article.
Collapse
|
37
|
Cahill KS, McCormick PC, Levi AD. A comprehensive assessment of the risk of bone morphogenetic protein use in spinal fusion surgery and postoperative cancer diagnosis. J Neurosurg Spine 2015; 23:86-93. [PMID: 25860517 DOI: 10.3171/2014.10.spine14338] [Citation(s) in RCA: 49] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
The risk of postoperative cancer following the use of recombinant human bone morphogenetic protein (BMP)-2 in spinal fusion is one potential complication that has received significant interest. Until recently, there has been little clinical evidence to support the assertion of potential cancer induction after BMP use in spinal surgery. This report aims to summarize the findings from clinical data available to date from the Yale University Open Data Access (YODA) project as well as more recently published large database studies regarding the association of BMP use in spinal fusion and the risk of postoperative cancer. A detailed review was based on online databases, primary studies, FDA reports, and bibliographies of key articles for studies that assessed the efficacy and safety of BMP in spinal fusion. In an analysis of the YODA project, one meta-analysis detected a statistically significant increase in cancer occurrence at 24 months but not at 48 months, and the other meta-analysis did not detect a significant increase in postoperative cancer occurrence. Analysis of 3 large health care data sets (Medicare, MarketScan, and PearlDiver) revealed that none were able to detect a significant increase in risk of malignant cancers when BMP was used compared with controls. The potential risk of postoperative cancer formation following the use of BMP in spinal fusion must be interpreted on an individual basis for each patient by the surgeon. There is no conclusive evidence that application of the common formulations of BMP during spinal surgery results in the formation of cancer locally or at a distant site.
Collapse
Affiliation(s)
- Kevin S Cahill
- Carolina Neurosurgery and Spine Associates, Charlotte, North Carolina
| | - Paul C McCormick
- Department of Neurological Surgery, Columbia University Medical Center, New York, New York; and
| | - Allan D Levi
- Department of Neurological Surgery and The Miami Project to Cure Paralysis, University of Miami Miller School of Medicine, Miami, Florida
| |
Collapse
|
38
|
|
39
|
Dey M, Stadnik A, Riad F, Zhang L, McBee N, Kase C, Carhuapoma JR, Ram M, Lane K, Ostapkovich N, Aldrich F, Aldrich C, Jallo J, Butcher K, Snider R, Hanley D, Ziai W, Awad IA. Bleeding and infection with external ventricular drainage: a systematic review in comparison with adjudicated adverse events in the ongoing Clot Lysis Evaluating Accelerated Resolution of Intraventricular Hemorrhage Phase III (CLEAR-III IHV) trial. Neurosurgery 2015; 76:291-300; discussion 301. [PMID: 25635887 PMCID: PMC4333009 DOI: 10.1227/neu.0000000000000624] [Citation(s) in RCA: 62] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Retrospective series report varied rates of bleeding and infection with external ventricular drainage (EVD). There have been no prospective studies of these risks with systematic surveillance, threshold definitions, or independent adjudication. OBJECTIVE To analyze the rate of complications in the ongoing Clot Lysis: Evaluating Accelerated Resolution of Intraventricular Hemorrhage Phase III (CLEAR III) trial, providing a comparison with a systematic review of complications of EVD in the literature. METHODS Patients were prospectively enrolled in the CLEAR III trial after placement of an EVD for obstructive intraventricular hemorrhage and randomized to receive recombinant tissue-type plasminogen activator or placebo. We counted any detected new hemorrhage (catheter tract hemorrhage or any other distant hemorrhage) on computed tomography scan within 30 days from the randomization. Meta-analysis of published series of EVD placement was compiled with STATA software. RESULTS Growing or unstable hemorrhage was reported as a cause of exclusion from the trial in 74 of 5707 cases (1.3%) screened for CLEAR III. The first 250 patients enrolled have completed adjudication of adverse events. Forty-two subjects (16.8%) experienced ≥1 new bleeds or expansions, and 6 of 250 subjects (2.4%) suffered symptomatic hemorrhages. Eleven cases (4.4%) had culture-proven bacterial meningitis or ventriculitis. CONCLUSION Risks of bleeding and infection in the ongoing CLEAR III trial are comparable to those previously reported in EVD case series. In the present study, rates of new bleeds and bacterial meningitis/ventriculitis are very low despite multiple daily injections, blood in the ventricles, the use of thrombolysis in half the cases, and generalization to >60 trial sites.
Collapse
Affiliation(s)
- Mahua Dey
- ‡Section of Neurosurgery and Neurovascular Surgery Program, Division of Biological Sciences and the Pritzker School of Medicine, University of Chicago, Chicago, Illinois; §Pritzker School of Medicine, University of Chicago, Chicago, Illinois; ¶Johns Hopkins Medicine, Baltimore, Maryland; ‖Boston Medical Center, Boston, Massachusetts; #Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland; **University of Maryland, Baltimore, Maryland; ‡‡Thomas Jefferson University, Philadelphia, Pennsylvania; §§University of Alberta, Edmonton, Alberta, Canada; ¶¶Stanford University, Stanford, California
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
40
|
Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol 2015; 68:617-26. [PMID: 25766056 DOI: 10.1016/j.jclinepi.2014.11.025] [Citation(s) in RCA: 154] [Impact Index Per Article: 17.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2014] [Revised: 10/24/2014] [Accepted: 11/19/2014] [Indexed: 01/08/2023]
Abstract
OBJECTIVES To determine whether librarian and information specialist authorship was associated with better reported systematic review (SR) search quality. STUDY DESIGN AND SETTING SRs from high-impact general internal medicine journals were reviewed for search quality characteristics and reporting quality by independent reviewers using three instruments, including a checklist of Institute of Medicine Recommended Standards for the Search Process and a scored modification of the Peer Review of Electronic Search Strategies instrument. RESULTS The level of librarian and information specialist participation was significantly associated with search reproducibility from reported search strategies (Χ(2) = 23.5; P < 0.0001). Librarian co-authored SRs had significantly higher odds of meeting 8 of 13 analyzed search standards than those with no librarian participation and six more than those with mentioned librarian participation. One-way ANOVA showed that differences in total search quality scores between all three groups were statistically significant (F2,267 = 10.1233; P < 0.0001). CONCLUSION Problems remain with SR search quality and reporting. SRs with librarian or information specialist co-authors are correlated with significantly higher quality reported search strategies. To minimize bias in SRs, authors and editors could encourage librarian engagement in SRs including authorship as a potential way to help improve documentation of the search strategy.
Collapse
Affiliation(s)
- Melissa L Rethlefsen
- Spencer S. Eccles Health Sciences Library, University of Utah, 10 N. 1900 E. Salt Lake City, UT 84112, USA.
| | - Ann M Farrell
- Mayo Clinic Libraries, Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, USA
| | | | - Tara J Brigham
- Mayo Clinic Libraries, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL 32224, USA
| |
Collapse
|
41
|
Al-Khouja LT, Baron EM, Johnson JP, Kim TT, Drazin D. Cost-effectiveness analysis in minimally invasive spine surgery. Neurosurg Focus 2015; 36:E4. [PMID: 24881636 DOI: 10.3171/2014.4.focus1449] [Citation(s) in RCA: 44] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
OBJECT Medical care has been evolving with the increased influence of a value-based health care system. As a result, more emphasis is being placed on ensuring cost-effectiveness and utility in the services provided to patients. This study looks at this development in respect to minimally invasive spine surgery (MISS) costs. METHODS A literature review using PubMed, the Cost-Effectiveness Analysis (CEA) Registry, and the National Health Service Economic Evaluation Database (NHS EED) was performed. Papers were included in the study if they reported costs associated with minimally invasive spine surgery (MISS). If there was no mention of cost, CEA, cost-utility analysis (CUA), quality-adjusted life year (QALY), quality, or outcomes mentioned, then the article was excluded. RESULTS Fourteen studies reporting costs associated with MISS in 12,425 patients (3675 undergoing minimally invasive procedures and 8750 undergoing open procedures) were identified through PubMed, the CEA Registry, and NHS EED. The percent cost difference between minimally invasive and open approaches ranged from 2.54% to 33.68%-all indicating cost saving with a minimally invasive surgical approach. Average length of stay (LOS) for minimally invasive surgery ranged from 0.93 days to 5.1 days compared with 1.53 days to 12 days for an open approach. All studies reporting EBL reported lower volume loss in an MISS approach (range 10-392.5 ml) than in an open approach (range 55-535.5 ml). CONCLUSIONS There are currently an insufficient number of studies published reporting the costs of MISS. Of the studies published, none have followed a standardized method of reporting and analyzing cost data. Preliminary findings analyzing the 14 studies showed both cost saving and better outcomes in MISS compared with an open approach. However, more Level I CEA/CUA studies including cost/QALY evaluations with specifics of the techniques utilized need to be reported in a standardized manner to make more accurate conclusions on the cost effectiveness of minimally invasive spine surgery.
Collapse
|
42
|
Reviewing the reviewers: Appraisal of the United States Veterans Health Administration report on femtosecond laser–assisted cataract surgery. J Cataract Refract Surg 2015; 41:237-9. [DOI: 10.1016/j.jcrs.2014.11.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2014] [Indexed: 11/21/2022]
|
43
|
Abstract
OBJECT Observational studies, such as cohort and case-control studies, are valuable instruments in evidence-based medicine. Case-control studies, in particular, are becoming increasingly popular in the neurosurgical literature due to their low cost and relative ease of execution; however, no one has yet systematically assessed these types of studies for quality in methodology and reporting. METHODS The authors performed a literature search using PubMed/MEDLINE to identify all studies that explicitly identified themselves as "case-control" and were published in the JNS Publishing Group journals (Journal of Neurosurgery, Journal of Neurosurgery: Pediatrics, Journal of Neurosurgery: Spine, and Neurosurgical Focus) or Neurosurgery. Each paper was evaluated for 22 descriptive variables and then categorized as having either met or missed the basic definition of a case-control study. All studies that evaluated risk factors for a well-defined outcome were considered true case-control studies. The authors sought to identify key features or phrases that were or were not predictive of a true case-control study. Those papers that satisfied the definition were further evaluated using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist. RESULTS The search detected 67 papers that met the inclusion criteria, of which 32 (48%) represented true case-control studies. The frequency of true case-control studies has not changed with time. Use of odds ratios (ORs) and logistic regression (LR) analysis were strong positive predictors of true case-control studies (for odds ratios, OR 15.33 and 95% CI 4.52-51.97; for logistic regression analysis, OR 8.77 and 95% CI 2.69-28.56). Conversely, negative predictors included focus on a procedure/intervention (OR 0.35, 95% CI 0.13-0.998) and use of the word "outcome" in the Results section (OR 0.23, 95% CI 0.082-0.65). After exclusion of nested case-control studies, the negative correlation between focus on a procedure/intervention and true case-control studies was strengthened (OR 0.053, 95% CI 0.0064-0.44). There was a trend toward a negative association between the use of survival analysis or Kaplan-Meier curves and true case-control studies (OR 0.13, 95% CI 0.015-1.12). True case-control studies were no more likely than their counterparts to use a potential study design "expert" (OR 1.50, 95% CI 0.57-3.95). The overall average STROBE score was 72% (range 50-86%). Examples of reporting deficiencies were reporting of bias (28%), missing data (55%), and funding (44%). CONCLUSIONS The results of this analysis show that the majority of studies in the neurosurgical literature that identify themselves as "case-control" studies are, in fact, labeled incorrectly. Positive and negative predictors were identified. The authors provide several recommendations that may reverse the incorrect and inappropriate use of the term "case-control" and improve the quality of design and reporting of true case-control studies in neurosurgery.
Collapse
|