1
|
Hu ZJ, Fusch G, Hu C, Wang JY, El Helou Z, Hassan MT, Mbuagbaw L, El Helou S, Thabane L. Completeness of reporting of quality improvement studies in neonatology is inadequate: a systematic literature survey. BMJ Open Qual 2021; 10:bmjoq-2020-001273. [PMID: 34127453 PMCID: PMC8204179 DOI: 10.1136/bmjoq-2020-001273] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 06/02/2021] [Indexed: 01/10/2023] Open
Abstract
INTRODUCTION Quality improvement (QI) is a growing field of inquiry in healthcare, but the reporting quality of QI studies in neonatology remains unclear. We conducted a systematic survey of the literature to assess the reporting quality of QI studies and factors associated with reporting quality. METHODS We searched Medline for publications of QI studies from 2016 to 16 April 2020. Pairs of reviewers independently screened citations and assessed reporting quality using a 31-item modified Standards for Quality Improvement Reporting Excellence, 2nd edition (SQUIRE 2.0) checklist. We reported the number (percentage) of studies that reported each item and their corresponding 95% CIs. We used Poisson regression to explore factors associated with reporting quality, namely, journal endorsement of SQUIRE 2.0, declaration of funding sources, year of publication and number of authors. The results were reported as incidence rate ratio (IRR) and 95% CI. RESULTS Of 1921 citations, 336 were eligible; among them, we randomly selected 100 articles to assess reporting quality. The mean (standard deviation) number of SQUIRE 2.0 items adhered to was 22.0 (4.5). Percentage of articles reporting each item varied from 26% to 100%. Journal endorsement of SQUIRE 2.0 (IRR=1.11, 95% CI 1.02 to 1.21, p=0.015), declaration of funding sources and increasing number of authors were significantly associated with better reporting. CONCLUSIONS Reporting quality of QI studies in neonatology is inadequate. Endorsing the SQUIRE 2.0 guideline is a step that journals can implement to enhance the completeness of reporting.
Collapse
Affiliation(s)
- Zheng Jing Hu
- Department of Health Research Methods Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Gerhard Fusch
- Division of Neonatology, Department of Pediatrics, McMaster University, Hamilton, Ontario, Canada
| | - Catherine Hu
- Bachelor of Arts and Science, McMaster University, Hamilton, Ontario, Canada
| | - Jie Yi Wang
- Bachelor of Medical Sciences, Schulich School of Medicine & Dentistry, University of Western Ontario, London, Ontario, Canada
| | - Zoe El Helou
- Bachelor of Health Sciences, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Muhammad Taaha Hassan
- Bachelor of Health Sciences, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Lawrence Mbuagbaw
- Department of Health Research Methods Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Salhab El Helou
- Division of Neonatology, Department of Pediatrics, McMaster University, Hamilton, Ontario, Canada
| | - Lehana Thabane
- Department of Health Research Methods Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
2
|
Elliott L, Coulman K, Blencowe NS, Qureshi MI, Lee KS, Hinchliffe RJ, Mouton R. A systematic review of reporting quality for anaesthetic interventions in randomised controlled trials. Anaesthesia 2021; 76:832-836. [PMID: 33150618 PMCID: PMC8246731 DOI: 10.1111/anae.15294] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/21/2020] [Indexed: 12/31/2022]
Abstract
Interventions from randomised controlled trials can only be replicated if they are reported in sufficient detail. The results of trials can only be confidently interpreted if the delivery of the intervention was systematic and the protocol adhered to. We systematically reviewed trials of anaesthetic interventions published in 12 journals from January 2016 to September 2019. We assessed the detail with which interventions were reported, using the Consolidated Standards of Reporting Trials statement for non-pharmacological treatments. We analysed 162 interventions reported by 78 trials in 18,675 participants. Detail sufficiently precise to replicate the intervention was reported for 111 (69%) interventions. Intervention standardisation was reported for 135 (83%) out of the 162 interventions, and protocol adherence was reported for 20 (12%) interventions. Sixty (77%) out of the 78 trials reported the administrative context in which interventions were delivered and 36 (46%) trials detailed the expertise of the practitioners. We conclude that bespoke reporting tools should be developed for anaesthetic interventions and interventions in other areas such as critical care.
Collapse
Affiliation(s)
- L. Elliott
- Department of AnaesthesiaBristol Centre for Surgical ResearchBristolUK
| | - K. Coulman
- Department of Vascular SurgeryBristol Centre for Surgical ResearchBristolUK
| | - N. S. Blencowe
- Department of Vascular SurgeryBristol Centre for Surgical ResearchBristolUK
| | - M. I. Qureshi
- Department of Vascular SurgeryBristol Centre for Surgical ResearchBristolUK
| | - K. S. Lee
- Bristol Centre for Surgical ResearchBristolUK
| | | | | |
Collapse
|
3
|
Shanthanna H. High thoracic ESP blockade for shoulder surgery: reporting on the consistency and completeness of blockade. PAIN MEDICINE 2021; 22:3103-3104. [PMID: 33839779 DOI: 10.1093/pm/pnab138] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Harsha Shanthanna
- Department of Anesthesia, McMaster University, 1280 Main St. W, , Hamilton, Ontario, L8S 4K1, Canada
| |
Collapse
|
4
|
Mbuagbaw L, Lawson DO, Puljak L, Allison DB, Thabane L. A tutorial on methodological studies: the what, when, how and why. BMC Med Res Methodol 2020; 20:226. [PMID: 32894052 PMCID: PMC7487909 DOI: 10.1186/s12874-020-01107-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 08/27/2020] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste. MAIN BODY We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies? CONCLUSION Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.
Collapse
Affiliation(s)
- Lawrence Mbuagbaw
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.
- Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph's Healthcare-Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada.
- Centre for the Development of Best Practices in Health, Yaoundé, Cameroon.
| | - Daeria O Lawson
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Livia Puljak
- Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia
| | - David B Allison
- Department of Epidemiology and Biostatistics, School of Public Health - Bloomington, Indiana University, Bloomington, IN, 47405, USA
| | - Lehana Thabane
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph's Healthcare-Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada
- Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON, Canada
- Centre for Evaluation of Medicine, St. Joseph's Healthcare-Hamilton, Hamilton, ON, Canada
- Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada
| |
Collapse
|
5
|
Shaqman M, Al-Abedalla K, Wagner J, Swede H, Gunsolley JC, Ioannidou E. Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes. PLoS One 2020; 15:e0230843. [PMID: 32302309 PMCID: PMC7164582 DOI: 10.1371/journal.pone.0230843] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Accepted: 03/10/2020] [Indexed: 12/21/2022] Open
Abstract
OBJECTIVE Poor reporting in randomized clinical trial (RCT) abstracts reduces quality and misinforms readers. Spin, a biased presentation of findings, could frequently mislead clinicians to accept a clinical intervention despite non-significant primary outcome. Therefore, good reporting practices and absence of spin enhances research quality. We aim to assess the reporting quality and spin in abstracts of RCTs evaluating the effect of periodontal therapy on cardiovascular (CVD) outcomes. METHODS PubMed, Scopus, the Cochrane Central Register of Controlled Trials (CENTRAL), and 17 trial registration platforms were searched. Cohort, non-randomized, non-English studies, and pediatric studies were excluded. RCT abstracts were reviewed by 2 authors using the CONSORT for abstracts and spin checklists for data extraction. Cohen's Kappa statistic was used to assess inter-rater agreement. Data on the selected RCT publication metrics were collected. Descriptive analysis was performed with non-parametric methods. Correlation analysis between quality, spin and bibliometric parameters was conducted. RESULTS 24 RCTs were selected for CONSORT analysis and 14 fulfilled the criteria for spin analysis. Several important RCT elements per CONSORT were neglected in the abstract including description of the study population (100%), explicitly stated primary outcome (87%), methods of randomization and blinding (100%), trial registration (87%). No RCT examined true outcomes (CVD events). A significant fraction of the abstracts appeared with at least one form of spin in the results and conclusions (86%) and claimed some treatment benefit in spite of non-significant primary outcome (64%). High-quality reporting had a significant positive correlation with reporting of trial registration (p = 0.04) and funding (p = 0.009). Spinning showed marginal negative correlation with reporting quality (p = 0.059). CONCLUSION Poor adherence to the CONSORT guidelines and high levels of data spin were found in abstracts of RCTs exploring the effects of periodontal therapy on CVD outcomes. Our findings indicate that journal editors and reviewers should consider strict adherence to proper reporting guidelines to improve reporting quality and reduce waste.
Collapse
Affiliation(s)
- Murad Shaqman
- Department of Oral Maxillofacial Surgery, Oral Medicine and Periodontology, School of Dentistry, The University of Jordan, Amman, Jordan
| | - Khadijeh Al-Abedalla
- Division of Periodontology, School of Dental Medicine, UCONN Health, Farmington, CT, United States of America
| | - Julie Wagner
- Division of Behavioral Science, School of Dental Medicine, UCONN Health, Farmington, CT, United States of America
| | - Helen Swede
- Department of Community Medicine, School of Medicine, UCONN Health, Farmington, CT, United States of America
| | - John Cart Gunsolley
- Department of Periodontics, School of Dentistry, Virginia Commonwealth University, Richmond, VA, United States of America
| | - Effie Ioannidou
- Division of Periodontology, School of Dental Medicine, UCONN Health, Farmington, CT, United States of America
| |
Collapse
|
6
|
Lawson DO, Leenus A, Mbuagbaw L. Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review. Pilot Feasibility Stud 2020; 6:13. [PMID: 32699641 PMCID: PMC7003412 DOI: 10.1186/s40814-019-0544-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2019] [Accepted: 12/20/2019] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs. OBJECTIVES This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews. METHODS Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval). RESULTS Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely. CONCLUSIONS Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.
Collapse
Affiliation(s)
- Daeria O. Lawson
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4K1 Canada
| | - Alvin Leenus
- Faculty of Health Sciences, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4K1 Canada
| | - Lawrence Mbuagbaw
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4K1 Canada
- Biostatistics Unit, Father Sean O’Sullivan Research Centre, St. Joseph’s Healthcare Hamilton, Hamilton, ON L8N 4A6 Canada
| |
Collapse
|
7
|
Elliott L, Coulman K, Blencowe NS, Qureshi M, Watson S, Mouton R, Hinchliffe RJ. Protocol for a systematic review of reporting standards of anaesthetic interventions in randomised controlled trials. BMJ Open 2020; 10:e034372. [PMID: 31937656 PMCID: PMC7045251 DOI: 10.1136/bmjopen-2019-034372] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2019] [Revised: 12/03/2019] [Accepted: 12/18/2019] [Indexed: 11/03/2022] Open
Abstract
INTRODUCTION There is significant variation in how anaesthesia is defined and reported in clinical research. This lack of standardisation complicates the interpretation of published evidence and planning of future clinical trials. This systematic review will assess the reporting of anaesthesia as an intervention in randomised controlled trials (RCT) against the Consolidated Standards of Reporting Trials for Non-Pharmacological Treatments (CONSORT-NPT) framework. METHODS AND ANALYSIS Online archives of the top six journals ranked by impact factor for anaesthesia and the top three general medicine and general surgery journals will be systematically hand searched over a 42-month time period to identify RCTs describing the use of anaesthetic interventions for any invasive procedure. All modes of anaesthesia and anaesthesia techniques will be included. All study data, including the type of anaesthetic intervention described, will be extracted in keeping with the CONSORT-NPT checklist. Descriptive statistics will be used to summarise general study details including types/modes of anaesthetic interventions, and reporting standards of the trials. ETHICS AND DISSEMINATION No ethical approval is required. The results will be used to inform a funding application to formally standardise general, local, regional anaesthesia and sedation for use in clinical research. The systematic review will be disseminated via peer-reviewed manuscript and conferences. PROSPERO REGISTRATION NUMBER CRD42019141670.
Collapse
Affiliation(s)
- Lucy Elliott
- Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
| | - Karen Coulman
- Population Health Sciences, University of Bristol Medical School, Bristol, UK
| | - Natalie S Blencowe
- Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
| | - Mahim Qureshi
- Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
| | - Sethina Watson
- Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| | - Ronelle Mouton
- Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
- Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| | - Robert J Hinchliffe
- Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
- Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| |
Collapse
|
8
|
Mbuagbaw L, Kosa SD, Lawson DO, Stalteri R, Olaiya OR, Alotaibi A, Thabane L. The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study. Pilot Feasibility Stud 2019; 5:120. [PMID: 31700654 PMCID: PMC6827233 DOI: 10.1186/s40814-019-0500-z] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Accepted: 09/16/2019] [Indexed: 11/10/2022] Open
Abstract
Introduction Pilot and feasibility trials are conducted to determine feasibility or to collect information that would inform the design of a larger definitive trial. Clear progression criteria are required to determine if a definitive or main trial is feasible and how it should be designed. We sought to determine how often progression criteria are reported and the associated factors. Methods We conducted a methodological review of protocols for pilot randomised trials published in three journals that publish research protocols (BMJ Open, Trials, Pilot and Feasibility Studies), using a PubMed search (2013–2017). We extracted bibliometric information including the country in which the study was conducted, source of funding, type of intervention, use of a primary feasibility outcome, sample size reporting, and justification. We used generalised linear models to determine the factors associated with reporting progression criteria. Results Our search retrieved 276 articles, of which 49 were not eligible. We included 227 articles. Overall, 45/227 (19.8%; 95% confidence interval [CI] 14.8–25.6) reported progression criteria. Protocols published in more recent years were significantly associated with higher odds of reporting progression criteria (adjusted odds ratio [aOR] 1.40; 95% CI 1.03–1.92; p = 0.034). Pilot trials from Europe (aOR 0.19; 95% CI 0.08–0.48; p < 0.001) and the rest of the world (aOR 0.05; 95% CI 0.01–0.18; p < 0.003) compared to North America were significantly associated with lower odds of reporting progression criteria. Journal, source of funding, sample size, intervention type, and having a primary outcome related to feasibility were not significantly associated with reporting progression criteria. Conclusion Progression criteria are not often explicitly stated in protocols of pilot trials leaving room for varied interpretation of findings. The development of formal guidance for progression criteria in protocols of pilot trials is warranted.
Collapse
Affiliation(s)
- Lawrence Mbuagbaw
- 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada.,2Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare-Hamilton, 50 Charlton Avenue East, 3rd Floor Martha Wing, Room H321, Hamilton, ON L8N 4A6 Canada.,Centre for the Development of Best Practices in Health, Yaoundé, Cameroon
| | - Sarah Daisy Kosa
- 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada.,4Toronto General Hospital, University Health Network, Toronto, ON Canada
| | - Daeria O Lawson
- 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada
| | - Rosa Stalteri
- 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada
| | - Oluwatobi R Olaiya
- 5Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON Canada
| | - Ahlam Alotaibi
- 6Department of Pediatrics, McMaster University, Hamilton, ON Canada
| | - Lehana Thabane
- 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada.,2Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare-Hamilton, 50 Charlton Avenue East, 3rd Floor Martha Wing, Room H321, Hamilton, ON L8N 4A6 Canada.,7Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON Canada.,8Centre for Evaluation of Medicine, St Joseph's Healthcare-Hamilton, Hamilton, ON Canada.,9Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON Canada
| |
Collapse
|
9
|
Charlesworth M, Klein AA, White SM. A bibliometric analysis of the conversion and reporting of pilot studies published in six anaesthesia journals. Anaesthesia 2019; 75:247-253. [DOI: 10.1111/anae.14817] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/25/2019] [Indexed: 12/17/2022]
Affiliation(s)
- M. Charlesworth
- Department of Cardiothoracic Anaesthesia Critical Care and ECMO Wythenshawe Hospital ManchesterUK
| | - A. A. Klein
- Department of Cardiothoracic Anaesthesia and Intensive Care Royal Papworth Hospital CambridgeUK
| | - S. M. White
- Department of Anaesthesia Royal Sussex County Hospital Brighton UK
| |
Collapse
|