Quality of reporting in systematic reviews of adverse events: systematic review

Towards the automatic risk of bias assessment on randomized controlled trials: A comparison of RobotReviewer and humans

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two different approaches: (1) manually by human reviewers, and (2) automatically by the RobotReviewer. The manual assessment was based on two groups independently, with two additional rounds of verification. The agreement between RobotReviewer and humans was measured via the concordance rate and Cohen's kappa statistics, based on the comparison of binary classification of the risk of bias (low vs. high/unclear) as restricted by RobotReviewer. The concordance rates varied by domain, ranging from 63.07% to 83.32%. Cohen's kappa statistics showed a poor agreement between humans and RobotReviewer for allocation concealment (κ = 0.25, 95% CI: 0.21-0.30), blinding of outcome assessors (κ = 0.27, 95% CI: 0.23-0.31); While moderate for random sequence generation (κ = 0.46, 95% CI: 0.41-0.50) and blinding of participants and personnel (κ = 0.59, 95% CI: 0.55-0.64). The findings demonstrate that there were domain-specific differences in the level of agreement between RobotReviewer and humans. We suggest that it might be a useful auxiliary tool, but the specific manner of its integration as a complementary tool requires further discussion.

Pediatric massage therapy in infants and children under 5 years: An umbrella review of systematic reviews

Objective

To systematically evaluate and synthesize quantitative evidence regarding the effects of pediatric massage in infants and children under five years.

Review methods

We conducted searches in databases including MEDLINE, Embase, Health Technology Assessment Database, the Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Allied and Complementary Medicine, Embase, the China National Knowledge Infrastructure, Wanfang Data, SinoMed, and CQVIP up to February 2024. Two reviewers independently screened articles, extracted data, and conducted quality appraisals on the included studies. We focused on systematic reviews with meta-analyses comparing pediatric tuina with a waitlist control, placebo, medication, massage combined with other interventions, and usual care. Outcomes analyzed included physical, psychological, developmental, and safety-related measures for children and their caregivers. Methodological quality was assessed using AMSTAR 2, and evidence quality was evaluated using the GRADE approach. Pooled effect estimates and heterogeneity were extracted for each meta-analyzed outcome. Evidence evaluated as 'high' and 'moderate' quality by GRADEpro with low heterogeneity are presented in the result section.

Results

Twenty-two systematic reviews with meta-analysis of 81 outcomes were included. Seven studies (31.8 %) were high quality, and the evidence of 6 (7.4 %) outcomes were evaluated as high quality. The meta-analysis results demonstrate significant benefits of pediatric massage for infants compared to routine care, with increased weight gain (MD 455.07 g; 95 % CI 86.33 to 823.8; I2 = 0 %) based on 2 studies with 157 cases, length growth (MD 1.58 cm; 95 % CI 1.42 to 1.74; I2 = 25 %) from 9 studies with 1294 cases, reduced fussing (MD -0.37 time; 95 % CI -0.53 to -0.21; I2 = 30 %) according to 3 studies with 271 cases, and lower post-intervention bilirubin levels (3 studies/345 cases; MD -31.75 mmol/L; 95 % CI -40.05 to -23.46; I2 = 0 %). Pediatric massage was associated with reduced diarrhea incidence compared to waitlist controls (2 RCTs/310 cases; RR 0.39; 95 % CI 0.2 to 0.76; I2 = 0 %) and improved psychomotor development indices (4 RCTs/466 cases; SMD -0.35; 95 % CI -0.54 to -0.15; I2 = 1.06 %). Additionally, pediatric massage significantly enhanced blood oxygen saturation variation post-procedure for pain management (5 RCTs/355 cases; MD 1.05; 95 % CI 0.51 to 1.58; I2 = 0 %).

Conclusions

Pediatric massage significantly enhances mother-child attachment, promotes physical growth, reduces fussing, lowers bilirubin levels, improves motor and psychomotor development, as well as manages digestive conditions and pain in children under 5 years. Future research should improve study quality and comprehensively report adverse events.

Development and Evaluation of a Framework for Identifying and Addressing Spin for Harms in Systematic Reviews of Interventions

"Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified.

The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews.

The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications.

Accelerating evidence synthesis for safety assessment through ClinicalTrials.gov platform: a feasibility study

BACKGROUND

Standard systematic review can be labor-intensive and time-consuming meaning that it can be difficult to provide timely evidence when there is an urgent public health emergency such as a pandemic. The ClinicalTrials.gov provides a promising way to accelerate evidence production.

METHODS

We conducted a search on PubMed to gather systematic reviews containing a minimum of 5 studies focused on safety aspects derived from randomized controlled trials (RCTs) of pharmacological interventions, aiming to establish a real-world dataset. The registration information of each trial from eligible reviews was further collected and verified. The meta-analytic data were then re-analyzed by using 1) the full meta-analytic data with all trials and 2) emulated rapid data with trials that had been registered and posted results on ClinicalTrials.gov, under the same synthesis methods. The effect estimates of the full meta-analysis and rapid meta-analysis were then compared.

RESULTS

The real-world dataset comprises 558 meta-analyses. Among them, 56 (10.0%) meta-analyses included RCTs that were not registered in ClinicalTrials.gov. For the remaining 502 meta-analyses, the median percentage of RCTs registered within each meta-analysis is 70.1% (interquartile range: 33.3% to 88.9%). Under a 20% bias threshold, rapid meta-analyses conducted through ClinicalTrials.gov achieved accurate point estimates ranging from 77.4% (using the MH model) to 83.1% (using the GLMM model); 91.0% to 95.3% of these analyses accurately predicted the direction of effects.

CONCLUSIONS

Utilizing the ClinicalTrials.gov platform for safety assessment with a minimum of 5 RCTs holds significant potential for accelerating evidence synthesis to support urgent decision-making.

AI as a Medical Device Adverse Event Reporting in Regulatory Databases: Protocol for a Systematic Review

BACKGROUND

The reporting of adverse events (AEs) relating to medical devices is a long-standing area of concern, with suboptimal reporting due to a range of factors including a failure to recognize the association of AEs with medical devices, lack of knowledge of how to report AEs, and a general culture of nonreporting. The introduction of artificial intelligence as a medical device (AIaMD) requires a robust safety monitoring environment that recognizes both generic risks of a medical device and some of the increasingly recognized risks of AIaMD (such as algorithmic bias). There is an urgent need to understand the limitations of current AE reporting systems and explore potential mechanisms for how AEs could be detected, attributed, and reported with a view to improving the early detection of safety signals.

OBJECTIVE

The systematic review outlined in this protocol aims to yield insights into the frequency and severity of AEs while characterizing the events using existing regulatory guidance.

METHODS

Publicly accessible AE databases will be searched to identify AE reports for AIaMD. Scoping searches have identified 3 regulatory territories for which public access to AE reports is provided: the United States, the United Kingdom, and Australia. AEs will be included for analysis if an artificial intelligence (AI) medical device is involved. Software as a medical device without AI is not within the scope of this review. Data extraction will be conducted using a data extraction tool designed for this review and will be done independently by AUK and a second reviewer. Descriptive analysis will be conducted to identify the types of AEs being reported, and their frequency, for different types of AIaMD. AEs will be analyzed and characterized according to existing regulatory guidance.

RESULTS

Scoping searches are being conducted with screening to begin in April 2024. Data extraction and synthesis will commence in May 2024, with planned completion by August 2024. The review will highlight the types of AEs being reported for different types of AI medical devices and where the gaps are. It is anticipated that there will be particularly low rates of reporting for indirect harms associated with AIaMD.

CONCLUSIONS

To our knowledge, this will be the first systematic review of 3 different regulatory sources reporting AEs associated with AIaMD. The review will focus on real-world evidence, which brings certain limitations, compounded by the opacity of regulatory databases generally. The review will outline the characteristics and frequency of AEs reported for AIaMD and help regulators and policy makers to continue developing robust safety monitoring processes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/48156.

Evaluating the efficacy and safety of therapeutic interventions for corneal neuropathy: A systematic review

Corneal neuropathy involves corneal nerve damage that disrupts ocular surface integrity, negatively impacting quality-of-life from pain and impaired vision. Any ocular or systemic condition that damages the trigeminal nerve can lead to corneal neuropathy. However, the condition currently does not have standardized diagnostic criteria or treatment protocols. The primary aim of this systematic review was to evaluate the efficacy and safety of interventions for treating corneal neuropathy. Randomized controlled trials (RCTs) that investigated corneal neuropathy treatments were eligible if the intervention(s) was compared to a placebo or active comparator. Comprehensive searches were conducted in Ovid MEDLINE, Ovid Embase and clinical trial registries from inception to July 2022. The Cochrane Risk-of-Bias 2 tool was used to assess study methodological quality. Certainty of the body of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Overall, 20 RCTs were included. Evaluated interventions comprised regenerative therapies (n = 6 studies), dietary supplements (n = 4), anti-glycemic agents (n = 3), combination therapy (n = 3), supportive therapies (n = 2) and systemic pain pharmacotherapies (n = 2). Nine RCTs were judged at high risk of bias for most outcomes. Definitions for corneal neuropathy in the populations varied substantially across studies, consistent with lack of consensus on diagnostic criteria. A diverse range of outcomes were quantified, likely reflecting absence of an agreed core outcome set. There was insufficient evidence to draw definitive conclusions on the efficacy or safety of any intervention. There was low or very low certainty evidence for several neuroregenerative agents and dietary supplements for improving corneal nerve fiber length in corneal neuropathy due to dry eye disease and diabetes. Low or very low certainty evidence was found for neuroregenerative therapies and dietary supplements not altering corneal immune cell density. This review identifies a need to standardize the clinical definition of corneal neuropathy and define a minimum set of core outcome measures. Together, this will provide a foundation for improved phenotyping of clinical populations in studies, and improve the capacity to synthesize data to inform evidence-based care. Protocol registration: PROSPERO ID: CRD42022348475.

Exploring the relationship between the number of systematic reviews and quality of evidence: an orthognathic surgery-based study

OBJECTIVE

We analyzed the quality and quantity of systematic reviews (SRs) of orthognathic surgery, the most frequently published topic in maxillofacial surgery.

STUDY DESIGN

We searched the PubMed database for SRs of orthognathic surgery with no restriction on the language of publication date. We assessed the certainty of evidence presented according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol and the Leiden Manifesto using the Grading of Recommendations Assessment, Development, and Evaluation approach. We analyzed the data using descriptive statistics, Pearson´s correlation test, and linear regression.

RESULTS

Of the 171 SRs evaluated, approximately one fifth presented evidence with a high level of certainty. The number of orthognathic surgery SRs has been increasing, and many SRs were published after very similar topics had already been published. There is no relationship between the impact factor and the certainty of evidence.

CONCLUSIONS

An excessive number of SRs of orthognathic surgery are published, and many SRs are superfluous, simply reporting previous findings. Clinicians should not base treatment decisions solely on the evidence presented in SRs, and journal editors and reviewers should evaluate these SRs more critically, particularly when they address topics that have already been covered in the literature.

The SMART Safety: An empirical dataset for evidence synthesis of adverse events

Evidence synthesis serves an important role to promote informed decision-making in healthcare practice. A key issue of evidence synthesis is the approach to deal with rare adverse events and the methods to address bias of harm effects. Empirical data is essential to help methodologists and statisticians to solve the issues in evidence synthesis of adverse events. For this reason, we have established SMART Safety dataset, the largest empirical dataset of meta-analyses of adverse events. The dataset contains 151 systematic reviews with 629 meta-analyses on safety outcomes, which covers more than 2,300 randomized controlled trials and 362 harm outcomes, with 10,069 rows and 45 columns of trial level information. All information was double- or even quadra-checked and further verified by referring the original source (e.g., the full-text of the included randomized trials) to ensure high validity of the data.

Justification for the 2017 periodontitis classification in the light of the Checklist for Modifying Disease Definitions: A narrative review

Once a while, disease classifications have needed revision because new knowledge has accumulated, and new technologies and better treatments have emerged. Changes made to disease classifications should be trustworthy and openly justified. The periodontitis definition and classification system was changed in 2017 by the 'World Workshop on the Classification of Periodontal and Peri-Implant Diseases and Conditions'. The workshop, comprising clinicians and researchers, resulted in the production of a 23-article special issue that introduced the new definitions and classifications of periodontitis. In this narrative review, we critically review how the changes made to the periodontitis definition and classification were justified in the light of the Checklist for Modifying Disease Definitions. Under each of the eight items of the checklist, we have discussed how the item was or could have been considered in the light of the checklist and its guidance. In our view, the new definition and classification of periodontitis was presented in an understandable way, even though the changes from the previous definition were not made visible. However, the issues of (1) estimated changes in prevalence or incidence, (2) triggers for the change, (3) prognostic ability, (4) repeatability or reproducibility, (5) incremental benefits, (6) incremental harms or (7) net benefits and harms related to the introduction of new classification were not considered in the way suggested in the checklist. Thereby, a balanced assessment of potential benefits and harms associated with the new periodontitis classification system was not presented, and to a large extent it remains unknown if the use of the new classification system will provide more net benefits to patients and to the community than previous systems. It is our view that patients and societies deserve transparent and balanced assessments of the potential benefits and harms associated with the periodontitis classification. Importantly, these should reflect the values and preferences also of the patients and the wider community and consider the impact on resource usage.

Harms Reporting in Systematic Reviews of the Microvascular Free Flap in Head and Neck Reconstruction

OBJECTIVE

To evaluate harms reporting in systematic reviews (SRs) of microvascular free flap (MFF) in head and neck reconstruction.

DATA SOURCES

This cross-sectional analysis included searches from the following major databases from 2012 to June 1, 2022: MEDLINE (Pubmed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews.

REVIEW METHODS

In a masked duplicate manner, screening was performed using Rayyan, and data were extracted using a pilot-tested Google form. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of reviews and the corrected covered area was calculated to detect primary study overlap across all reviews. Reviews were then grouped in pairs of 2, called dyads, and the corrected covered area was calculated again for each individual dyad. Dyads with high overlap (≥50%) were further investigated for the accuracy of harms reporting.

RESULTS

Our initial search yielded 268 records, with 50 SRs meeting the inclusion criteria. A total of 46 (92%) of the included reviews demonstrated 50% or more adherence to the items assessed in our harms checklist. Our corrected covered area tool revealed 0.6% primary study overlap across all reviews, and 1 dyad with high overlap (≥50%). No statistically significant relationship was observed between the completeness of harms reporting and reviews listing harms as a primary outcome, reviews reporting adherence to Preferred Reporting Items of Systematic Reviews and Meta-Analyses, or a review's AMSTAR rating.

CONCLUSION

This study identifies how harms reporting in SRs of MFF reconstruction of the head and neck can be improved and provides suggestions with the potential to mitigate the paucity in current literature.

Comparison of statistical methods for integrating real-world evidence in a rare events meta-analysis of randomized controlled trials

Rare events meta-analyses of randomized controlled trials (RCTs) are often underpowered because the outcomes are infrequent. Real-world evidence (RWE) from non-randomized studies may provide valuable complementary evidence about the effects of rare events, and there is growing interest in including such evidence in the decision-making process. Several methods for combining RCTs and RWE studies have been proposed, but the comparative performance of these methods is not well understood. We describe a simulation study that aims to evaluate an array of alternative Bayesian methods for including RWE in rare events meta-analysis of RCTs: the naïve data synthesis, the design-adjusted synthesis, the use of RWE as prior information, the three-level hierarchical models, and the bias-corrected meta-analysis model. The percentage bias, root-mean-square-error, mean 95% credible interval width, coverage probability, and power are used to measure performance. The various methods are illustrated using a systematic review to evaluate the risk of diabetic ketoacidosis among patients using sodium/glucose co-transporter 2 inhibitors as compared with active-comparators. Our simulations show that the bias-corrected meta-analysis model is comparable to or better than the other methods in terms of all evaluated performance measures and simulation scenarios. Our results also demonstrate that data solely from RCTs may not be sufficiently reliable for assessing the effects of rare events. In summary, the inclusion of RWE could increase the certainty and comprehensiveness of the body of evidence of rare events from RCTs, and the bias-corrected meta-analysis model may be preferable.

Is acupuncture safe in the ICU? A systematic review and meta-analysis

Background and purpose

The safety of interventions for critically ill patients is a crucial issue. In recent years, several studies have treated critically ill patients with acupuncture. However, the safety of acupuncture in this setting remains to be systematically measured.

Methods

In May 2022, the electronic databases of PubMed and the Cochrane Library were searched for studies comparing acupuncture interventions to control interventions in critically ill patients. Study outcomes examined the incidence of severe adverse events (AEs), minor AEs, adverse reactions, ICU stays, and 28-day mortality.

Results

A total of 31 articles were analyzed, and no serious AEs related to acupuncture treatment were identified. No significant differences were found between the groups in the meta-analysis of minor AEs (risk ratio [RR] 5.69 [0.34, 96.60], P = 0.23, I2 = 76%). A reduced risk in the incidence of adverse reactions following acupuncture intervention was evidenced (RR 0.33 [0.22, 0.50], P = 0.00001, I2 = 44%). The patients in the acupuncture arm spent significantly less time in the intensive care unit (ICU) (Mean difference -1.45 [-11.94, -10.97], P = 0.00001, I2 = 56%) and also exhibited lower 28-day mortality rates (odds ratio 0.61 [0.48, 0.78], P = 0.0001, I2 = 0%).

Conclusion

There is no evidence to indicate a higher risk of severe or minor AEs in patients who receive acupuncture. Acupuncture demonstrated favorable results in both ICU stay and 28-day mortality measurements, in addition to presenting with fewer adverse reactions compared to routine ICU care. However, the low certainty of the evidence resulting from a high risk of bias in the included studies merits substantial consideration, and further research is still warranted.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=142131, identifier: CRD42020142131.

Evaluating the impact of including non-randomised studies of interventions in meta-analysis of randomised controlled trials: a protocol for a meta-epidemiological study

INTRODUCTION

Although interest in including non-randomised studies of interventions (NRSIs) in meta-analysis of randomised controlled trials (RCTs) is growing, estimates of effectiveness obtained from NRSIs are vulnerable to greater bias than RCTs. The objectives of this study are to: (1) explore how NRSIs can be integrated into a meta-analysis of RCTs; (2) assess concordance of the evidence from non-randomised and randomised trials and explore factors associated with agreement; and (3) investigate the impact on estimates of pooled bodies of evidence when NRSIs are included.

METHODS AND ANALYSIS

We will conduct a systematic survey of 210 systematic reviews that include both RCTs and NRSIs, published from 2017 to 2022. We will randomly select reviews, stratified in a 1:1 ratio by Core vs non-Core clinical journals, as defined by the National Library of Medicine. Teams of paired reviewers will independently determine eligibility and abstract data using standardised, pilot-tested forms. The concordance of the evidence will be assessed by exploring agreement in the relative effect reported by NRSIs and RCT addressing the same clinical question, defined as similarity of the population, intervention/exposure, control and outcomes. We will conduct univariable and multivariable logistic regression analyses to examine the association of prespecified study characteristics with agreement in the estimates between NRSIs and RCTs. We will calculate the ratio of the relative effect estimate from NRSIs over that from RCTs, along with the corresponding 95% CI. We will use a bias-corrected meta-analysis model to investigate the influence on pooled estimates when NRSIs are included in the evidence synthesis.

ETHICS AND DISSEMINATION

Ethics approval is not required. The findings of this study will be disseminated through peer-reviewed publications, conference presentations and condensed summaries for clinicians, health policymakers and guideline developers regarding the design, conduct, analysis, and interpretation of meta-analysis that integrate RCTs and NRSIs.

Seeking adverse effects in systematic reviews of orthodontic interventions: a cross-sectional study (part 1)

BACKGROUND

Systematic reviews that assess the benefits of interventions often do not completely capture all dimensions of the adverse effects. This cross-sectional study (part 1 of 2 studies) assessed whether adverse effects were sought, whether the findings on these effects were reported, and what types of adverse effects were identified in systematic reviews of orthodontic interventions.

METHODS

Systematic reviews of orthodontic interventions on human patients of any health status, sex, age, and demographics, and socio-economic status, in any type of setting assessing any type of adverse effect scored at any endpoint or timing were eligible. The Cochrane Database of Systematic Reviews and 5 leading orthodontic journals were manually searched for eligible reviews between August 1 2009 and July 31 2021. Study selection and data extraction was conducted by two researchers independently. Prevalence proportions were calculated for four outcomes on seeking and reporting of adverse effects of orthodontic interventions. Univariable logistic regression models were used to determine the association between each one of these outcomes and the journal in which the systematic review was published using the eligible Cochrane reviews as reference.

RESULTS

Ninety-eight eligible systematic reviews were identified. 35.7% (35/98) of reviews defined seeking of adverse effects as a research objective, 85.7% (84/98) sought adverse effects, 84.7% (83/98) reported findings related to adverse effects, and 90.8% (89/98) considered or discussed potential adverse effects in the review. Reviews in the journal Orthodontics and Craniofacial Research compared with Cochrane reviews had approximately 7 times the odds (OR 7.20, 95% CI 1.08 to 47.96) to define seeking of adverse effects in the research objectives. Five of the 12 categories of adverse effects accounted for 83.1% (162/195) of all adverse effects sought and reported.

CONCLUSIONS

Although the majority of included reviews sought and reported adverse effects of orthodontic interventions, end-users of these reviews should beware that these findings do not give the complete spectrum on these effects and that they could be jeopardized by the risk of non-systematically assessing and reporting of adverse effects in these reviews and in the primary studies that feed them. Much research is ahead such as developing core outcome sets on adverse effects of interventions for both primary studies and systematic reviews.

Spin on adverse effects in abstracts of systematic reviews of orthodontic interventions: a cross-sectional study (part 2)

BACKGROUND

It is critical that abstracts of systematic reviews transparently report both the beneficial and adverse effects of interventions without misleading the readers. This cross-sectional study assessed whether adverse effects of interventions were reported or considered in abstracts of systematic reviews of orthodontic interventions and whether spin on adverse effects was identified when comparing the abstracts with what was sought and reported in these reviews.

METHODS

This cross-sectional study (part 2 of 2) used the same sample of 98 systematic reviews orthodontic interventions as used in part 1. Eligible reviews were retrieved from the Cochrane Database of Systematic Reviews and the 5 leading orthodontic journals between August 1 2009 and July 31 2021. Prevalence proportions were sought for 3 outcomes as defined in the published protocol. Univariable logistic regression models were built to explore associations between the presence of spin in the abstract and a series of predictors. Odds ratios (OR) 95% confidence intervals (95% CI) were used to quantify the strength of associations and their precision.

RESULTS

76.5% (75/98) of eligible reviews reported or considered (i.e., discussed, weighted etc.) potential adverse effects of orthodontic interventions in the abstract and the proportion of spin on adverse effects was 40.8% (40/98) in the abstract of these reviews. Misleading reporting was the predominant category of spin, i.e., 90% (36/40). Our explorative analyses found that compared to the Cochrane Database of Systematic Reviews all 5 orthodontic journals had similar odds of the presence of spin on adverse effects in abstracts of systematic reviews of orthodontic interventions. The odds of the presence of spin did not change over the sampled years (OR: 1.03, 95% CI: 0.9 to 1.16) and did not depend on the number of authors (OR: 0.93, 95% CI: 0.71 to 1.21), or on the type of orthodontic intervention (OR: 1.1, 95% CI: 0.45 to 2.67), or whether conflicts of interests were reported (OR: 0.74, 95% CI: 0.32 to 1.68).

CONCLUSION

End users of systematic reviews of orthodontic interventions have to be careful when interpreting results on adverse effects in the abstracts of these reviews, because they could be jeopardized by uncertainties such as not being reported and misleading reporting as a result of spin.

The (cost-)effectiveness of combined lifestyle interventions for people with persistent low-back pain who are overweight or obese: A systematic review

INTRODUCTION

Lifestyle factors are expected to contribute to the persistence and burden of low-back pain (LBP). However, there are no systematic reviews on the (cost-)effectiveness of combined lifestyle interventions for overweight or obese people with LBP.

AIM

To assess whether combined lifestyle interventions are (cost-)effective for people with persistent LBP who are overweight or obese, based on a systematic review.

DESIGN

Systematic review METHOD: PubMed, Cochrane, Embase, CINAHL, PsycINFO and the Wiley/Cochrane Library were searched from database inception till January 6^th 2023. Two independent reviewers performed study selection, data-extraction and risk of bias scoring using the Cochrane RoB tool 2 and/or the Consensus Health Economic Criteria list. GRADE was used to assess the level of certainty of the evidence.

RESULTS

In total 2510 records were screened, and 4 studies on 3 original RCTs with 216 participants were included. Low certainty evidence (1 study) showed that combined lifestyle interventions were not superior to usual care for physical functioning, pain and lifestyle outcomes. Compared to usual care, moderate certainty evidence showed that healthcare (-$292, 95%CI: 872; -33), medication (-$30, 95% CI -65; -4) and absenteeism costs (-$1000, 95%CI: 3573; -210) were lower for the combined lifestyle interventions.

CONCLUSION

There is low certainty evidence from 3 studies with predominantly small sample sizes, short follow-up and low intervention adherence that combined lifestyle interventions are not superior to physical functioning, pain and lifestyle outcomes compared to usual care, but are likely to be cost-effective.

Reporting of adverse events associated with spinal manipulation in randomised clinical trials: an updated systematic review

OBJECTIVES

To describe if there has been a change in the reporting of adverse events associated with spinal manipulation in randomised clinical trials (RCTs) since 2016.

DESIGN

A systematic literature review.

DATA SOURCES

Databases were searched from March 2016 to May 2022: MEDLINE (Ovid), Embase, CINAHL, ICL, PEDro and Cochrane Library. The following search terms and their derivatives were adapted for each platform: spinal manipulation; chiropractic; osteopathy; physiotherapy; naprapathy; medical manipulation and clinical trial.

METHODS

Domains of interest (pertaining to adverse events) included: completeness and location of reporting; nomenclature and description; spinal location and practitioner delivering manipulation; methodological quality of the studies and details of the publishing journal. Frequencies and proportions of studies reporting on each of these domains were calculated. Univariable and multivariable logistic regression models were fitted to examine the effect of potential predictors on the likelihood of studies reporting on adverse events.

RESULTS

There were 5399 records identified by the electronic searches, of which 154 (2.9%) were included in the analysis. Of these, 94 (61.0%) reported on adverse events with only 23.4% providing an explicit description of what constituted an adverse event. Reporting of adverse events in the abstract has increased (n=29, 30.9%) while reporting in the results section has decreased (n=83, 88.3%) over the past 6 years. Spinal manipulation was delivered to 7518 participants in the included studies. No serious adverse events were reported in any of these studies.

CONCLUSIONS

While the current level of reporting of adverse events associated with spinal manipulation in RCTs has increased since our 2016 publication on the same topic, the level remains low and inconsistent with established standards. As such, it is imperative for authors, journal editors and administrators of clinical trial registries to ensure there is more balanced reporting of both benefits and harms in RCTs involving spinal manipulation.

Systematic review: Safety of surgical male circumcision in context of HIV prevention public health programmes

Background: Since the recommendation of voluntary medical male circumcision (VMMC) to reduce the risk of heterosexually acquired HIV, a number of adolescent boys and men in 15 priority countries in Africa have been circumcised. Our primary goal was to identify the incidence of adverse events (AEs) associated with VMMC and to assess the safety profile among adolescent boys 10 - 14 years. Methods: We searched the databases MEDLINE and Embase, WHO, and conference abstracts from 2005 to 2019. The incidence of AEs was estimated by type of AE, size of study and age. Results: We retained 40 studies. Severe and moderate AEs overall were estimated at 0.30 per 100 VMMC clients with wide variability per study type. A higher rate was noted in small and moderate scale programmes and device method research studies compared with larger scale programmes. There was a limited number of studies reporting AEs among younger adolescent boys and they had higher infection-related AEs than those aged 20 years and older. Case studies noted rare AEs such as necrotizing fasciitis, tetanus, and glans injury. Conclusions: AE rates were comparable to those from the randomized controlled trials (RCTs) that led to recommendations and implementation of VMMC in high HIV burden countries, despite being implemented in low resource settings. Clients over time have increasingly included adolescents under the age of 15 years. Studies suggest potentially higher risks in this age group. As VMMC services are sustained, patient safety surveillance systems and promoting a patient safety culture are crucial to identify and mitigate potential harms from medical male circumcision.

Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses

Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.

Pharmacological treatments for low back pain in adults: an overview of Cochrane Reviews

BACKGROUND

Pharmacological interventions are the most used treatment for low back pain (LBP). Use of evidence from systematic reviews of the effects of pharmacological interventions for LBP published in the Cochrane Library, is limited by lack of a comprehensive overview.

OBJECTIVES

To summarise the evidence from Cochrane Reviews of the efficacy, effectiveness, and safety of systemic pharmacological interventions for adults with non-specific LBP.

METHODS

The Cochrane Database of Systematic Reviews was searched from inception to 3 June 2021, to identify reviews of randomised controlled trials (RCTs) that investigated systemic pharmacological interventions for adults with non-specific LBP. Two authors independently assessed eligibility, extracted data, and assessed the quality of the reviews and certainty of the evidence using the AMSTAR 2 and GRADE tools. The review focused on placebo comparisons and the main outcomes were pain intensity, function, and safety.

MAIN RESULTS

Seven Cochrane Reviews that included 103 studies (22,238 participants) were included. There is high confidence in the findings of five reviews, moderate confidence in one, and low confidence in the findings of another. The reviews reported data on six medicines or medicine classes: paracetamol, non-steroidal anti-inflammatory drugs (NSAIDs), muscle relaxants, benzodiazepines, opioids, and antidepressants. Three reviews included participants with acute or sub-acute LBP and five reviews included participants with chronic LBP. Acute LBP Paracetamol There was high-certainty evidence for no evidence of difference between paracetamol and placebo for reducing pain intensity (MD 0.49 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -1.99 to 2.97), reducing disability (MD 0.05 on a 0 to 24 scale (higher scores indicate worse disability), 95% CI -0.50 to 0.60), and increasing the risk of adverse events (RR 1.07, 95% CI 0.86 to 1.33). NSAIDs There was moderate-certainty evidence for a small between-group difference favouring NSAIDs compared to placebo at reducing pain intensity (MD -7.29 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -10.98 to -3.61), high-certainty evidence for a small between-group difference for reducing disability (MD -2.02 on a 0-24 scale (higher scores indicate worse disability), 95% CI -2.89 to -1.15), and very low-certainty evidence for no evidence of an increased risk of adverse events (RR 0.86, 95% CI 0. 63 to 1.18). Muscle relaxants and benzodiazepines There was moderate-certainty evidence for a small between-group difference favouring muscle relaxants compared to placebo for a higher chance of pain relief (RR 0.58, 95% CI 0.45 to 0.76), and higher chance of improving physical function (RR 0.55, 95% CI 0.40 to 0.77), and increased risk of adverse events (RR 1.50, 95% CI 1. 14 to 1.98). Opioids None of the included Cochrane Reviews aimed to identify evidence for acute LBP. Antidepressants No evidence was identified by the included reviews for acute LBP. Chronic LBP Paracetamol No evidence was identified by the included reviews for chronic LBP. NSAIDs There was low-certainty evidence for a small between-group difference favouring NSAIDs compared to placebo for reducing pain intensity (MD -6.97 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -10.74 to -3.19), reducing disability (MD -0.85 on a 0-24 scale (higher scores indicate worse disability), 95% CI -1.30 to -0.40), and no evidence of an increased risk of adverse events (RR 1.04, 95% CI -0.92 to 1.17), all at intermediate-term follow-up (> 3 months and ≤ 12 months postintervention). Muscle relaxants and benzodiazepines There was low-certainty evidence for a small between-group difference favouring benzodiazepines compared to placebo for a higher chance of pain relief (RR 0.71, 95% CI 0.54 to 0.93), and low-certainty evidence for no evidence of difference between muscle relaxants and placebo in the risk of adverse events (RR 1.02, 95% CI 0.67 to 1.57). Opioids There was high-certainty evidence for a small between-group difference favouring tapentadol compared to placebo at reducing pain intensity (MD -8.00 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -1.22 to -0.38), moderate-certainty evidence for a small between-group difference favouring strong opioids for reducing pain intensity (SMD -0.43, 95% CI -0.52 to -0.33), low-certainty evidence for a medium between-group difference favouring tramadol for reducing pain intensity (SMD -0.55, 95% CI -0.66 to -0.44) and very low-certainty evidence for a small between-group difference favouring buprenorphine for reducing pain intensity (SMD -0.41, 95% CI -0.57 to -0.26). There was moderate-certainty evidence for a small between-group difference favouring strong opioids compared to placebo for reducing disability (SMD -0.26, 95% CI -0.37 to -0.15), moderate-certainty evidence for a small between-group difference favouring tramadol for reducing disability (SMD -0.18, 95% CI -0.29 to -0.07), and low-certainty evidence for a small between-group difference favouring buprenorphine for reducing disability (SMD -0.14, 95% CI -0.53 to -0.25). There was low-certainty evidence for a small between-group difference for an increased risk of adverse events for opioids (all types) compared to placebo; nausea (RD 0.10, 95% CI 0.07 to 0.14), headaches (RD 0.03, 95% CI 0.01 to 0.05), constipation (RD 0.07, 95% CI 0.04 to 0.11), and dizziness (RD 0.08, 95% CI 0.05 to 0.11). Antidepressants There was low-certainty evidence for no evidence of difference for antidepressants (all types) compared to placebo for reducing pain intensity (SMD -0.04, 95% CI -0.25 to 0.17) and reducing disability (SMD -0.06, 95% CI -0.40 to 0.29).

AUTHORS' CONCLUSIONS

We found no high- or moderate-certainty evidence that any investigated pharmacological intervention provided a large or medium effect on pain intensity for acute or chronic LBP compared to placebo. For acute LBP, we found moderate-certainty evidence that NSAIDs and muscle relaxants may provide a small effect on pain, and high-certainty evidence for no evidence of difference between paracetamol and placebo. For safety, we found very low- and high-certainty evidence for no evidence of difference with NSAIDs and paracetamol compared to placebo for the risk of adverse events, and moderate-certainty evidence that muscle relaxants may increase the risk of adverse events. For chronic LBP, we found low-certainty evidence that NSAIDs and very low- to high-certainty evidence that opioids may provide a small effect on pain. For safety, we found low-certainty evidence for no evidence of difference between NSAIDs and placebo for the risk of adverse events, and low-certainty evidence that opioids may increase the risk of adverse events.

Measuring the impact of zero-cases studies in evidence synthesis practice using the harms index and benefits index (Hi-Bi)

OBJECTIVES

In evidence synthesis practice, dealing with studies with no cases in both arms has been a tough problem, for which there is no consensus in the research community. In this study, we propose a method to measure the potential impact of studies with no cases for meta-analysis results which we define as harms index (Hi) and benefits index (Bi) as an alternative solution for deciding how to deal with such studies.

METHODS

Hi and Bi are defined by the minimal number of cases added to the treatment arm (Hi) or control arm (Bi) of studies with no cases in a meta-analysis that lead to a change of the direction of the estimates or its statistical significance. Both exact and approximating methods are available to calculate Hi and Bi. We developed the "hibi" module in Stata so that researchers can easily implement the method. A real-world investigation of meta-analyses from Cochrane reviews was employed to evaluate the proposed method.

RESULTS

Based on Hi and Bi, our results suggested that 21.53% (Hi) to 26.55% (Bi) of Cochrane meta-analyses may be potentially impacted by studies with no cases, for which studies with no cases could not be excluded from the synthesis. The approximating method shows excellent specificity (100%) for both Hi and Bi, moderate sensitivity (68.25%) for Bi, and high sensitivity (80.61%) for Hi compared to the exact method.

CONCLUSIONS

The proposed method is practical and useful for systematic reviewers to measure whether studies with no cases impact the results of meta-analyses and may act as an alternative solution for review authors to decide whether to include studies with no events for the synthesis or not.

A cross-sectional analysis of harms reporting in systematic reviews evaluating laminectomy

Background Context

Laminectomy is a common vertebral decompression procedure that has multiple potential adverse events which are not always reported in SRs.

Purpose

To evaluate the completeness of harms reporting in systematic reviews (SRs) on laminectomy.

Study Design

Cross-sectional analysis.

Methods

Eligible studies were SRs that evaluated laminectomy for any indication. MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched in May 2022 to locate studies for inclusion. Screening and data extraction on harms reporting and study characteristics were performed in duplicate. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR pairs.

Results

We included 26 SRs comprising 426 primary studies. Most SRs studied laminectomy for spinal stenosis, declared harms as a secondary outcome, and lacked or did not mention funding. Two SRs completely omitted harms, 9 had between 0% and 50.0% completion of harms items, and 15 had more than 50.1% completion. AMSTAR-2 graded 25 SRs (25/26, 96.2%) as 'critically low' and 1 SR (1/26, 3.8%) as 'low'. We found a statistically significant association between completeness of harms reporting and outcome specification. No other associations were statistically significant. Three SR pairs had CCAs >50% and were compared for unique and shared harms.

Conclusions

The completeness of harms reporting in SRs was inadequate. Because SRs often serve as tools for constructing clinical practice guidelines and clinical decision making, improvements must be made to enhance and refine harms reporting.

Assessing the Reporting and Frequency of Harms in Systematic Reviews Focused on Minimally Invasive Hysterectomies: A Cross-sectional Analysis

OBJECTIVE

To assess the quality of harms reporting in systematic reviews (SRs) regarding minimally invasive hysterectomies (MIHs).

DATA SOURCES

We performed a cross-sectional analysis on SRs addressing MIHs to assess completeness of harms reporting. Hysterectomies are common gynecologic procedures and are associated with complications. Such adverse events can directly affect physician decision making and patient outcomes. Thus, it is important that SRs equally weigh the harms and benefits surrounding MIHs.

METHODS OF STUDY SELECTION

On May 15, 2022, we searched MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews for SRs with or without a meta-analysis on MIH for any indication. Eligible studies underwent full-text screening, data extraction, harms reporting assessment, and A Measurement Tool to Assess Systematic Reviews-2 quality assessment in a masked, duplicate fashion. The corrected covered area was calculated to indicate any overlap between SR dyads.

TABULATION, INTEGRATION, AND RESULTS

A total of 52 SRs met the inclusion criteria for data extraction. We found that >44 of included SRs (of 52; 84.6%) reported >50% of the harms items. Completeness of harms reporting was significantly associated with harms specification as a primary outcome (p <.05). The corrected covered area was 0.60%.

CONCLUSION

The harms reporting was more complete than hypothesized, but still had deficiencies throughout, such as inconsistent use of severity scales to classify harms. Even though MIH harms reporting in sample SRs was more complete than other fields of medicine, it was still suboptimal overall and requires refinement.

Reporting Quality of Oral TCM Systematic Reviews Based on the PRISMA Harms Checklist from 2013 to 2020

Background

Systematic reviews focusing on the effectiveness of different kinds of healthcare interventions have been widely published, but there were few guidelines for reporting safety concerns before 2016. The PRISMA harms checklist, which was published in 2016, can standardize reporting quality.

Objectives

To evaluate the safety information reporting quality of oral traditional Chinese medicine (TCM) in systematic reviews before and after the PRISMA harms checklist was published and to explore factors associated with better reporting.

Methods

We searched PubMed, the Cochrane Library, and Embase to identify all systematic reviews using oral TCM as interventions published before (from 2013 to 2015) and after (from 2017 to 2020) the PRISMA harms checklist was published. We used the PRISMA harms checklist to assess the quality of reporting of the safety information to included systematic reviews.

Results

In total, 200 systematic reviews were sampled from eligible studies published between 2013 and 2020. Reviews from 2016 were excluded. Scores on the PRISMA harms checklist (23 items) ranged from 0 to 12. A systematic reviews published after 2016 had better reporting quality compared with studies published before 2016 with regard to the title (P=0.03), results of individual studies (P=0.016), and risk of bias across studies (P=0.043). In all included systematic reviews of our study, the state conclusion in coherence with review findings was reported adequately with the proportion of adherence at 95%; other items had a reporting proportion ranging from 0% to 57%. The four essential reporting items of the PRISMA harms checklist also had a low reporting quality ranging from 0% to 4%.

Conclusions

Oral TCM systematic reviews reported inadequate safety information before and after the PRISMA harms checklist was published. This survey suggested that the PRISMA harms checklist should be recommended more to both original research and systematic review authors.

Harms reporting by systematic reviews for functional endoscopic sinus surgery: a cross-sectional analysis

OBJECTIVES

To evaluate the completeness of harms reporting in systematic reviews (SRs) pertaining to functional endoscopic sinus surgery (FESS).

METHODS

Using a cross-sectional study design, we performed a comprehensive search using MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and the Cochrane Database of Systematic Reviews databases for SRs regarding FESS on May 15th, 2022. Returns were screened and data were extracted in a masked, duplicate manner. Following established methodology, we extracted general study characteristics, harms items, and overall methodological quality for each SR in our sample. Corrected covered area (CCA) was calculated for SR dyads. For data analysis, using Stata 16.1 we performed a bivariate analysis between variables.

RESULTS

Fifty-five SR's were included in our sample after excluding 375 studies that did not meet our inclusion criteria. Of the included SRs, 19 (19/55, 34.5%) did not report harms and 39 (39/55, 70.9%) reported half of the harms items or fewer. Our study found that 23 (23/55, 41.8%) of SRs demonstrated a method of harms data collection, 26 (26/55, 47.3%) of SRs had patients available for harms analysis in their results, and 25 (25/55, 45.5%) of SRs had a balanced discussion of harms and benefits of FESS. Fifty-two SRs were appraised as "critically low" quality using AMSTAR-2. A significant association was found between completeness of harms reporting (Mahady) and whether harms were listed as a primary outcome. No other associations were statistically significant. Two SR dyads had CCAs between 20% and 50% overlap and were compared for unique and shared harms.

CONCLUSIONS

Our study demonstrates gaps in harms reporting regarding FESS in SRs. We recommend future studies implement guidelines such as the STROCCS guidelines or the harms extension of the PRISMA guidelines to improve harms reporting. Accurate harms reporting may advance patient safety and promote a more objective risk-benefit analysis for physicians and patients.

Impact of osteopathic manipulative techniques on the management of dizziness caused by neuro-otologic disorders: systematic review and meta-analysis

CONTEXT

Osteopathic manipulative treatment (OMT) has been utilized by osteopathic clinicians as primary or adjunctive management for dizziness caused by neuro-otologic disorders. To our knowledge, no current systematic reviews provide pooled estimates that evaluate the impact of OMT on dizziness.

OBJECTIVES

We aimed to systematically evaluate the effectiveness and safety of OMT and analogous techniques in the treatment of dizziness.

METHODS

We performed a literature search in CINAHL, Embase, MEDLINE, Allied and Complementary Medicine Database (AMED), EMCare, Physiotherapy Evidence Database (PEDro), PubMed, PsycINFO, Osteopathic Medicine Digital Library (OSTMED.DR), and Cochrane Central Register of Controlled Trials (CENTRAL) from inception to March 2021 for randomized controlled trials (RCTs) and prospective or retrospective observational studies of adult patients experiencing dizziness from neuro-otological disorders. Eligible studies compared the effectiveness of OMT or OMT analogous techniques with a comparator intervention, such as a sham manipulation, a different manual technique, standard of care, or a nonpharmacological intervention like exercise or behavioral therapy. Assessed outcomes included disability associated with dizziness, dizziness severity, dizziness frequency, risk of fall, improvement in quality of life (QOL), and return to work (RTW). Assessed harm outcomes included all-cause dropout (ACD) rates, dropouts due to inefficacy, and adverse events. The meta-analysis was based on the similarities between the OMT or OMT analogous technique and the comparator interventions. The risk of bias (ROB) was assessed utilizing a modified version of the Cochrane Risk of Bias Tool for RCTs and the Cochrane Risk of Bias in Non-randomized Studies - of Interventions (ROBINS-I) for observational studies. The quality of evidence was determined utilizing the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach.

RESULTS

There were 3,375 studies identified and screened, and the full text of 47 of them were reviewed. Among those, 12 (11 RCTs, 1 observational study, n=367 participants) met the inclusion criteria for data extraction. Moderate-quality evidence showed that articular OMT techniques were associated with decreases (all p<0.01) in disability associated with dizziness (n=141, mean difference [MD]=-11, 95% confidence interval [CI]=-16.2 to -5.9), dizziness severity (n=158, MD=-1.6, 95% CI=-2.4 to -0.7), and dizziness frequency (n=136, MD=-0.6, 95% CI=-1.1 to -0.2). Low-quality evidence showed that articular OMT was not associated with ACD rates (odds ratio [OR]=2.2, 95% CI=0.5 to 10.2, p=0.31). When data were pooled for any type of OMT technique, findings were similar; however, disability associated with dizziness and ACD rates had high heterogeneity (I²=59 and 46%). No studies met all of the criteria for ROB.

CONCLUSIONS

The current review found moderate-quality evidence that treatment with articular OMT techniques was significantly associated with decreased disability associated with dizziness, dizziness severity, and dizziness frequency. However, our findings should be interpreted cautiously because of the high ROB and small sample sizes in the eligible studies.

An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions

Introduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events. Methods: We used the dataset of systematic reviews with at least one meta-analysis including both RCTs and NRSIs and collected the 2 × 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (RORs) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR). Results: We included systematic reviews with 178 meta analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to that of RCTs was estimated to be 0.96 (95% confidence interval: 0.87 and 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase in sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly. Discussion: There was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered a supplement to RCTs for safety assessment.

Systematic review: Safety of surgical male circumcision in context of HIV prevention public health programmes

Background: Since the recommendation of voluntary medical male circumcision (VMMC) to reduce the risk of heterosexually acquired HIV, a number of adolescent boys and men in 15 priority countries in Africa have been circumcised. Our primary goal was to identify the incidence of adverse events (AEs) associated with VMMC and to assess the safety profile among adolescent boys 10 - 14 years. Methods: We searched the databases MEDLINE and Embase, WHO, and conference abstracts from 2005 to 2019. The incidence of AEs was estimated by type of AE, size of study and age. Results: We retained 40 studies. Severe and moderate AEs overall were estimated at 0.30 per 100 VMMC clients with wide variability per study type. A higher rate was noted in small and moderate scale programmes and device method research studies compared with larger scale programmes. There was a limited number of studies reporting AEs among younger adolescent boys and they had higher infection-related AEs than those aged 20 years and older. Case studies noted rare AEs such as necrotizing fasciitis, tetanus, and glans injury. Conclusions: AE rates were comparable to those from the randomized controlled trials (RCTs) that led to recommendations and implementation of VMMC in high HIV burden countries, despite being implemented in low resource settings. Clients over time have increasingly included adolescents under the age of 15 years. Studies suggest potentially higher risks in this age group. As VMMC services are sustained, patient safety surveillance systems and promoting a patient safety culture are crucial to identify and mitigate potential harms from medical male circumcision.

Biases in reporting of adverse effects in clinical trials, and potential impact on safety assessments in systematic reviews and therapy guidelines

BACKGROUND

Clinical trials are an important source of adverse effects data, including analyses in systematic reviews and recommendations in therapy guidelines. Trial publication bias may have profound effects on safety perceptions. This MiniReview presents and discusses biases in reporting of safety data in clinical trials and the implications for systematic reviews and guidelines.

OBJECTIVES

The objectives of this work are to analyse risk of gastrointestinal bleeding in systemic corticosteroid trials and to assess adverse effects reporting in a fluoxetine trial in depression (Treatment for Adolescents With Depression Study [TADS]) and descriptions of adverse effects in adolescent depression therapy guidelines.

METHODS

We performed literature reviews and descriptive analyse of clinical trials with corticosteroids, and publications from the TADS trial. Risk of gastrointestinal bleeding from corticosteroids was analysed by meta-analysis.

FINDINGS

Gastrointestinal bleeding definitions varied considerably between trials. The incidence was significantly increased in hospitalized, but not in ambulant, patients compared to placebo. We identified several biases concerning TADS safety reporting, including severity thresholds and nonpublication of most adverse effects data beyond the initial 12 weeks. Therapy guidelines on adolescent depression mentioned suicidality risk, but many failed to mention other adverse effects.

CONCLUSIONS

We identified several pitfalls in adverse effects reporting in clinical trials. These include heterogeneous disease definitions, reporting thresholds, and incomplete reporting. Trial bias may have great impact on risk assessments in systematic reviews and meta-analyses.

Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials

In clinical trials, harms (i.e., adverse events) are often reported by simply counting the number of people who experienced each event. Reporting only frequencies ignores other dimensions of the data that are important for stakeholders, including severity, seriousness, rate (recurrence), timing, and groups of related harms. Additionally, application of selection criteria to harms prevents most from being reported. Visualization of data could improve communication of multidimensional data. We replicated and compared the characteristics of 6 different approaches for visualizing harms: dot plot, stacked bar chart, volcano plot, heat map, treemap, and tendril plot. We considered binary events using individual participant data from a randomized trial of gabapentin for neuropathic pain. We assessed their value using a heuristic approach and a group of content experts. We produced all figures using R and share the open-source code on GitHub. Most original visualizations propose presenting individual harms (e.g., dizziness, somnolence) alone or alongside higher level (e.g., by body systems) summaries of harms, although they could be applied at either level. Visualizations can present different dimensions of all harms observed in trials. Except for the tendril plot, all other plots do not require individual participant data. The dot plot and volcano plot are favored as visualization approaches to present an overall summary of harms data. Our value assessment found the dot plot and volcano plot were favored by content experts. Using visualizations to report harms could improve communication. Trialists can use our provided code to easily implement these approaches.

The Prevalence of Adverse Drug Reactions and Adverse Drug Events from Heart Failure Medications in Frail Older Adults: A Systematic Review

Introduction

Frailty is highly prevalent in heart failure populations and a major risk factor for adverse drug reactions (ADRs) and adverse drug events (ADEs). This review aimed to describe the prevalence, causality and severity of ADRs or ADEs from heart failure medications among frail compared with non-frail older adults.

Methods

A systematic search of CENTRAL, MEDLINE, Embase, Ageline, CINAHL, International Pharmaceutical Abstracts, PsychInfo, Scopus, registries and citations prior to 18 May 2021 was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 checklist. Risk of bias and quality of evidence were assessed. Eligible studies included randomised controlled trials (RCTs) and observational studies of people diagnosed with heart failure, aged ≥ 65 years, with frailty defined by an objective measurement, and reported ADRs/ADEs from/with heart failure medications.

Results

Two reviewers screened 2419 articles; interrater reliability kappa = 0.88. Three observational studies (n = 2596), a secondary analysis of two RCTs (n = 2098) and two cohort studies (n = 498) were included in a narrative synthesis. Frail patients in randomised trials of sacubitril/valsartan, aliskiren, or enalapril had twice the risk of mortality (hazard ratio [HR] 2.09, 1.62–2.71) and hospitalisations (HR 1.82, 1.37–2.41) compared with robust patients, which may reflect responsiveness to medications and/or factors unrelated to medication use. Hospitalisations from falls, tiredness and nausea were probably attributable to digoxin and possibly preventable according to the Naranjo and Hallas scales, respectively.

Conclusion

The potential harms from heart failure medications in frail older people are poorly studied and understood. Clinical trials and pharmacovigilance studies should include frailty as a covariate to inform medication optimisation for this vulnerable and growing population.

Registration

Prospero registration number: CRD 42021253762.

Graphical Abstract

Supplementary Information

The online version contains supplementary material available at 10.1007/s40266-022-00957-8.

Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study

OBJECTIVES

To investigate the validity of data extraction in systematic reviews of adverse events, the effect of data extraction errors on the results, and to develop a classification framework for data extraction errors to support further methodological research.

DESIGN

Reproducibility study.

DATA SOURCES

PubMed was searched for eligible systematic reviews published between 1 January 2015 and 1 January 2020. Metadata from the randomised controlled trials were extracted from the systematic reviews by four authors. The original data sources (eg, full text and ClinicalTrials.gov) were then referred to by the same authors to reproduce the data used in these meta-analyses.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Systematic reviews were included when based on randomised controlled trials for healthcare interventions that reported safety as the exclusive outcome, with at least one pair meta-analysis that included five or more randomised controlled trials and with a 2×2 table of data for event counts and sample sizes in intervention and control arms available for each trial in the meta-analysis.

MAIN OUTCOME MEASURES

The primary outcome was data extraction errors summarised at three levels: study level, meta-analysis level, and systematic review level. The potential effect of such errors on the results was further investigated.

RESULTS

201 systematic reviews and 829 pairwise meta-analyses involving 10 386 randomised controlled trials were included. Data extraction could not be reproduced in 1762 (17.0%) of 10 386 trials. In 554 (66.8%) of 829 meta-analyses, at least one randomised controlled trial had data extraction errors; 171 (85.1%) of 201 systematic reviews had at least one meta-analysis with data extraction errors. The most common types of data extraction errors were numerical errors (49.2%, 867/1762) and ambiguous errors (29.9%, 526/1762), mainly caused by ambiguous definitions of the outcomes. These categories were followed by three others: zero assumption errors, misidentification, and mismatching errors. The impact of these errors were analysed on 288 meta-analyses. Data extraction errors led to 10 (3.5%) of 288 meta-analyses changing the direction of the effect and 19 (6.6%) of 288 meta-analyses changing the significance of the P value. Meta-analyses that had two or more different types of errors were more susceptible to these changes than those with only one type of error (for moderate changes, 11 (28.2%) of 39 v 26 (10.4%) 249, P=0.002; for large changes, 5 (12.8%) of 39 v 8 (3.2%) of 249, P=0.01).

CONCLUSION

Systematic reviews of adverse events potentially have serious issues in terms of the reproducibility of the data extraction, and these errors can mislead the conclusions. Implementation guidelines are urgently required to help authors of future systematic reviews improve the validity of data extraction.

Broad Medical Uncertainty and the ethical obligation for openness

This paper argues that there exists a collective epistemic state of 'Broad Medical Uncertainty' (BMU) regarding the effectiveness of many medical interventions. We outline the features of BMU, and describe some of the main contributing factors. These include flaws in medical research methodologies, bias in publication practices, financial and other conflicts of interest, and features of how evidence is translated into practice. These result in a significant degree of uncertainty regarding the effectiveness of many medical treatments and unduly optimistic beliefs about the benefit/harm profiles of such treatments. We argue for an ethical presumption in favour of openness regarding BMU as part of a 'Corrective Response'. We then consider some objections to this position (the 'Anti-Corrective Response'), including concerns that public honesty about flaws in medical research could undermine trust in healthcare institutions. We suggest that, as it stands, the Anti-Corrective Response is unconvincing.

Adverse and other unintended consequences of setting-based interventions to prevent illicit drug use: A systematic review of reviews

This article explores adverse and unintended consequences (AUCs) of setting-based public health interventions to prevent illicit drug use, including the mechanisms leading to these AUCs. Additionally, the reporting of AUCs in systematic reviews was assessed. Therefore, we conducted a systematic review of reviews and searched four big databases were searched. We included systematic reviews concerned with setting-based interventions to prevent illicit drug use. We used AMSTAR 2 to rate the overall confidence of the results presented in the reviews. Data on study characteristics, types and mechanisms of AUCs were extracted. An a priori categorisation of consequences drew on the WHO-INTEGRATE framework, and the categorisation of mechanisms on the Behaviour Change Wheel. For reviews reporting AUCs, the same information was also retrieved from relevant primary studies. Findings were synthesised narratively and in tables. Finally, we included 72 reviews, of which 18 reported on AUCs. From these, 11 primary studies were identified. Most of the reviews and primary studies were conducted in educational settings. The most prevalent AUCs reported in systematic reviews and primary studies were paradoxical health effects (i.e. increase of drug use). Potential mechanisms discussed primarily focussed on the change though social norms and practices. Changes of knowledge and perception were also mentioned. Concluding, the identified reviews and primary studies paid insufficient attention to AUCs of public health interventions to prevent illicit drug use. Where reported, it was mostly as an afterthought and narrowly framed as health related. No mentions of potential broader social consequences were found.

Harms in Systematic Reviews Paper 3: Given the same data sources, systematic reviews of gabapentin have different results for harms

OBJECTIVE

In this methodologic study (Part 2 of 2), we examined the overlap in sources of evidence and the corresponding results for harms in systematic reviews for gabapentin.

STUDY DESIGN & SETTING

We extracted all citations referenced as sources of evidence for harms of gabapentin from 70 systematic reviews, as well as the harms assessed and numerical results. We assessed consistency of harms between pairs of reviews with a high degree of overlap in sources of evidence (>50%) as determined by corrected covered area (CCA).

RESULTS

We found 514 reports cited across 70 included reviews. Most reports (244/514, 48%) were not cited in more than one review. Among 18 pairs of reviews, we found reviews had differences in which harms were assessed and their choice to meta-analyze estimates or present descriptive summaries. When a specific harm was meta-analyzed in a pair of reviews, we found similar effect estimates.

CONCLUSION

Differences in harms results across reviews can occur because the choice of harms is driven by reviewer preferences, rather than standardized approaches to selecting harms for assessment. A paradigm shift is needed in the current approach to synthesizing harms.

Harms in Systematic Reviews Paper 2: Methods used to assess harms are neglected in systematic reviews of gabapentin

OBJECTIVE

We compared methods used with current recommendations for synthesizing harms in systematic reviews and meta-analyses (SRMAs) of gabapentin.

STUDY DESIGN & SETTING

We followed recommended systematic review practices. We selected reliable SRMAs of gabapentin (i.e., met a pre-defined list of methodological criteria) that assessed at least one harm. We extracted and compared methods in four areas: pre-specification, searching, analysis, and reporting. Whereas our focus in this paper is on the methods used, Part 2 examines the results for harms across reviews.

RESULTS

We screened 4320 records and identified 157 SRMAs of gabapentin, 70 of which were reliable. Most reliable reviews (51/70; 73%) reported following a general guideline for SRMA conduct or reporting, but none reported following recommendations specifically for synthesizing harms. Across all domains assessed, review methods were designed to address questions of benefit and rarely included the additional methods that are recommended for evaluating harms.

CONCLUSION

Approaches to assessing harms in SRMAs we examined are tokenistic and unlikely to produce valid summaries of harms to guide decisions. A paradigm shift is needed. At a minimal, reviewers should describe any limitations to their assessment of harms and provide clearer descriptions of methods for synthesizing harms.

Harms in Systematic Reviews Paper 1: An introduction to research on harms

OBJECTIVE

Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews.

STUDY DESIGN AND SETTING

Commentary describing challenges with assessing harm.

RESULTS

Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing.

CONCLUSION

The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms.

Outcome measurement in adult flexor tendon injury: A systematic review

BACKGROUND

Defining the optimal, evidence-based management of flexor tendon injury remains challenging. Lack of consensus on which measures to use to assess the outcome of interventions is a key issue, especially with regard to patient-reported outcome measures (PROMs). This systematic review defines the landscape of outcome measurement in studies on interventions for flexor tendon injuries to guide future research.

METHODS

A PRISMA-compliant systematic review was conducted using bespoke search strategies applied to MEDLINE, EMBASE, PsycINFO, CENTRAL, CINAHL and AMED. A protocol was developed and registered prospectively (CRD42020186780). We identified all studies describing adult patients undergoing interventions for acute hand flexor tendon injuries.

RESULTS

Of the 4844 studies, 114 studies met the final inclusion criteria for evaluating the outcomes of 8127 participants with 9071 injured digits. Studies included 24 randomised controlled trials, 19 cohort studies and 61 case series. Nine different PROMs were used in 24 studies (22%): three site-specific PROMs, one generic quality-of-life measure and four visual analogue scales. Clinician-reported outcome measures were used in 103 studies (96%), such as the range of motion reported in 102 studies (94%). Adverse outcomes were reported in 96 studies (89%), with the most frequently reported adverse outcomes being tendon rupture and infection. Re-operation was reported in 21 studies (19%). The most frequently reported health economic outcome measure was the length of work absence, reported in ten studies (9%).

CONCLUSIONS

There is variability in the use of outcome measures used to study interventions for flexor tendon injuries. An independent systematic review of the psychometric properties of the identified outcome measures and a specific multi-stakeholder consensus process may support optimal choice and standardisation for future studies.

School-Based Nutrition Interventions in Children Aged 6 to 18 Years: An Umbrella Review of Systematic Reviews

Schools are identified as a key setting to influence children's and adolescents' healthy eating. This umbrella review synthesised evidence from systematic reviews of school-based nutrition interventions designed to improve dietary intake outcomes in children aged 6 to 18 years. We undertook a systematic search of six electronic databases and grey literature to identify relevant reviews of randomized controlled trials. The review findings were categorised for synthesis by intervention type according to the World Health Organisation Health Promoting Schools (HPS) framework domains: nutrition education; food environment; all three HPS framework domains; or other (not aligned to HPS framework domain). Thirteen systematic reviews were included. Overall, the findings suggest that school-based nutrition interventions, including nutrition education, food environment, those based on all three domains of the HPS framework, and eHealth interventions, can have a positive effect on some dietary outcomes, including fruit, fruit and vegetables combined, and fat intake. These results should be interpreted with caution, however, as the quality of the reviews was poor. Though these results support continued public health investment in school-based nutrition interventions to improve child dietary intake, the limitations of this umbrella review also highlight the need for a comprehensive and high quality systematic review of primary studies.

Reporting of methods to prepare, pilot and perform data extraction in systematic reviews: analysis of a sample of 152 Cochrane and non-Cochrane reviews

BACKGROUND

Previous research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews.

METHODS

We included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively.

RESULTS

Of the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%).

CONCLUSIONS

Reporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.

More consideration is needed for retracted non-Cochrane systematic reviews in medicine: a systematic review

OBJECTIVE

To analyze the retraction status and reasons of non-Cochrane systematic reviews (SRs) in medicine.

STUDY DESIGN AND SETTING

MEDLINE, Embase, Retraction Watch Database and Google Scholar were systematically searched to find all retracted non-Cochrane SRs.

RESULTS

Of 159 non-Cochrane SRs in medicine retracted between 2004 and 2020, more than 70% were led by authors from China and affiliated with hospitals. The largest proportion of retraction notices were issued by the publisher and editor(s) jointly. Fraudulent peer-review was the most common reason for retraction, followed by unreliable data meaning errors in study selection or data analysis. The median time between publication and retraction was 14 months, and SRs retracted due to research misconduct took longer to retract than honest error.

CONCLUSION

The total number of retracted SRs is increasing worldwide, in particular in China. The most common reasons for retraction are fraudulent peer-review and unreliable data, and in most cases the SR is retracted more than a year after publication. Better systems of ethical oversight and culture to improve the process of peer review and adherence to the COPE retraction guidance are needed, and authors should strengthen their skills in SR methodology.

Methodological quality for systematic reviews of adverse events with surgical interventions: a cross-sectional survey

BACKGROUND

An increasing number of systematic reviews assessed the safety of surgical interventions over time. How well these systematic reviews were designed and conducted determines the reliability of evidence. In this study, we aimed to assess the methodological quality of systematic reviews on the safety of surgical interventions.

METHODS

We searched PubMed for systematic reviews of surgical interventions with safety as the exclusive outcome from 1st-Jan, 2015 to 1st-Jan, 2020. The methodological quality of eligible systematic reviews was evaluated according to the AMSTAR 2.0 instrument. The primary outcomes were the number of methodological weaknesses and the global methodological quality. The proportion of each methodological weakness among eligible systematic reviews was compared by three pre-defined stratification variables. The absolute difference of the proportion (PD) was used as the effect estimator, with the two-tailed z-test for the significance.

RESULTS

We identified 127 systematic reviews from 18,636 records. None (n = 0, 0.00%) of them could be rated as "high" in terms of the global methodological quality; in contrast, they were either rated as "low" (n = 18, 14.17%) or as "critically low" (n = 109, 85.83%). The median number of methodological weaknesses of these systematic reviews was 8 (interquartile range, IQR: 6 to 9), in which 4 (IQR: 2 to 4) were critical weaknesses. Systematic reviews that used any reporting guideline (e.g., domain 13, PD = -0.22, 95% CI: - 0.39, - 0.06; p = 0.01) and developed a protocol in advance (e.g., domain 6, PD = -0.20, 95% CI: - 0.39, - 0.01; p = 0.04) were less likely to have methodological weakness in some domains but not for the rest (e.g., domain 8, PD = 0.04, 95% CI: - 0.14, 0.21; p = 0.68; with protocol vs. without).

CONCLUSIONS

The methodological quality of current systematic reviews of adverse events with surgical interventions was poor. Further efforts, for example, encouraging researchers to develop a protocol in advance, are needed to enhance the methodological quality of these systematic reviews.

New Classification for the Reporting of Complications in Retinal Detachment Surgical Trials

IMPORTANCE

Quantifying severity of complications in rhegmatogenous retinal detachment (RRD) surgical trials is needed. A consensus classification system will help surgeons to more effectively compare harms of different surgical techniques.

OBJECTIVE

To develop a new consensus-based classification to quantify severity of complications of RRD surgery.

DESIGN, SETTING, AND PARTICIPANTS

A comprehensive list of complications was developed followed by a Delphi consensus survey of international vitreoretinal surgeons. The survey was conducted in 17 countries in mainland Europe, the United Kingdom, the United States, Asia, South Africa, and Australia. Seventy vitreoretinal surgeons were invited to take part in the Delphi survey; 45 agreed to participate. Participants were selected through boards/members lists of retinal societies. Data were analyzed between April 2019 and August 2019.

MAIN OUTCOMES AND MEASURES

Consensus-derived classification of complications of RRD surgery, according to their severity.

RESULTS

Forty-three of 45 vitreoretinal surgeons who agreed to participate in the Delphi survey completed round 1 (96%); all but 1 (98%) completed round 2. Consensus was reached for 96% of the 84 complications assessed. Examples of complications classified as least severe (graded 1) included subconjunctival hemorrhage and chemosis while those classified as most severe included endophthalmitis and sympathetic ophthalmia (graded 9) and phthisis (graded 10).

CONCLUSIONS AND RELEVANCE

We propose a new classification for quantifying severity of surgical complications based on an international consensus of vitreoretinal surgeons to quantify harm and improve the reporting of complications of RRD surgery.

Quality of Literature Searches Published in Leading Neurosurgical Journals: A Review of Reviews

BACKGROUND

There is mounting evidence that the search strategies upon which systematic reviews (SRs) are based frequently contain errors are incompletely reported or insensitive.

OBJECTIVE

To appraise the quality of search strategies in the 10 leading specialty neurosurgical journals and identify factors associated with superior searches.

METHODS

This research-on-research study systematically surveyed SRs published in the 10 leading neurosurgical journals between 01/10/2017 and 31/10/2019. All SRs were eligible for assessment using a predefined coding manual that was adapted from the preferred reporting items for systematic reviews and meta-analyses (PRISMA), a measurement tool to assess systematic reviews (AMSTAR), and Cochrane Collaboration guidelines. The PubMed interface was used to search the MEDLINE database, which was supplemented by individual journal searches. Descriptive statistics were utilized to identify factors associated with improved search strategies.

RESULTS

A total of 633 articles were included and contained a median of 19.00 (2.00-1654.00) studies. Less than half (45.97%) of included search strategies were considered to be reproducible. Aggregated reporting score was positively associated with in-text reference to reporting guideline adherence (τb = 0.156, P < .01). The number of articles retrieved by a search (τb = 0.11, P < .01) was also associated with the reporting of a reproducible search strategy.

CONCLUSION

This study demonstrates that the search strategies used in neurosurgical SRs require improvement. In addition to increasing awareness of reporting standards, we propose that this be achieved by the incorporation of PRISMA and other guidelines into article submission and peer-review processes. This may lead to the conduct of more informative SRs, which may result in improved clinician decision-making and patient outcomes.

Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

BACKGROUNDS

Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events.

METHODS

We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews.

RESULTS

We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=-0.09, 95% CI -0.21 to 0.03, p = 0.12).

CONCLUSION

Systematic review authors routinely treated studies with zero-events in both arms as "non-informative" carriers and excluded them from their reviews. Whether studies with no events are "informative" or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses.

Side effects of acetazolamide: a systematic review and meta-analysis assessing overall risk and dose dependence

Introduction

Acetazolamide (AZM) is used for various conditions (eg, altitude sickness, sleep apnoea, glaucoma), but therapy is often limited by its side effect profile. Our objective was to estimate the risk of commonly reported side effects based on meta-analyses. We hypothesised that these risks are dose-dependent.

Methods

We queried MEDLINE/EMBASE (Medical Literature Analysis and Retrieval System Online/Excerpta Medica dataBASE) up until 04/10/2019, including any randomised placebo-controlled trial in which adults received oral AZM versus placebo reporting side effects. Eligibility assessment was performed by two independent reviewers. Data were abstracted by one reviewer who verified key entries at a second time point. For side effects reported by >3 studies a pooled effect estimate was calculated, and heterogeneity assessed via I²; for outcomes reported by >5 studies effect modification by total daily dose (EMbyTDD; <400 mg/d, 400–600 mg/d, >600 mg/d) was assessed via meta-regression. For pre-specified, primary outcomes (paraesthesias, taste disturbances, polyuria and fatigue) additional subgroup analyses were performed using demographics, intervention details, laboratory changes and risk of bias.

Results

We included 42 studies in the meta-analyses (N_subjects=1274/1211 in AZM/placebo groups). AZM increased the risk of all primary outcomes (p<0.01, I² ≤16% and low-to-moderate quality of evidence for all)—the numbers needed to harm (95% CI; n_Studies) for each were: paraesthesias 2.3 (95% CI 2 to 2.7; n=39), dysgeusia 18 (95% CI 10 to 38, n=22), polyuria 17 (95% CI 9 to 49; n=22), fatigue 11 (95% CI 6 to 24; n=14). The risk for paraesthesias (beta=1.8 (95% CI 1.1 to 2.9); P_EMbyTDD=0.01) and dysgeusia (beta=3.1 (95% CI 1.2 to 8.2); P_EMbyTDD=0.02) increased with higher AZM doses; the risk of fatigue also increased with higher dose but non-significantly (beta=2.6 (95% CI 0.7 to 9.4); P_EMbyTDD=0.14).

Discussion

This comprehensive meta-analysis of low-to-moderate quality evidence defines risk of common AZM side effects and corroborates dose dependence of some side effects. These results may inform clinical decision making and support efforts to establish the lowest effective dose of AZM for various conditions.

Cannabis in the management of PTSD: a systematic review

Introduction

Existing reviews exploring cannabis effectiveness have numerous limitations including narrow search strategies. We systematically explored cannabis effects on PTSD symptoms, quality of life (QOL), and return to work (RTW). We also investigated harm outcomes such as adverse effects and dropouts due to adverse effects, inefficacy, and all-cause dropout rates.

Methods

Our search in MEDLINE, EMBASE, PsycInfo, CINAHL, Web of Science, CENTRAL, and PubMed databases, yielded 1 eligible RCT and 10 observational studies (n = 4672). Risk of bias (RoB) was assessed with the Cochrane risk of bias tool and ROBINS-I.

Results

Evidence from the included studies was mainly based on non-randomized studies with no comparators. Results from unpooled, high RoB studies showed that cannabis was associated with a reduction in overall PTSD symptoms and improved QOL. Dry mouth, headaches, and psychoactive effects such as agitation and euphoria were the commonly reported adverse effects. In most studies, cannabis was well tolerated, but small proportions of patients experienced a worsening of PTSD symptoms.

Conclusion

Evidence in the current study primarily stems from low quality and high RoB observational studies. Further RCTs investigating cannabis effects on PTSD treatment should be conducted with larger sample sizes and explore a broader range of patient-important outcomes.