Side effects are incompletely reported among systematic reviews in gastroenterology

Harms Reporting in Systematic Reviews of the Microvascular Free Flap in Head and Neck Reconstruction

OBJECTIVE

To evaluate harms reporting in systematic reviews (SRs) of microvascular free flap (MFF) in head and neck reconstruction.

DATA SOURCES

This cross-sectional analysis included searches from the following major databases from 2012 to June 1, 2022: MEDLINE (Pubmed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews.

REVIEW METHODS

In a masked duplicate manner, screening was performed using Rayyan, and data were extracted using a pilot-tested Google form. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of reviews and the corrected covered area was calculated to detect primary study overlap across all reviews. Reviews were then grouped in pairs of 2, called dyads, and the corrected covered area was calculated again for each individual dyad. Dyads with high overlap (≥50%) were further investigated for the accuracy of harms reporting.

RESULTS

Our initial search yielded 268 records, with 50 SRs meeting the inclusion criteria. A total of 46 (92%) of the included reviews demonstrated 50% or more adherence to the items assessed in our harms checklist. Our corrected covered area tool revealed 0.6% primary study overlap across all reviews, and 1 dyad with high overlap (≥50%). No statistically significant relationship was observed between the completeness of harms reporting and reviews listing harms as a primary outcome, reviews reporting adherence to Preferred Reporting Items of Systematic Reviews and Meta-Analyses, or a review's AMSTAR rating.

CONCLUSION

This study identifies how harms reporting in SRs of MFF reconstruction of the head and neck can be improved and provides suggestions with the potential to mitigate the paucity in current literature.

Seeking adverse effects in systematic reviews of orthodontic interventions: a cross-sectional study (part 1)

BACKGROUND

Systematic reviews that assess the benefits of interventions often do not completely capture all dimensions of the adverse effects. This cross-sectional study (part 1 of 2 studies) assessed whether adverse effects were sought, whether the findings on these effects were reported, and what types of adverse effects were identified in systematic reviews of orthodontic interventions.

METHODS

Systematic reviews of orthodontic interventions on human patients of any health status, sex, age, and demographics, and socio-economic status, in any type of setting assessing any type of adverse effect scored at any endpoint or timing were eligible. The Cochrane Database of Systematic Reviews and 5 leading orthodontic journals were manually searched for eligible reviews between August 1 2009 and July 31 2021. Study selection and data extraction was conducted by two researchers independently. Prevalence proportions were calculated for four outcomes on seeking and reporting of adverse effects of orthodontic interventions. Univariable logistic regression models were used to determine the association between each one of these outcomes and the journal in which the systematic review was published using the eligible Cochrane reviews as reference.

RESULTS

Ninety-eight eligible systematic reviews were identified. 35.7% (35/98) of reviews defined seeking of adverse effects as a research objective, 85.7% (84/98) sought adverse effects, 84.7% (83/98) reported findings related to adverse effects, and 90.8% (89/98) considered or discussed potential adverse effects in the review. Reviews in the journal Orthodontics and Craniofacial Research compared with Cochrane reviews had approximately 7 times the odds (OR 7.20, 95% CI 1.08 to 47.96) to define seeking of adverse effects in the research objectives. Five of the 12 categories of adverse effects accounted for 83.1% (162/195) of all adverse effects sought and reported.

CONCLUSIONS

Although the majority of included reviews sought and reported adverse effects of orthodontic interventions, end-users of these reviews should beware that these findings do not give the complete spectrum on these effects and that they could be jeopardized by the risk of non-systematically assessing and reporting of adverse effects in these reviews and in the primary studies that feed them. Much research is ahead such as developing core outcome sets on adverse effects of interventions for both primary studies and systematic reviews.

A cross-sectional analysis of harms reporting in systematic reviews evaluating laminectomy

Background Context

Laminectomy is a common vertebral decompression procedure that has multiple potential adverse events which are not always reported in SRs.

Purpose

To evaluate the completeness of harms reporting in systematic reviews (SRs) on laminectomy.

Study Design

Cross-sectional analysis.

Methods

Eligible studies were SRs that evaluated laminectomy for any indication. MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched in May 2022 to locate studies for inclusion. Screening and data extraction on harms reporting and study characteristics were performed in duplicate. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR pairs.

Results

We included 26 SRs comprising 426 primary studies. Most SRs studied laminectomy for spinal stenosis, declared harms as a secondary outcome, and lacked or did not mention funding. Two SRs completely omitted harms, 9 had between 0% and 50.0% completion of harms items, and 15 had more than 50.1% completion. AMSTAR-2 graded 25 SRs (25/26, 96.2%) as 'critically low' and 1 SR (1/26, 3.8%) as 'low'. We found a statistically significant association between completeness of harms reporting and outcome specification. No other associations were statistically significant. Three SR pairs had CCAs >50% and were compared for unique and shared harms.

Conclusions

The completeness of harms reporting in SRs was inadequate. Because SRs often serve as tools for constructing clinical practice guidelines and clinical decision making, improvements must be made to enhance and refine harms reporting.

Assessing the Reporting of Harms in Systematic Reviews Focused on the Therapeutic and Cosmetic Uses of Botulinum Toxin

BACKGROUND AND OBJECTIVE

The expanding use of botulinum toxin (BoNT) in medical practice demonstrates the need to highlight whether there is adequate information regarding its safety profile. The aim of our study was to identify completeness of harms reporting for BoNT treatment within systematic reviews (SRs), assess quality of SRs using the AMSTAR-2 tool, and determine the degree of overlap among primary studies within each SR.

METHODS

On May 31, 2022, we searched Embase, Epistemonikos, MEDLINE, and the Cochrane Database of Systematic Reviews for SRs on BoNT therapy. Screening and data extraction were performed in a masked, duplicate fashion. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR dyads.

RESULTS

Of the 90 included SRs, we found that 70 completed less than 50% of harms items. The most reported items were BoNT as a favorable intervention (73/90, 81.1%) and harms as a primary outcome (72/90, 80.0%). The least reported items were grades and severity scales used to classify harms (8/90, 8.9%) and number of treatment discontinuations in each arm (10/90, 11.1%). Eighty-three SRs were rated "critically low" (83/90, 92.2%), while 5 SRs were rated "high" (5/90, 5.6%) via AMSTAR-2 tool. Significant associations were found between completion of harms reporting and: (1) a "critically low" appraisal on AMSTAR-2 tool (p = 0.0060) and (2) whether harms was reported as a primary outcome (p = 0.0001). The total CCA overlap was determined to be 0.8%.

CONCLUSION

Our results demonstrate that harms are underreported within BoNT SRs. Because healthcare professionals often refer to SRs to guide clinical decision making, it is important to continue to explore shortcomings among BoNT literature in future studies.

Harms Reporting Is Inadequate in Systematic Reviews Regarding Hip Arthroscopy

Purpose

To investigate the quality of harms reporting in systematic reviews (SRs) regarding hip arthroscopy in the current literature.

Methods

In May 2022, an extensive search of 4 major databases was performed identifying SRs regarding hip arthroscopy: MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and Cochrane Database of Systematic Reviews. A cross-sectional analysis was conducted, in which investigators performed screening and data extraction of the included studies in a masked, duplicate fashion. AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) was used to assess the methodologic quality and bias of the included studies. The corrected covered area was calculated for SR dyads.

Results

A total of 82 SRs were included in our study for data extraction. Of these SRs, 37 reported under 50% of the harms criteria (37 of 82, 45.1%) and 9 did not report harms at all (9 of 82, 10.9%). A significant relation was found between completeness of harms reporting and overall AMSTAR appraisal (P = .0261), as well as whether a harm was listed as a primary or secondary outcome (P = .0001). Eight SR dyads had corrected covered areas of 50% or greater and were compared for shared harms reported.

Conclusions

In this study, we found inadequate harms reporting in most SRs concerning hip arthroscopy.

Clinical Relevance

With the magnitude of hip arthroscopic procedures being performed, adequate reporting of harms-related information in the research surrounding this treatment is essential in assessing the efficacy of the treatment. This study provides data in relation to harms reporting in SRs regarding hip arthroscopy.

Assessing the Reporting and Frequency of Harms in Systematic Reviews Focused on Minimally Invasive Hysterectomies: A Cross-sectional Analysis

OBJECTIVE

To assess the quality of harms reporting in systematic reviews (SRs) regarding minimally invasive hysterectomies (MIHs).

DATA SOURCES

We performed a cross-sectional analysis on SRs addressing MIHs to assess completeness of harms reporting. Hysterectomies are common gynecologic procedures and are associated with complications. Such adverse events can directly affect physician decision making and patient outcomes. Thus, it is important that SRs equally weigh the harms and benefits surrounding MIHs.

METHODS OF STUDY SELECTION

On May 15, 2022, we searched MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews for SRs with or without a meta-analysis on MIH for any indication. Eligible studies underwent full-text screening, data extraction, harms reporting assessment, and A Measurement Tool to Assess Systematic Reviews-2 quality assessment in a masked, duplicate fashion. The corrected covered area was calculated to indicate any overlap between SR dyads.

TABULATION, INTEGRATION, AND RESULTS

A total of 52 SRs met the inclusion criteria for data extraction. We found that >44 of included SRs (of 52; 84.6%) reported >50% of the harms items. Completeness of harms reporting was significantly associated with harms specification as a primary outcome (p <.05). The corrected covered area was 0.60%.

CONCLUSION

The harms reporting was more complete than hypothesized, but still had deficiencies throughout, such as inconsistent use of severity scales to classify harms. Even though MIH harms reporting in sample SRs was more complete than other fields of medicine, it was still suboptimal overall and requires refinement.

Harms reporting by systematic reviews for functional endoscopic sinus surgery: a cross-sectional analysis

OBJECTIVES

To evaluate the completeness of harms reporting in systematic reviews (SRs) pertaining to functional endoscopic sinus surgery (FESS).

METHODS

Using a cross-sectional study design, we performed a comprehensive search using MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and the Cochrane Database of Systematic Reviews databases for SRs regarding FESS on May 15th, 2022. Returns were screened and data were extracted in a masked, duplicate manner. Following established methodology, we extracted general study characteristics, harms items, and overall methodological quality for each SR in our sample. Corrected covered area (CCA) was calculated for SR dyads. For data analysis, using Stata 16.1 we performed a bivariate analysis between variables.

RESULTS

Fifty-five SR's were included in our sample after excluding 375 studies that did not meet our inclusion criteria. Of the included SRs, 19 (19/55, 34.5%) did not report harms and 39 (39/55, 70.9%) reported half of the harms items or fewer. Our study found that 23 (23/55, 41.8%) of SRs demonstrated a method of harms data collection, 26 (26/55, 47.3%) of SRs had patients available for harms analysis in their results, and 25 (25/55, 45.5%) of SRs had a balanced discussion of harms and benefits of FESS. Fifty-two SRs were appraised as "critically low" quality using AMSTAR-2. A significant association was found between completeness of harms reporting (Mahady) and whether harms were listed as a primary outcome. No other associations were statistically significant. Two SR dyads had CCAs between 20% and 50% overlap and were compared for unique and shared harms.

CONCLUSIONS

Our study demonstrates gaps in harms reporting regarding FESS in SRs. We recommend future studies implement guidelines such as the STROCCS guidelines or the harms extension of the PRISMA guidelines to improve harms reporting. Accurate harms reporting may advance patient safety and promote a more objective risk-benefit analysis for physicians and patients.

An analysis of harms reporting in systematic reviews regarding ketorolac for management of perioperative pain

BACKGROUND

Owing to the frequent perioperative use of ketorolac tromethamine and its ability to minimise postoperative opioid requirements, it is important to continually reassess harms associated with its use. Our primary objective was to investigate the extent of harms reporting in systematic reviews (SRs) on ketorolac for perioperative pain.

METHODS

In May 2022, we conducted a search of major databases, MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews to identify eligible SRs on ketorolac for perioperative pain. Screening and data extraction were performed in masked, duplicate fashion. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of included SRs. Corrected covered area (CCA) was calculated to determine overlap of primary studies between SR dyads.

RESULTS

A total of 28 SRs evaluating 630 primary studies met the inclusion criteria. Seven SRs (7/28, 25%) reported no harms and 17 SRs (17/28, 60.7%) reported ≤50% of harms items. A significant association was found between completeness of harms reporting and whether harms were specified as a primary outcome (P<0.001). No other associations were statistically significant. Regarding methodological quality, 22 SRs were appraised as 'critically low' (22/28, 78.6%), 5 as 'low' (5/28, 17.9%), and 1 as 'high' (1/28, 3.6%). One SR dyad had a CCA >50% but neither reported harms.

CONCLUSIONS

The extent of harms reporting in systematic reviews was inadequate. Given the importance that systematic reviews have on guiding perioperative decision-making, it is essential to improve the completeness of harms reporting.

A comparison between two recommendations to conduct and report systematic reviews on drug's safety

BACKGROUND

Several recommendations are available to conduct and report a systematic review of adverse drug reactions. This study is aimed at identifying and comparing the methodologies of the two most commonly used recommendations to conduct and report systematic reviews on drug's safety.

METHODS

Two systematic reviews were conducted following the recommendations "Cochrane Handbook for Systematic Reviews of Interventions" and "Systematic Reviews' Centre for Reviews and Dissemination guidance for undertaking reviews in healthcare." The methods of each recommendation were characterized, and the results and the discussion of each systematic review were also evaluated.

RESULTS

The methodologies of both recommendations are similar. The review question was structured. Both recommendations suggest to include pre- and post-marketing data. The recommended data sources differed and, consequently, the results of the systematic reviews (37 vs. 35 studies). Other aspects of search literature were identical. Different tools are suggested to evaluate the methodological quality of the included studies. For case reports, both recommendations only report some questions that may be helpful to assess risk of bias. The reporting of the results and discussion is also identical for both recommendations.

CONCLUSIONS

Few methodological differences were observed between the analyzed recommendations to conduct a systematic review on drug's safety. Combining their methods into a single and recognized recommendation could be of great value.

Seeking adverse effects in systematic reviews of orthodontic interventions: protocol for a cross-sectional study

BACKGROUND

Before implementing healthcare interventions, clinicians need to weigh the beneficial and adverse effects of interventions. However, a large body of evidence has demonstrated that seeking and reporting of adverse effects is suboptimal in clinical trials and in systematic reviews of interventions. This cross-sectional study will investigate the status of this problem in orthodontics. This study will assess whether adverse effects were sought and whether findings related to adverse effects were reported in systematic reviews of orthodontic interventions in the five leading orthodontic journals and in the Cochrane Database of Systematic Reviews.

METHODS

Systematic reviews of clinical orthodontic interventions published between 01 August 2009 and 31 July 2019 in the five leading orthodontic journals and in the Cochrane Database will be included. Empty reviews will be excluded. The reporting of outcomes on adverse effects will not determine eligibility, i.e., reviews will not be excluded, because they did not report usable data. Study selection and data extraction will be conducted independently by two authors. Our primary outcome will be the prevalence of systematic reviews of orthodontic interventions that sought any findings related to adverse effects in the included studies. Additional prevalence statistics will be calculated on a series of items related to seeking of adverse effects in the eligible reviews. All statistics will be calculated for (1) all journals together, (2) the group of five orthodontic journals and the Cochrane Database of Systematic Reviews separately, and (3) each individual journal separately. Chi-square tests of independence will be used to compare these groups.

DISCUSSION

This study will assess whether adverse effects were sought in systematic reviews of orthodontic interventions. This knowledge is important, because reviews that present an incomplete picture on adverse effects can have unfavorable consequences for the end-users. Also not reporting that no adverse effects were assessed in eligible studies included in a systematic review can mislead pertinent stakeholders. Our findings could have policy implications for making judgments on accepting or rejecting an intervention systematic review for publication, for example, by directing editors and peer-reviewers to adopt the various items on adverse effects defined in the MECIR standards and in the PRISMA harm checklist.

Implementing core outcomes in kidney disease: report of the Standardized Outcomes in Nephrology (SONG) implementation workshop

There are an estimated 14,000 randomized trials published in chronic kidney disease. The most frequently reported outcomes are biochemical endpoints, rather than clinical and patient-reported outcomes including cardiovascular disease, mortality, and quality of life. While many trials have focused on optimizing kidney health, the heterogeneity and uncertain relevance of outcomes reported across trials may limit their policy and practice impact. The international Standardized Outcomes in Nephrology (SONG) Initiative was formed to identify core outcomes that are critically important to patients and health professionals, to be reported consistently across trials. We convened a SONG Implementation Workshop to discuss the implementation of core outcomes. Eighty-two patients/caregivers and health professionals participated in plenary and breakout discussions. In this report, we summarize the findings of the workshop in two main themes: socializing the concept of core outcomes, and demonstrating feasibility and usability. We outline implementation strategies and pathways to be established through partnership with stakeholders, which may bolster acceptance and reporting of core outcomes in trials, and encourage their use by end-users such as guideline producers and policymakers to help improve patient-important outcomes.

Preferred reporting items for overviews of systematic reviews including harms checklist: a pilot tool to be used for balanced reporting of benefits and harms

OBJECTIVES

An overview of systematic reviews (OoSRs) is a study designed to synthesize multiple evidence from existing systematic reviews on a specific domain. The aim of this paper was to offer a pilot version checklist with Preferred Reporting Items for OoSRs (PRIO-harms) to promote a more balanced reporting of benefits and harms in OoSRs of health care interventions.

STUDY DESIGN AND SETTING

The included items were developed by combining key features from health care OoSRs designs with recommendations from statements of other relevant checklists and pertinent methodological review articles. Two raters independently used the PRIO-harms checklist to assess a sample of 20 OoSRs.

RESULTS

The PRIO-harms tool consists of a 27-item (56 (sub-)items in total) checklist and is accompanied by a five-stage process flow diagram (identification, screening, eligibility, inclusion, and separation of relevant studies). The mean interrater reliability (Gwet's AC1 statistic) between reviewers was 0.90 (95% confidence interval: 0.88, 0.92) indicating a very good agreement.

CONCLUSION

The PRIO-harms tool can be used in every OoSRs that addresses health care interventions. This instrument will assist overview authors to improve completeness and transparency of research reporting with emphasis on harms. However, it might benefit from critical review and further validation from experts and research teams that produce OoSRs.