Inadequate Reporting of Complications in Randomized Controlled Trials Cited as Supporting Evidence Underpinning AAOS CPG Recommendations for Hip and Knee Osteoarthritis: Application of the CONSORT Harms Checklist

Reporting of Complications in Rhinoplasty Randomized Controlled Trials: An Analysis Using the CONSORT Extension for Harms Checklist

OBJECTIVE

This study aimed to assess the completeness of adverse event (AE) reporting in randomized control trials (RCTs) focused on rhinoplasty, using the Consolidated Standards for Reporting (CONSORT) Extension for Harms checklist.

STUDY DESIGN

A cross-sectional design was employed to review RCTs related to rhinoplasty published between January 1, 2005, and January 28, 2022.

SETTING

The study analyzed clinical trials on rhinoplasty retrieved from PubMed.

METHODS

We performed a comprehension search on PubMed, blind and duplicate screening, and data extraction. Adherence to the 18 recommendations of the CONSORT Extension for Harms was evaluated, with 1 point assigned for each adhered item. Percent adherence was calculated based on the 18 points, taking into account the multiple subcategories within some recommendations. Descriptive statistics were used to summarize adherence-including frequencies, percentages, and 95% confidence intervals.

RESULTS

Our search returned 240 articles, of which 56 met inclusion criteria. No RCTs adhered to all 18 CONSORT Extension for Harms items. Twenty-six (26/56, 46.4%) adhered to ≥50% of the items, and 30 (30/56, 53.6%) adhered to ≥33.3% of the items. Seven (7/56, 12.5%) RCTs adhered to no items. Across all RCTs, the average number of CONSORT-Harms items adhered to was 7.2 (7.2/18, 40.0%). The most adhered to item was item 10. Discussion balanced with regard to efficacy and AEs (80.4%, [70.0-90.8]).

CONCLUSION

This study highlights the inadequacy of AE reporting in rhinoplasty RCTs according to CONSORT-Harms guidelines. Urgent efforts are required to bridge this reporting gap and enhance transparency in surgical research, ultimately safeguarding patient well-being.

Side-effects of mdma-assisted psychotherapy: a systematic review and meta-analysis

Evidence suggests that MDMA-assisted psychotherapy (MDMA-AP) has therapeutic potential for treatment of psychiatric illness. We conducted the first comprehensive systematic review and meta-analysis of the side effects of MDMA-AP across indications. We also assessed the quality of side effects-reporting in published trials of MDMA-AP. PubMed, EMBASE, PsycINFO, MEDLINE and Cochrane Central Register of Controlled Trials (CENTRAL) were systematically searched. Phase 2 and 3 MDMA-AP studies were included; Phase 1 studies, which assessed MDMA without psychotherapy, were not. Quality of side effects-reporting was assessed against the CONSORT Harms 2022 guidelines. We also compared numbers of adverse events reported in publications to those recorded in ClinicalTrial.gov registers. Thirteen studies were included, with eight contributing to the meta-analysis. In Phase 2 studies, MDMA-AP was associated with increased odds of any side effect during medication sessions (OR = 1.67, 95%CI (1.12, 2.49)) and in the 7 days following (OR = 1.59, 95%CI (1.12, 2.24)) relative to control conditions. In Phase 3 studies, MDMA-AP was associated with increased odds of any adverse event during the treatment period relative to placebo-assisted psychotherapy (OR = 3.51, 95%CI (2.76, 4.46)). The majority of RCTs were rated as having high risk of bias. Certainty of the evidence was rated as very low to moderate according to the GRADE framework. No included RCT had adequate adherence to the CONSORT Harms 2022 recommendations and reporting rates were also low. Compared to placebo, MDMA-AP was associated with increased odds of side effects, which were largely transient and mild or moderate in severity. However, identified limitations in existing evidence indicate that further investigation is needed to better characterize the safety profile of MDMA-AP and guide implementation.

Transparency Matters: Assessing Harms Reporting Compliance in Randomized Controlled Trials Underpinning the AAOS Clinical Practice Guideline for Anterior Cruciate Ligament Injuries

BACKGROUND

The American Academy of Orthopaedic Surgeons (AAOS) has developed a clinical practice guideline (CPG) for management of anterior cruciate ligament (ACL) injuries. Primary studies such as randomized controlled trials (RCTs) are cited as evidence for the guidelines. Given the influence that these trials have on patient care, adherence to standardized protocols for conducting and reporting RCTs is essential.

PURPOSE

To evaluate the CONSORT (Consolidated Standards of Reporting Trials) Extension for Harms-related reporting of RCTs cited as supporting evidence for the AAOS CPG on the management of ACL tears.

STUDY DESIGN

Cross-sectional study.

METHODS

The reference section of the AAOS guideline for ACL tears was first screened for RCTs cited in the CPG. Next, each RCT was evaluated for adherence to the CONSORT Extension for Harms checklist. Both identification of RCTs and assessment of adherence were performed in a masked and duplicate process. Descriptive statistics were used to summarize adherence to CONSORT Extension for Harms items. A Pearson correlation test was conducted to assess the relationship between the year of publication and adherence to CONSORT harms reporting.

RESULTS

The sample included 113 RCTs, of which 16 (14.2%) were published before the CONSORT Extension for Harms was implemented in 2004. Sample sizes ranged from 24 to 4564 participants, with a mean of 228. None of the included RCTs included all 18 items in the CONSORT Extension for Harms checklist. The mean number of checklist items reported was 4 (of 18; 22.2%). A moderate, positive, and statistically significant correlation was found between the RCT publication year and the adherence with reporting of the CONSORT Extension for Harms (t111 = 3.54; P < .001) (r = 0.32; 95% CI, 0.14-0.47).

CONCLUSION

Harms were infrequently reported in RCTs cited as supporting evidence in the AAOS CPG for the management of ACL tears. One encouraging finding was the positive correlation between the year when RCTs were published and how well they adhered to reporting harms. Efforts to improve adverse event reporting are warranted, as RCTs are commonly used to make clinical decisions in orthopaedic surgery.

Harms reporting in randomized controlled trials underpinning the American Academy of Orthopaedic Surgeons clinical practice guidelines for glenohumeral osteoarthritis

BACKGROUND

Glenohumeral osteoarthritis is one of the most common causes of shoulder pain. As such, the American Academy of Orthopaedic Surgeons (AAOS) has developed clinical practice guidelines (CPGs) to address the management of glenohumeral osteoarthritis. These CPG recommendations stem from the findings of randomized controlled trials (RCTs), which have been shown to influence clinical decision making and health policy. Therefore, it is essential that trial outcomes, including harms data (ie, adverse events), are adequately reported. We intend to evaluate the reporting quality of harms-related data in orthopedic literature specifically relating to AAOS CPG recommendations on the management of glenohumeral osteoarthritis.

METHODS

We adhered to the Preferred Reporting Items for Systematic Reviews (PRISMA) as well as guidance for reporting meta-research. The AAOS CPGs for glenohumeral osteoarthritis were obtained from orthoguidelines.org, and 2 authors independently screened the guidelines for the RCTs referenced. A total of 14 studies were identified. Data were extracted from the 14 included studies independently by the same 2 authors. Adherence to the Consolidated Standards of Reporting Trials (CONSORT) Extension for Harms Checklist was assessed using an 18-item scoring chart, with 1 point being awarded for meeting a checklist item and 0 points being awarded for not meeting a checklist item. Descriptive statistics, such as frequencies, percentages, and 95% confidence intervals were used to summarize RCT adherence to the CONSORT checklist.

RESULTS

The average score among the studies included was 7.36/18 items (39% adherence). No study adhered to all criteria, with the highest-performing study meeting 11 of 18 items (58%) and the lowest meeting 3 of 18 items (16%). A positive correlation between checklist score and year of publication was observed, with studies published more recently receiving a higher score on the CONSORT checklist (P < .05). Studies that disclosed funding information received a higher score than those that did not (P < .05), but there was no significant difference when the different funding sources were compared. Finally, double-blinded studies scored higher on the checklist than those with lower levels of blinding (single or no blinding, P < .05).

CONCLUSION

Adverse events are poorly reported amongst RCTs cited as supporting evidence for AAOS Management of Glenohumeral Osteoarthritis CPGs, evidenced by a CONSORT checklist compliance rate of only 41% in this study. We recommend the development of an updated checklist with information that makes it easier for authors to recognize, evaluate, and report on harms data. Additionally, we encourage authors to include information about adverse events or negative outcomes in the abstract.

Inadequate harms reporting among randomized controlled trials cited as supporting evidence in the AAOS management of hip fractures in older adults clinical practice guideline recommendations

BACKGROUND

Recommendations within clinical practice guidelines (CPGs) are heavily influenced by results from randomized controlled trials (RCTs). Therefore, it is imperative that all RCT outcomes are reported thoroughly to ensure CPGs are created using accurate information. Here, we evaluate the quality of harms reporting using the CONSORT Extension for Harms in RCTs underpinning recommendations in the American Academy of Orthopedic Surgeons (AAOS) Management of Hip Fractures in Older Adults CPG.

METHODS

Each RCT cited as evidence for recommendations in the AAOS Management of Hip Fractures in Older Adults CPG was evaluated using the CONSORT Extension for Harms to determine the quality of harms reporting. Descriptive statistics (frequencies, percentages, 95 % confidence intervals) were used to summarize adherence to CONSORT Harms items. A linear regression model was used to evaluate the CONSORT Harms influence on the quality of reporting over time.

RESULTS

Among the 156 RCTs identified, there were a total of 31,848 participants. Most RCTs were conducted at a single center (137; 87.8 %) and in a single-blind manner (130; 83.3 %). Fifty-four (34.6 %) RCTs did not provide funding statements. Trials adequately reported an average of 6.65 out of 18 CONSORT Extension for Harms items (37.0 %). One RCT adequately reported all items, while five reported zero items. Forty-seven RCTs (30.1 %) reported ≥ 50 % of items and 73 (46.8 %) reported ≤ 33.3 % of items. The linear regression model demonstrated no significant increase in mean adherence over time (adjusted R2 = -0.006; p = 0.563).

CONCLUSION

Our results highlight inadequate harms reporting among RCTs in the AAOS Management of Hip Fractures in Older Patients CPG. While the CONSORT Harms Extension was intended to enhance reporting, the linear regression model did not demonstrate significant improvements over time.

Adherence to Complication Reporting for Randomized Controlled Trials Contained in Clinical Practice Guidelines for the Management of Carpal Tunnel Syndrome

Purpose

Randomized controlled trials (RCTs) are frequently used in creating recommendations contained within clinical practice guidelines (CPGs). However, investigations outside of hand surgery have reported that RCTs within CPGs infrequently report complications and harms-related data. Our purpose was to assess adherence to complication reporting and harms-related outcomes contained in the Consolidated Standards for Reporting (CONSORT) Extension of Harms and Standards for Reporting of Diagnostic Accuracy Studies (STARD) reporting checklists for RCTs within the American Academy of Orthopaedic Surgery (AAOS) CPGs for carpal tunnel syndrome (CTS).

Methods

We identified all RCTs within the AAOS CTS CPGs. All therapeutic RCTs and diagnostic studies were included. We used the CONSORT Harms Checklist criteria to assess adherence to the reporting of adverse events for therapeutic RCTs and the STARD criteria to assess the diagnostic accuracy of the articles. We defined adequate compliance as adherence to ≥50% of the checklist items.

Results

We identified 82 therapeutic RCTs and 90 diagnostic accuracy articles within the AAOS CTS CPG. For therapeutic RCTs, we found that the average compliance with the published checklists was 19%. For diagnostic studies, the average compliance with checklists was found to be 55%. Eleven therapeutic RCTs (13%) and 60 diagnostic studies (67%) were determined to have adequate compliance for the CONSORT and STARD checklists, respectively.

Conclusions

Randomized controlled trials in the AAOS CPGs for CTS have low compliance with the CONSORT Extension for Harms Checklist. Although the overall adherence to the items published in the STARD statement for diagnostic accuracy evaluation remains higher, future efforts should be made to improve the adherence rates to both checklists.

Clinical relevance

Improved standardization of complication reporting may aid in comparing outcomes across multiple clinical investigations of upper-extremity procedures.

Harms Reporting in Randomized Controlled Trials Underpinning the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines for Distal Radius Fractures

PURPOSE

The purpose of this study was to measure the harms-related reporting among randomized controlled trials (RCTs) cited as supporting evidence for the American Academy of Orthopaedic Surgeons clinical practice guidelines regarding the management of distal radius fractures.

METHODS

We adhered to the guidance for reporting metaresearch and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines throughout the course of this investigation. We used the American Academy of Orthopaedic Surgeons clinical practice guidelines for distal radius fractures available on Orthoguidelines.org. A linear regression analysis was conducted to model the relationship between the year of publication and the total Consolidated Standards of Reporting Trials percentage adherence over time.

RESULTS

Thirty-five RCTs were included in the final sample. The average number of Consolidated Standards of Reporting Trials Extension for Harms items adequately reported across all included RCTs was 9.2 (9.2/18, 50.9%). None of the included trials adequately reported all 18 items. Ten items had a compliance of more than 50% (10/18, 55.6%), 4 items had a compliance of 20%-50% (4/18, 22.2%), and 4 items had a compliance of less than 20% (4/18, 22.2%). The results of the linear regression model showed no significant improvement in Consolidated Standards of Reporting Trials Harms reporting over time.

CONCLUSIONS

Adverse events are incompletely reported among RCTs cited as supporting evidence for American Academy of Orthopaedic Surgeons clinical practice guidelines for the management of distal radius fractures.

CLINICAL RELEVANCE

Given our findings, specific attention should be paid to improving the standardization of the classification of adverse events to facilitate ease in the reporting process.

A cross-sectional analysis of harms reporting in systematic reviews evaluating laminectomy

Background Context

Laminectomy is a common vertebral decompression procedure that has multiple potential adverse events which are not always reported in SRs.

Purpose

To evaluate the completeness of harms reporting in systematic reviews (SRs) on laminectomy.

Study Design

Cross-sectional analysis.

Methods

Eligible studies were SRs that evaluated laminectomy for any indication. MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched in May 2022 to locate studies for inclusion. Screening and data extraction on harms reporting and study characteristics were performed in duplicate. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR pairs.

Results

We included 26 SRs comprising 426 primary studies. Most SRs studied laminectomy for spinal stenosis, declared harms as a secondary outcome, and lacked or did not mention funding. Two SRs completely omitted harms, 9 had between 0% and 50.0% completion of harms items, and 15 had more than 50.1% completion. AMSTAR-2 graded 25 SRs (25/26, 96.2%) as 'critically low' and 1 SR (1/26, 3.8%) as 'low'. We found a statistically significant association between completeness of harms reporting and outcome specification. No other associations were statistically significant. Three SR pairs had CCAs >50% and were compared for unique and shared harms.

Conclusions

The completeness of harms reporting in SRs was inadequate. Because SRs often serve as tools for constructing clinical practice guidelines and clinical decision making, improvements must be made to enhance and refine harms reporting.

Harms Reporting Is Inadequate in Systematic Reviews Regarding Hip Arthroscopy

Purpose

To investigate the quality of harms reporting in systematic reviews (SRs) regarding hip arthroscopy in the current literature.

Methods

In May 2022, an extensive search of 4 major databases was performed identifying SRs regarding hip arthroscopy: MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and Cochrane Database of Systematic Reviews. A cross-sectional analysis was conducted, in which investigators performed screening and data extraction of the included studies in a masked, duplicate fashion. AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) was used to assess the methodologic quality and bias of the included studies. The corrected covered area was calculated for SR dyads.

Results

A total of 82 SRs were included in our study for data extraction. Of these SRs, 37 reported under 50% of the harms criteria (37 of 82, 45.1%) and 9 did not report harms at all (9 of 82, 10.9%). A significant relation was found between completeness of harms reporting and overall AMSTAR appraisal (P = .0261), as well as whether a harm was listed as a primary or secondary outcome (P = .0001). Eight SR dyads had corrected covered areas of 50% or greater and were compared for shared harms reported.

Conclusions

In this study, we found inadequate harms reporting in most SRs concerning hip arthroscopy.

Clinical Relevance

With the magnitude of hip arthroscopic procedures being performed, adequate reporting of harms-related information in the research surrounding this treatment is essential in assessing the efficacy of the treatment. This study provides data in relation to harms reporting in SRs regarding hip arthroscopy.