Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews

Inclusion of harm outcomes in core outcome sets requires careful consideration

OBJECTIVES

The objective of this study was to determine the proportion of all published core outcome set (COS) studies that include an adverse event or harm outcome, to determine the proportion of individual vs pooled harms, and to investigate characteristics that influence their inclusion.

METHODS

We examined the extent to which a sample of 100 published COS studies (from January 2021 to January 2023) include both pooled and individual harms in the final COS. One investigator extracted the information from the COS studies, which was cross-checked against previous COS investigational research, and where possible verified with COS authors or a pharmacologist. Using Qualtrics™, we conducted a personalized online survey of developers of the 100 COS to ask them about the importance, their experiences, and methodological approaches for dealing with harms within their COS development studies.

RESULTS

One hundred COS were identified from 91 separate COS studies, the majority of which considered most of the minimum standards for development. Two-thirds (65%) of the COS included at least 1 harm outcome. In total, 1104 core outcomes were identified across the 100 COS, of which 184 (17%) were harm outcomes (154 individual vs 56 pooled). Individual harms were more likely to be included in a final COS if they were developed for single treatment interventions (50%) compared to those being developed for multitreatment modalities (39%). Some COS developers adopted outcome frameworks as part of their COS development process to facilitate the inclusion of harm outcomes in their final COS. A third (33%) of respondents felt that harm outcomes should be included in all COS but over half (56%) thought this would be dependent on some aspect of the scope of the COS and improved methodology and awareness of how to deal with harm outcomes in the COS development process.

CONCLUSION

Harm outcomes are already included in many COS either as individual or pooled harms. It is evident that there are some challenges with regards to both the methodology and necessity to include harms within a COS (pooled or individual. COS developers should carefully consider the need to include important harms outcomes in relation to the scope of the COS that they are developing.

Development and Evaluation of a Framework for Identifying and Addressing Spin for Harms in Systematic Reviews of Interventions

"Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified.

The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews.

The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications.

Self-awareness and quality of relationships after acquired brain injury: Systematic review without meta-analysis (SWiM)

BACKGROUND

Relational aspects of self-awareness following Acquired Brain Injury (ABI) are increasingly being recognized. However, research underpinning the nature of the association between self-awareness and quality of relationships has yet to be synthesized.

METHOD

Searches, which were completed between February 2022 and February 2023, consisted of combining terms related to ABI, self-awareness, and quality of relationships. Data were analyzed using the Synthesis Without Meta-Analysis (SWiM) approach.

RESULTS

Associations between self-awareness and relationship quality across eight studies identified for this review differed in direction and significance. A more consistent pattern emerged, however, when studies assessing the quality of specific types of relationships i.e., spousal (N = 1) and therapeutic (N = 3), were compared to studies assessing the quality of a person's broader network of relationships (N = 4). In particular, good awareness was positively associated with the quality of specific relationships (r = 0.66) whereas it was negatively associated with the quality of a person's broader network of relationships (r = -0.35).

CONCLUSION

Results are discussed with consideration given to measures assessing the quality of specific relationships. In particular, such measures may tap into important patterns of interaction between two individuals, such as those related to attunement or communication, which may be valuable preconditions for improving awareness.

Outcome Reporting Bias in Clinical Trials Researching Disease-Modifying Therapy in Patients With Multiple Sclerosis

BACKGROUND AND OBJECTIVES

Outcome reporting bias occurs when publication of trial results is dependent on clinical significance, thereby threatening the validity of trial results. Research on immunomodulatory drugs in multiple sclerosis has thrived in recent years. We aim to comprehensively examine to what extent outcome reporting bias is present in these trials and the possible underlying factors.

METHODS

We identified clinical trials evaluating the efficacy and safety of immunomodulatory drugs in patients with multiple sclerosis (MS) registered in ClinicalTrials.gov after September 2007 and completed before the end of 2018. Information about study design, type of funding, and primary and secondary outcome measures was extracted from the registry. Timing of registration in relation to study initiation and subsequent amendments to the planned outcomes were reviewed. Publications related to these trials were identified in several bibliographic databases using the trial registration number. Registered primary and secondary outcomes were recorded for each trial and compared with outcomes in the publication describing the main outcomes of the trial.

RESULTS

A search of ClinicalTrials.gov identified 535 eligible registered clinical trials; of these, 101 had a matching publication. Discrepancies between registered and published primary and secondary outcomes were found in 95% of the trials, including discrepancies between the registered and published primary outcomes in 26 publications. Forty-four percent of the published secondary outcomes were not included in the registry. A similar proportion of registered and nonregistered reported primary efficacy outcomes were positive (favoring the intervention). Nonindustry-funded and open-label trials in MS were more prone to selective primary outcome reporting, although these findings did not reach statistical significance. Only two-thirds of the trials were registered in ClinicalTrials.gov before the trial start date, and 62% of trials made amendments in registered outcomes during or after the trial period.

DISCUSSION

Selective outcome reporting is prevalent in trials of disease-modifying drugs in people with MS. We propose methods to diminish the occurrence of this bias in future research.

Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

Clinical utility of overviews on adverse events of pharmacological interventions

BACKGROUND

Overviews (i.e., systematic reviews of systematic reviews, meta-reviews, umbrella reviews) are a relatively new type of evidence synthesis. Among others, one reason to conduct an overview is to investigate adverse events (AEs) associated with a healthcare intervention. Overviews aim to provide easily accessible information for healthcare decision-makers including clinicians. We aimed to evaluate the clinical utility of overviews investigating AEs.

METHODS

We used a sample of 27 overviews exclusively investigating drug-related adverse events published until 2021 identified in a prior project. We defined clinical utility as the extent to which overviews are perceived to be useful in clinical practice. Each included overview was assigned to one of seven pharmacological experts with expertise on the topic of the overview. The clinical utility and value of these overviews were determined using a self-developed assessment tool. This included four open-ended questions and a ranking of three clinical utility statements completed by clinicians. We calculated frequencies for the ranked clinical utility statements and coded the answers to the open-ended questions using an inductive approach.

RESULTS

The overall agreement with the provided statements was high. According to the assessments, 67% of the included overviews generated new knowledge. In 93% of the assessments, the overviews were found to add value to the existing literature. The overviews were rated as more useful than the individual included systematic reviews (SRs) in 85% of the assessments. The answers to the open-ended questions revealed two key aspects of clinical utility in the included overviews. Firstly, it was considered useful that they provide a summary of available evidence (e.g., along with additional assessments, or across different populations, or in different settings that have not been evaluated together in the included SRs). Secondly, it was found useful if overviews conducted a new meta-analysis to answer specific research questions that had not been answered previously.

CONCLUSIONS

Overviews on drug-related AEs are considered valuable for clinical practice by clinicians. They can make available evidence on AEs more accessible and provide a comprehensive view of available evidence. As the role of overviews evolves, investigations such as this can identify areas of value.

Spin on adverse effects in abstracts of systematic reviews of orthodontic interventions: a cross-sectional study (part 2)

BACKGROUND

It is critical that abstracts of systematic reviews transparently report both the beneficial and adverse effects of interventions without misleading the readers. This cross-sectional study assessed whether adverse effects of interventions were reported or considered in abstracts of systematic reviews of orthodontic interventions and whether spin on adverse effects was identified when comparing the abstracts with what was sought and reported in these reviews.

METHODS

This cross-sectional study (part 2 of 2) used the same sample of 98 systematic reviews orthodontic interventions as used in part 1. Eligible reviews were retrieved from the Cochrane Database of Systematic Reviews and the 5 leading orthodontic journals between August 1 2009 and July 31 2021. Prevalence proportions were sought for 3 outcomes as defined in the published protocol. Univariable logistic regression models were built to explore associations between the presence of spin in the abstract and a series of predictors. Odds ratios (OR) 95% confidence intervals (95% CI) were used to quantify the strength of associations and their precision.

RESULTS

76.5% (75/98) of eligible reviews reported or considered (i.e., discussed, weighted etc.) potential adverse effects of orthodontic interventions in the abstract and the proportion of spin on adverse effects was 40.8% (40/98) in the abstract of these reviews. Misleading reporting was the predominant category of spin, i.e., 90% (36/40). Our explorative analyses found that compared to the Cochrane Database of Systematic Reviews all 5 orthodontic journals had similar odds of the presence of spin on adverse effects in abstracts of systematic reviews of orthodontic interventions. The odds of the presence of spin did not change over the sampled years (OR: 1.03, 95% CI: 0.9 to 1.16) and did not depend on the number of authors (OR: 0.93, 95% CI: 0.71 to 1.21), or on the type of orthodontic intervention (OR: 1.1, 95% CI: 0.45 to 2.67), or whether conflicts of interests were reported (OR: 0.74, 95% CI: 0.32 to 1.68).

CONCLUSION

End users of systematic reviews of orthodontic interventions have to be careful when interpreting results on adverse effects in the abstracts of these reviews, because they could be jeopardized by uncertainties such as not being reported and misleading reporting as a result of spin.

PROTOCOL: Assessment of outcome reporting bias in studies included in Campbell systematic reviews

This is the protocol for a Campbell systematic review. The objectives are as follows: To identify methods used to assess the risk of outcome reporting bias (ORB) in studies included in recent Campbell systematic reviews of intervention effects. The review will answer the following questions: What proportion of recent Campbell reviews included assessment of ORB? How did recent reviews define levels of risk of ORB (what categories, labels, and definitions did they use)? To what extent and how did these reviews use study protocols as sources of data on ORB? To what extent and how did reviews document reasons for judgments about risk of ORB? To what extent and how did reviews assess the inter-rater reliability of ORB ratings? To what extent and how were issues of ORB considered in the review's abstract, plain language summary, and conclusions?

Effect of important modifiers on harmful effects in evidence synthesis practice of adverse events were insufficiently investigated: an empirical investigation

BACKGROUND

Safety is important in the assessment of health interventions, while the results of adverse events are often susceptive to potential effect modifiers since the event risk tends to be rare. In this study, we investigated whether the potential impact of the important effect modifiers on harmful effects was analyzed in meta-analyses of adverse events.

METHODS

Systematic reviews of healthcare interventions, had adverse events as the exclusive outcomes, had at least one meta-analysis, and published between 1st January 2015, and 1st January 2020 were collected. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. Six effect modifiers that are the most important for harmful effects were identified by a group discussion. The proportions of eligible systematic reviews that investigated the potential impact of the six effect modifiers on harmful effects were summarized.

RESULTS

We identified 279 systematic reviews eligible for this study. Except for the modifier of interventions/controls (70.61%, 197/279), most of the systematic reviews failed to investigate the potential impact of treatment duration (21.15%, 59/279), dosage (24.73%, 69/279), age (11.47%, 32/279), risk of bias (6.45%, 18/279), and source of funding (1.08%, 3/279) on harmful effects. Systematic reviews with meta-analyses containing more studies were more likely to investigate the potential impacts of these modifiers on the effects, but the proportion was still low (2.3% to 33.3%). Systematic reviews that developed a protocol were significantly more likely to investigate the potential impact of all these effect modifiers (e.g. treatment duration: odds ratio = 5.08, 95% CI: 2.76 to 9.35) on the results.

CONCLUSIONS

Current systematic reviews rarely investigated the potential impact of the important effect modifiers on harmful effects. Methodological guidelines for meta-analysis of adverse events should consider "effect modifier" as one of the domains to help systematic review authors better investigate harmful effects.

Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses

Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.

Assessing the Reporting of Harms in Systematic Reviews Focused on the Therapeutic and Cosmetic Uses of Botulinum Toxin

BACKGROUND AND OBJECTIVE

The expanding use of botulinum toxin (BoNT) in medical practice demonstrates the need to highlight whether there is adequate information regarding its safety profile. The aim of our study was to identify completeness of harms reporting for BoNT treatment within systematic reviews (SRs), assess quality of SRs using the AMSTAR-2 tool, and determine the degree of overlap among primary studies within each SR.

METHODS

On May 31, 2022, we searched Embase, Epistemonikos, MEDLINE, and the Cochrane Database of Systematic Reviews for SRs on BoNT therapy. Screening and data extraction were performed in a masked, duplicate fashion. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR dyads.

RESULTS

Of the 90 included SRs, we found that 70 completed less than 50% of harms items. The most reported items were BoNT as a favorable intervention (73/90, 81.1%) and harms as a primary outcome (72/90, 80.0%). The least reported items were grades and severity scales used to classify harms (8/90, 8.9%) and number of treatment discontinuations in each arm (10/90, 11.1%). Eighty-three SRs were rated "critically low" (83/90, 92.2%), while 5 SRs were rated "high" (5/90, 5.6%) via AMSTAR-2 tool. Significant associations were found between completion of harms reporting and: (1) a "critically low" appraisal on AMSTAR-2 tool (p = 0.0060) and (2) whether harms was reported as a primary outcome (p = 0.0001). The total CCA overlap was determined to be 0.8%.

CONCLUSION

Our results demonstrate that harms are underreported within BoNT SRs. Because healthcare professionals often refer to SRs to guide clinical decision making, it is important to continue to explore shortcomings among BoNT literature in future studies.

Harms Reporting Is Inadequate in Systematic Reviews Regarding Hip Arthroscopy

Purpose

To investigate the quality of harms reporting in systematic reviews (SRs) regarding hip arthroscopy in the current literature.

Methods

In May 2022, an extensive search of 4 major databases was performed identifying SRs regarding hip arthroscopy: MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and Cochrane Database of Systematic Reviews. A cross-sectional analysis was conducted, in which investigators performed screening and data extraction of the included studies in a masked, duplicate fashion. AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) was used to assess the methodologic quality and bias of the included studies. The corrected covered area was calculated for SR dyads.

Results

A total of 82 SRs were included in our study for data extraction. Of these SRs, 37 reported under 50% of the harms criteria (37 of 82, 45.1%) and 9 did not report harms at all (9 of 82, 10.9%). A significant relation was found between completeness of harms reporting and overall AMSTAR appraisal (P = .0261), as well as whether a harm was listed as a primary or secondary outcome (P = .0001). Eight SR dyads had corrected covered areas of 50% or greater and were compared for shared harms reported.

Conclusions

In this study, we found inadequate harms reporting in most SRs concerning hip arthroscopy.

Clinical Relevance

With the magnitude of hip arthroscopic procedures being performed, adequate reporting of harms-related information in the research surrounding this treatment is essential in assessing the efficacy of the treatment. This study provides data in relation to harms reporting in SRs regarding hip arthroscopy.

Harms reporting by systematic reviews for functional endoscopic sinus surgery: a cross-sectional analysis

OBJECTIVES

To evaluate the completeness of harms reporting in systematic reviews (SRs) pertaining to functional endoscopic sinus surgery (FESS).

METHODS

Using a cross-sectional study design, we performed a comprehensive search using MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and the Cochrane Database of Systematic Reviews databases for SRs regarding FESS on May 15th, 2022. Returns were screened and data were extracted in a masked, duplicate manner. Following established methodology, we extracted general study characteristics, harms items, and overall methodological quality for each SR in our sample. Corrected covered area (CCA) was calculated for SR dyads. For data analysis, using Stata 16.1 we performed a bivariate analysis between variables.

RESULTS

Fifty-five SR's were included in our sample after excluding 375 studies that did not meet our inclusion criteria. Of the included SRs, 19 (19/55, 34.5%) did not report harms and 39 (39/55, 70.9%) reported half of the harms items or fewer. Our study found that 23 (23/55, 41.8%) of SRs demonstrated a method of harms data collection, 26 (26/55, 47.3%) of SRs had patients available for harms analysis in their results, and 25 (25/55, 45.5%) of SRs had a balanced discussion of harms and benefits of FESS. Fifty-two SRs were appraised as "critically low" quality using AMSTAR-2. A significant association was found between completeness of harms reporting (Mahady) and whether harms were listed as a primary outcome. No other associations were statistically significant. Two SR dyads had CCAs between 20% and 50% overlap and were compared for unique and shared harms.

CONCLUSIONS

Our study demonstrates gaps in harms reporting regarding FESS in SRs. We recommend future studies implement guidelines such as the STROCCS guidelines or the harms extension of the PRISMA guidelines to improve harms reporting. Accurate harms reporting may advance patient safety and promote a more objective risk-benefit analysis for physicians and patients.

The Association Between Aspirin and Basal Cell Carcinoma: A Clinical and Financial Analysis

Background

Nonmelanoma skin cancer (NMSC) is the most common malignancy. Basal cell carcinoma (BCC) comprises about 80% of all NMSCs and its incidence continues to rise. Although BCC rarely leads to metastases or increased mortality, its effects on healthcare costs and quality of life are substantial. Aspirin may prevent the development of basal cell carcinoma (BCC) by the inhibition of cyclooxygenase (COX) enzymes, which are associated with carcinogenesis and inflammation. This study therefore examined the effect of aspirin on the risk of BCC, its clinical outcomes, and its treatment costs.

Methods

A retrospective study (2010 - 2018) was conducted using the Humana Health Insurance Database. International Classification of Disease ninth and 10th codes and National Drug Codes were used to identify BCC diagnoses and aspirin prescriptions. Patients were matched for age, sex, Charlson Comorbidity Score (CCI), and region of residence. Chi-squared, logistic regression, and odds ratio (OR) analyses were utilized to test for significance and to estimate relative risk.

Results

Aspirin use was associated with a decreased incidence of BCC in unmatched (OR = 0.658, 95% confidence interval (CI) 0.526 - 0.820) and matched (OR = 0.54, 95% CI 0.47 - 0.61) analyses. Aspirin was also associated with a decreased BCC risk when stratified by hypertension (P = 3.888 × 10^-5), chronic obstructive pulmonary disease (COPD) (P = 0.014), diabetes (P = 0.049) and tobacco use (P = 0.017). Aspirin use was not associated with risk of BCC when stratified by obesity (P = 0.408). The average paid per patient for BCC treatment was significantly higher for patients in the aspirin use group than in the aspirin nonuse group (P = 0.0087).

Conclusions

While the high incidence and cost of treatment of BCC are demanding both clinically and financially, the low cost of aspirin and its widespread use may have vital implications for its preventative role in this disease. This study concluded that aspirin use was associated with a significantly decreased risk of BCC.

Biases in reporting of adverse effects in clinical trials, and potential impact on safety assessments in systematic reviews and therapy guidelines

BACKGROUND

Clinical trials are an important source of adverse effects data, including analyses in systematic reviews and recommendations in therapy guidelines. Trial publication bias may have profound effects on safety perceptions. This MiniReview presents and discusses biases in reporting of safety data in clinical trials and the implications for systematic reviews and guidelines.

OBJECTIVES

The objectives of this work are to analyse risk of gastrointestinal bleeding in systemic corticosteroid trials and to assess adverse effects reporting in a fluoxetine trial in depression (Treatment for Adolescents With Depression Study [TADS]) and descriptions of adverse effects in adolescent depression therapy guidelines.

METHODS

We performed literature reviews and descriptive analyse of clinical trials with corticosteroids, and publications from the TADS trial. Risk of gastrointestinal bleeding from corticosteroids was analysed by meta-analysis.

FINDINGS

Gastrointestinal bleeding definitions varied considerably between trials. The incidence was significantly increased in hospitalized, but not in ambulant, patients compared to placebo. We identified several biases concerning TADS safety reporting, including severity thresholds and nonpublication of most adverse effects data beyond the initial 12 weeks. Therapy guidelines on adolescent depression mentioned suicidality risk, but many failed to mention other adverse effects.

CONCLUSIONS

We identified several pitfalls in adverse effects reporting in clinical trials. These include heterogeneous disease definitions, reporting thresholds, and incomplete reporting. Trial bias may have great impact on risk assessments in systematic reviews and meta-analyses.

Core Outcome Sets in Child Health: A Systematic Review

IMPORTANCE

Developing core outcome sets is essential to ensure that results of clinical trials are comparable and useful. A number of core outcome sets in pediatrics have been published, but a comprehensive in-depth understanding of core outcome sets in this field is lacking.

OBJECTIVE

To systematically identify core outcome sets in child health, collate the diseases to which core outcome sets have been applied, describe the methods used for development and stakeholder participation, and evaluate the methodological quality of existing core outcome sets.

EVIDENCE REVIEW

MEDLINE, SCOPUS, Cochrane Library, and CINAHL were searched using relevant search terms, such as clinical trials, core outcome, and children, along with relevant websites, such as Core Outcome Measures in Effectiveness Trials (COMET). Four researchers worked in teams of 2, performed literature screening and data extraction, and evaluated the methodological quality of core outcome sets using the Core Outcome Set-Standards for Development (COS-STAD).

FINDINGS

A total of 77 pediatric core outcome sets were identified, mainly developed by organizations or researchers in Europe, North America, and Australia and mostly from the UK (22 [29%]) and the US (22 [29%]). A total of 77 conditions were addressed; the most frequent International Classification of Diseases, 11th Revision category was diseases of the digestive system (14 [18%]). Most of the outcomes in pediatric core outcome sets were unordered (34 [44%]) or presented in custom classifications (29 [38%]). Core outcome sets used 1 or more of 8 development methods; the most frequent combination of methods was systematic review/literature review/scoping review, together with the Delphi approach and consensus for decision-making (10 [14%]). Among the 6 main types of stakeholders, clinical experts were the most frequently involved (74 [100%]), while industry representatives were rarely involved (4 [5%]). Only 6 core outcome sets (8%) met the 12 criteria of COS-STAD.

CONCLUSIONS AND RELEVANCE

Future quality of pediatric core outcome sets should be improved based on the standards proposed by the COMET initiative, while core outcome sets methodology and reporting standards should be extended to pediatric populations to help improve the quality of core outcome sets in child health. In addition, the COMET outcome taxonomy should also add items applicable to children.

The incidence of complications after derotational femoral and/or tibial osteotomies in patellofemoral disorders in adolescents and active young patients: a systematic review with meta-analysis

PURPOSE

To quantify the risk of perioperative and postoperative complications of derotational femoral and/or tibial osteotomies in patellofemoral disorders (anterior knee pain and patellar instability) in adolescents and active young patients.

METHODS

MEDLINE, EMBASE, Cochrane and Scopus databases were used to identify studies published from database inception and June 30, 2021. Meta-analysis was performed to pool the rates of complications related to femur and tibia osteotomies. Values of proportion of complications were expressed as proportions and 95% confidence intervals (CI) and then transformed using a Freeman Tukey double arcsine transformation. Meta-regression was used to explore factors that potentially may influence on heterogeneity such as year of publication, quality of the included studies and site of the osteotomy.

RESULTS

The 22 studies identified included a total of 648 derotational osteotomies in 494 patients. Studies consisted of 20 case series (non-comparative) and 2 comparative observational non-randomized cohorts. Tibial osteotomies showed higher risk of complications than femoral osteotomies (random pooled prevalence 9%; 95% CI 4-15% versus 1%; 95% CI 0-5%, respectively, p < 0.01). The meta-regression analysis of the articles showed that the only parameters responsible of the variance in number of complications were the osteotomy site.

CONCLUSIONS

Derotational femoral and/or tibial osteotomy is a safe surgical procedure in the treatment of patellofemoral disorders (anterior knee pain and patellar instability) in adolescents and active young people.

LEVEL OF EVIDENCE

IV.

Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials

In clinical trials, harms (i.e., adverse events) are often reported by simply counting the number of people who experienced each event. Reporting only frequencies ignores other dimensions of the data that are important for stakeholders, including severity, seriousness, rate (recurrence), timing, and groups of related harms. Additionally, application of selection criteria to harms prevents most from being reported. Visualization of data could improve communication of multidimensional data. We replicated and compared the characteristics of 6 different approaches for visualizing harms: dot plot, stacked bar chart, volcano plot, heat map, treemap, and tendril plot. We considered binary events using individual participant data from a randomized trial of gabapentin for neuropathic pain. We assessed their value using a heuristic approach and a group of content experts. We produced all figures using R and share the open-source code on GitHub. Most original visualizations propose presenting individual harms (e.g., dizziness, somnolence) alone or alongside higher level (e.g., by body systems) summaries of harms, although they could be applied at either level. Visualizations can present different dimensions of all harms observed in trials. Except for the tendril plot, all other plots do not require individual participant data. The dot plot and volcano plot are favored as visualization approaches to present an overall summary of harms data. Our value assessment found the dot plot and volcano plot were favored by content experts. Using visualizations to report harms could improve communication. Trialists can use our provided code to easily implement these approaches.

Assessing the quality of evidence on safety: specifications for application and suggestions for adaptions of the GRADE-criteria in the context of preparing a list of potentially inappropriate medications for older adults

Background

Systematic reviews that synthesize safety outcomes pose challenges (e.g. rare events), which raise questions for grading the strength of the body of evidence. This is maybe one reason why in many potentially inappropriate medication (PIM) lists the recommendations are not based on formalized systems for assessing the quality of the body of evidence such as GRADE.

In this contribution, we describe specifications and suggest adaptions of the GRADE system for grading the quality of evidence on safety outcomes, which were developed in the context of preparing a PIM-list, namely PRISCUS.

Methods

We systematically assessed each of the five GRADE domains for rating-down (study limitations, imprecision, inconsistency, indirectness, publication bias) and the criteria for rating-up, considering if special considerations or revisions of the original approach were indicated. The result was gathered in a written document and discussed in a group-meeting of five members with various background until consensus. Subsequently, we performed a proof-of-concept application using a convenience sample of systematic reviews and applied the approach to systematic reviews on 19 different clinical questions.

Results

We describe specifications and suggest adaptions for the criteria “study limitations”, imprecision, “publication bias” and “rating-up for large effect”. In addition, we suggest a new criterion to account for data from subgroup-analyses. The proof-of-concept application did not reveal a need for further revision and thus we used the approach for the systematic reviews that were prepared for the PRISCUS-list.

We assessed 51 outcomes. Each of the proposed adaptions was applied. There were neither an excessive number of low and very low ratings, nor an excessive number of high ratings, but the different methodological quality of the safety outcomes appeared to be well reflected.

Conclusion

The suggestions appear to have the potential to overcome some of the challenges when grading the methodological quality of harms and thus may be helpful for producers of evidence syntheses considering safety.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01715-5.

Adverse Effects of Angiotensin-Converting Enzyme Inhibitors in Humans: A Systematic Review and Meta-Analysis of 378 Randomized Controlled Trials

Background: Although angiotensin-converting enzyme (ACE) inhibitors are among the most-prescribed medications in the world, the extent to which they increase the risk of adverse effects remains uncertain. This study aimed to systematically determine the adverse effects of ACE inhibitors versus placebo across a wide range of therapeutic settings. Methods: Systematic searches were conducted on PubMed, Web of Science, and Cochrane Library databases. Randomized controlled trials (RCTs) comparing an ACE inhibitor to a placebo were retrieved. The relative risk (RR) and its 95% confidence interval (95% CI) were utilized as a summary effect measure. A random-effects model was used to calculate pooled-effect estimates. Results: A total of 378 RCTs fulfilled the eligibility criteria, with 257 RCTs included in the meta-analysis. Compared with a placebo, ACE inhibitors were associated with an significantly increased risk of dry cough (RR = 2.66, 95% CI = 2.20 to 3.20, p < 0.001), hypotension (RR = 1.98, 95% CI = 1.66 to 2.35, p < 0.001), dizziness (RR = 1.46, 95% CI = 1.26 to 1.70, p < 0.001), and hyperkalemia (RR = 1.24, 95% CI = 1.01 to 1.52, p = 0.037). The risk difference was quantified to be 0.037, 0.030, 0.017, and 0.009, respectively. Conclusions: We quantified the relative risk of numerous adverse events associated with the use of ACE inhibitors in a variety of demographics. This information can help healthcare providers be fully informed about any potential adverse consequences and make appropriate suggestions for their patients requiring ACE inhibitor therapy.

Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study

OBJECTIVES

To investigate the validity of data extraction in systematic reviews of adverse events, the effect of data extraction errors on the results, and to develop a classification framework for data extraction errors to support further methodological research.

DESIGN

Reproducibility study.

DATA SOURCES

PubMed was searched for eligible systematic reviews published between 1 January 2015 and 1 January 2020. Metadata from the randomised controlled trials were extracted from the systematic reviews by four authors. The original data sources (eg, full text and ClinicalTrials.gov) were then referred to by the same authors to reproduce the data used in these meta-analyses.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Systematic reviews were included when based on randomised controlled trials for healthcare interventions that reported safety as the exclusive outcome, with at least one pair meta-analysis that included five or more randomised controlled trials and with a 2×2 table of data for event counts and sample sizes in intervention and control arms available for each trial in the meta-analysis.

MAIN OUTCOME MEASURES

The primary outcome was data extraction errors summarised at three levels: study level, meta-analysis level, and systematic review level. The potential effect of such errors on the results was further investigated.

RESULTS

201 systematic reviews and 829 pairwise meta-analyses involving 10 386 randomised controlled trials were included. Data extraction could not be reproduced in 1762 (17.0%) of 10 386 trials. In 554 (66.8%) of 829 meta-analyses, at least one randomised controlled trial had data extraction errors; 171 (85.1%) of 201 systematic reviews had at least one meta-analysis with data extraction errors. The most common types of data extraction errors were numerical errors (49.2%, 867/1762) and ambiguous errors (29.9%, 526/1762), mainly caused by ambiguous definitions of the outcomes. These categories were followed by three others: zero assumption errors, misidentification, and mismatching errors. The impact of these errors were analysed on 288 meta-analyses. Data extraction errors led to 10 (3.5%) of 288 meta-analyses changing the direction of the effect and 19 (6.6%) of 288 meta-analyses changing the significance of the P value. Meta-analyses that had two or more different types of errors were more susceptible to these changes than those with only one type of error (for moderate changes, 11 (28.2%) of 39 v 26 (10.4%) 249, P=0.002; for large changes, 5 (12.8%) of 39 v 8 (3.2%) of 249, P=0.01).

CONCLUSION

Systematic reviews of adverse events potentially have serious issues in terms of the reproducibility of the data extraction, and these errors can mislead the conclusions. Implementation guidelines are urgently required to help authors of future systematic reviews improve the validity of data extraction.

Pregabalin add-on for drug-resistant focal epilepsy

BACKGROUND

This is an updated version of the Cochrane Review last published in Issue 7, 2019; it includes two additional studies. Epilepsy is a common neurological disease that affects approximately 1% of the UK population. Approximately one-third of these people continue to have seizures despite drug treatment. Pregabalin is one of the newer antiepileptic drugs that has been developed to improve outcomes. In this review we summarised the current evidence regarding pregabalin when used as an add-on treatment for drug-resistant focal epilepsy.

OBJECTIVES

To assess the efficacy and tolerability of pregabalin when used as an add-on treatment for drug-resistant focal epilepsy.

SEARCH METHODS

For the latest update we searched the following databases on 16 November 2020: Cochrane Register of Studies (CRS Web), and MEDLINE (Ovid, 1946 to 16 November 2020). CRS Web includes randomised or quasi-randomised, controlled trials from PubMed, Embase, ClinicalTrials.gov, the World Health Organisation International Clinical Trials Registry Platform (ICTRP), the Cochrane Central Register of Controlled Trials (CENTRAL), and the Specialised Registers of Cochrane Review Groups, including Epilepsy. We imposed no language restrictions. We contacted the manufacturers of pregabalin and authors in the field to identify any relevant unpublished studies.

SELECTION CRITERIA

We included randomised controlled trials comparing pregabalin with placebo or an alternative antiepileptic drug as an add-on for people of any age with drug-resistant focal epilepsy. Double-blind and single-blind trials were eligible for inclusion. The primary outcome was 50% or greater reduction in seizure frequency; secondary outcomes were seizure freedom, treatment withdrawal for any reason, treatment withdrawal due to adverse effects, and proportion of individuals experiencing adverse effects.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected trials for inclusion and extracted the relevant data. Primary analyses were intention-to-treat (ITT). We presented summary risk ratios (RRs) and odds ratios (ORs) with 95% confidence intervals (CIs). We evaluated dose response in regression models. We carried out a risk of bias assessment for each included study using the Cochrane risk of bias tool and assessed the overall certainty of evidence using the GRADE approach.

MAIN RESULTS

We included 11 randomised controlled trials (3949 participants). Nine trials compared pregabalin to placebo. For the primary outcome, participants randomised to pregabalin were significantly more likely to attain a 50% or greater reduction in seizure frequency compared to placebo (RR 1.95, 95% CI 1.40 to 2.72, 9 trials, 2663 participants, low-certainty evidence). The odds of response doubled with an increase in dose from 300 mg/day to 600 mg/day (OR 1.99, 95% CI 1.74 to 2.28), indicating a dose-response relationship. Pregabalin was significantly associated with seizure freedom (RR 3.94, 95% CI 1.50 to 10.37, 4 trials, 1125 participants, moderate-certainty evidence). Participants were significantly more likely to withdraw from pregabalin treatment than placebo for any reason (RR 1.33, 95% CI 1.10 to 1.60; 9 trials, 2663 participants; moderate-certainty evidence) and for adverse effects (RR 2.60, 95% CI 1.86 to 3.64; 9 trials, 2663 participants; moderate-certainty evidence). Three trials compared pregabalin to three active-control drugs: lamotrigine, levetiracetam and gabapentin. Participants allocated to pregabalin were significantly more likely to achieve a 50% or greater reduction in seizure frequency than those allocated to lamotrigine (RR 1.47, 95% CI 1.03 to 2.12; 1 trial, 293 participants) but not those allocated to levetiracetam (RR 0.94, 95% CI 0.80 to 1.11; 1 trial, 509 participants) or gabapentin (RR 0.96, 95% CI 0.82 to 1.12; 1 trial, 484 participants). We found no significant differences between pregabalin and lamotrigine for seizure freedom (RR 1.39, 95% CI 0.40 to 4.83). However, significantly fewer participants achieved seizure freedom with add-on pregabalin compared to levetiracetam (RR 0.50, 95% CI 0.30 to 0.85). No data were reported for this outcome for pregabalin versus gabapentin. We detected no significant differences in treatment withdrawal rate for any reason or due to adverse effects, specifically, during either pooled analysis or subgroup analysis. Ataxia, dizziness, somnolence, weight gain, headache and fatigue were significantly associated with pregabalin than in active control. We rated the overall risk of bias in the included studies as low or unclear due to the possibility of publication bias and lack of methodological details provided. We assessed all the studies to be at a high risk of funding bias as they were all sponsored by Pfizer. We rated the certainty of the evidence as very low to moderate using the GRADE approach.

AUTHORS' CONCLUSIONS

For people with drug-resistant focal epilepsy, pregabalin when used as an add-on treatment was significantly more effective than placebo at producing a 50% or greater seizure reduction and seizure freedom. Results demonstrated efficacy for doses from 150 mg/day to 600 mg/day, with increasing effectiveness at 600 mg doses, although there were issues with tolerability at higher doses. However, the trials included in this review were of short duration, and longer-term trials are needed to inform clinical decision-making. This review focused on the use of pregabalin in drug-resistant focal epilepsy, and the results cannot be generalised to add-on treatment for generalised epilepsies. Likewise, no inference can be made about the effects of pregabalin when used as monotherapy.

Harms in Systematic Reviews Paper 2: Methods used to assess harms are neglected in systematic reviews of gabapentin

OBJECTIVE

We compared methods used with current recommendations for synthesizing harms in systematic reviews and meta-analyses (SRMAs) of gabapentin.

STUDY DESIGN & SETTING

We followed recommended systematic review practices. We selected reliable SRMAs of gabapentin (i.e., met a pre-defined list of methodological criteria) that assessed at least one harm. We extracted and compared methods in four areas: pre-specification, searching, analysis, and reporting. Whereas our focus in this paper is on the methods used, Part 2 examines the results for harms across reviews.

RESULTS

We screened 4320 records and identified 157 SRMAs of gabapentin, 70 of which were reliable. Most reliable reviews (51/70; 73%) reported following a general guideline for SRMA conduct or reporting, but none reported following recommendations specifically for synthesizing harms. Across all domains assessed, review methods were designed to address questions of benefit and rarely included the additional methods that are recommended for evaluating harms.

CONCLUSION

Approaches to assessing harms in SRMAs we examined are tokenistic and unlikely to produce valid summaries of harms to guide decisions. A paradigm shift is needed. At a minimal, reviewers should describe any limitations to their assessment of harms and provide clearer descriptions of methods for synthesizing harms.

Harms in Systematic Reviews Paper 1: An introduction to research on harms

OBJECTIVE

Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews.

STUDY DESIGN AND SETTING

Commentary describing challenges with assessing harm.

RESULTS

Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing.

CONCLUSION

The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms.

Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

BACKGROUNDS

Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events.

METHODS

We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews.

RESULTS

We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=-0.09, 95% CI -0.21 to 0.03, p = 0.12).

CONCLUSION

Systematic review authors routinely treated studies with zero-events in both arms as "non-informative" carriers and excluded them from their reviews. Whether studies with no events are "informative" or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses.

Effects of selective outcome reporting on risk perception

The current study aimed to investigate how selective reporting of study results indicating increased health effects will influence its receiver's risk perception. Using the example of the Interphone Study from 2010 on mobile phone usage and cancer, an online experiment was conducted separating respondents into two groups. One group of subjects was informed selectively about a relationship between heavy mobile phone use and an elevated risk of glioma (brain cancer) only. The other group of subjects was informed about the full results of the analyses of glioma risk by cumulative call time, which suggests that other than for the heavy users, there were no statistically significant elevated risks related to mobile phone use. The results showed that selective reporting of risk information increased risk perception when compared to receiving the full information. Additionally, the selectively informed subjects revealed a stronger tendency towards overgeneralization of the 'elevated brain cancer risk' to all mobile phone users, although this did not extend to an overgeneralization to other electromagnetic field sources or differences in the perception of a usage time dependency for possible health risks. These results indicate that reporting of full results is an important factor in effective risk communication.

Meta-research publications in dentistry: a review

The present scoping review has the objective of providing an overview of meta-research in dentistry. A search of the PubMed database was performed for the period 11 October 2014 to 10 October 2019. Study selection and data extraction were performed independently by one author; prior to this, a random sample of 10% of the retrieved titles and abstracts were independently screened by two authors, achieving agreement of >80% on eligibility for initial inclusion, corresponding to good agreement. The following information was extracted from the full text of each article: meta-research area of interest; study design; type of studies evaluated in the meta-research; type of methodology used in assessment of the primary research; conflicts of interest reported; sponsorships reported; dental discipline; journal of publication; country of the first author; number of citations; and impact factor. A total of 7800 documents were initially retrieved. After analysis of the title/abstract and the full text of each article, and a snowballing procedure, 155 meta-research studies were identified and included. The 'methods' and 'reporting' meta-research areas were the most prevalent, with 73 (47%) and 61 (40%) studies, respectively. General dentistry, and orthodontics and dentofacial orthopaedics were the dental specialties with the greatest number/proportion of included studies with 45 (29%) and 28 (18%) studies, respectively. These findings may help to prioritize future meta-research in dentistry, consequently avoiding unnessecary investigations, and increasing the value of oral and dental research.

Adverse drug reaction risks obtained from meta-analyses and pharmacovigilance disproportionality analyses are correlated in most cases

OBJECTIVE

We aimed at testing if a correlation between adverse drug reactions relative risks estimated from meta-analyses and disproportionality analyses calculated from pharmacovigilance spontaneous reporting systems databases exist, and if methodological choices modify this correlation.

STUDY DESIGN

We extracted adverse drug reactions (ADR) odds ratios (ORs) from meta-analyses used as reference and calculated corresponding Reporting Odds Ratios (RORs) from the WHO pharmacovigilance database according to five different designs. We also calculated the relative bias and agreement of ROR compared to ORs.

RESULTS

We selected five meta-analyses which displayed a panel of 13 ADRs. A significant correlation for 7 out of the 13 ADRs studied in the primary analysis was found. The methods for ROR calculation impacted the results but none systematically improved the correlations. Whereas correlation was found between OR and ROR, agreement was poor and relative bias was important.

CONCLUSION

Despite the large variation in disproportionality analyses results due to design specification, this study provides further evidence that relative risks obtained from meta-analyses and from disproportionality analyses correlate in most cases, in particular for objective ADR not associated with the underlying pathology.

Immediate-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is characterized by symptoms of inattention or impulsivity or both, and hyperactivity, which affect children, adolescents, and adults. In some countries, methylphenidate is the first option to treat adults with moderate or severe ADHD. However, evidence on the efficacy and adverse events of immediate-release (IR) methylphenidate in the treatment of ADHD in adults is limited and controversial.

OBJECTIVES

To evaluate the efficacy and harms (adverse events) of IR methylphenidate for treating ADHD in adults.

SEARCH METHODS

In January 2020, we searched CENTRAL, MEDLINE, Embase, eight additional databases and three trial registers. We also searched internal reports on the European Medicines Agency and the US Food and Drug Administration websites. We checked citations of included trials to identify additional trials not captured by the electronic searches.

SELECTION CRITERIA

Randomized controlled trials (RCTs) comparing IR methylphenidate, at any dose, with placebo or other pharmacological interventions (including extended-release formulations of methylphenidate) for ADHD in adults. Primary outcomes comprised changes in the symptoms of ADHD (efficacy) and harms. Secondary outcomes included changes in the clinical impression of severity and improvement, level of functioning, depression, anxiety and quality of life. Outcomes could have been rated by investigators or participants.

DATA COLLECTION AND ANALYSIS

Two review authors extracted data independently on the characteristics of the trials, participants, interventions; outcomes and financial conflict of interests. We resolved disagreements by discussion or consulting a third review author. We obtained additional, unpublished information from the authors of one included trial that had reported efficacy data in a graph. We calculated mean differences (MDs) or standardized MDs (SMDs) with 95% confidence intervals (CIs) for continuous data reported on the same or different scales, respectively. We summarized dichotomous variables as risk ratios (RRs) with 95% CI.

MAIN RESULTS

We included 10 trials published between 2001 and 2016 involving 497 adults with ADHD. Three trials were conducted in Europe and one in Argentina; the remaining trials did not report their location. The RCTs compared IR methylphenidate with placebo, an osmotic-release oral system (OROS) of methylphenidate (an extended-release formulation), an extended-release formulation of bupropion, lithium, and Pycnogenol® (maritime pine bark extract). Participants comprised outpatients, inpatients in addiction treatment, and adults willing to attend an intensive outpatient program for cocaine dependence. The duration of the follow-up ranged from 6 to 18 weeks. IR methylphenidate versus placebo We found very low-certainty evidence that, compared with placebo, IR methylphenidate may reduce symptoms of ADHD when measured with investigator-rated scales (MD -20.70, 95% CI -23.97 to -17.43; 1 trial, 146 participants; end scores; Adult ADHD Investigator Symptom Report Scale (AISRS), scored from 0 to 54), but the evidence is uncertain. The effect of IR methylphenidate on ADHD symptoms when measured with participant-rated scales was moderate, but the certainty of the evidence is very low (SMD -0.59, 95% CI -1.25 to 0.06; I² = 69%; 2 trials, 138 participants; end scores). There is very low-certainty evidence that, compared with placebo, IR methylphenidate may reduce the clinical impression of the severity of ADHD symptoms (MD -0.57, 95% CI -0.85 to -0.28; 2 trials, 139 participants; I² = 0%; change and end scores; Clinical Global Impression (CGI)-Severity scale (scored from 1 (very much improved) to 7 (very much worse))). There is low-certainty evidence that, compared with placebo, IR methylphenidate may slightly impact the clinical impression of an improvement in symptoms of ADHD (MD -0.94, 95% CI -1.37 to -0.51; 1 trial, 49 participants; end scores; CGI-Improvement scale (scored from 1 (very much improved) to 7 (very much worse))). There is no clear evidence of an effect on anxiety (MD -0.20, 95% CI -4.84 to 4.44; 1 trial, 19 participants; change scores; Hamilton Anxiety Scale (HAM-A; scored from 0 to 56); very low-certainty evidence) or depression (MD 2.80, 95% CI -0.09 to 5.69; 1 trial, 19 participants; change scores; Hamilton Depression Scale (HAM-D; scored from 0 to 52); very low-certainty evidence) in analyses comparing IR methylphenidate with placebo. IR methylphenidate versus lithium Compared with lithium, it is uncertain whether IR methylphenidate increases or decreases symptoms of ADHD (MD 0.60, 95% CI -3.11 to 4.31; 1 trial, 46 participants; end scores; Conners' Adult ADHD Rating Scale (scored from 0 to 198); very low-certainty evidence); anxiety (MD -0.80, 95% CI -4.49 to 2.89; 1 trial, 46 participants; end scores; HAM-A; very low-certainty evidence); or depression (MD -1.20, 95% CI -3.81 to 1.41, 1 trial, 46 participants; end scores; HAM-D scale; very low-certainty evidence). None of the included trials assessed participant-rated changes in symptoms of ADHD, or clinical impression of severity or improvement in participants treated with IR methylphenidate compared with lithium. Adverse events were poorly assessed and reported. We rated all trials at high risk of bias due to selective outcome reporting of harms and masking of outcome assessors (failure to blind outcome assessor to measure adverse events). Overall, four trials with 203 participants who received IR methylphenidate and 141 participants who received placebo described the occurrence of harms. The use of IR methylphenidate in these trials increased the risk of gastrointestinal complications (RR 1.96, 95% CI 1.13 to 2.95) and loss of appetite (RR 1.77, 95% CI 1.06 to 2.96). Cardiovascular adverse events were reported inconsistently, preventing a comprehensive analysis. One trial comparing IR methylphenidate to lithium reported five and nine adverse events, respectively. We considered four trials to have notable concerns of vested interests influencing the evidence, and authors from two trials omitted information related to the sources of funding and conflicts of interest.

AUTHORS' CONCLUSIONS

We found no certain evidence that IR methylphenidate compared with placebo or lithium can reduce symptoms of ADHD in adults (low- and very low-certainty evidence). Adults treated with IR methylphenidate are at increased risk of gastrointestinal and metabolic-related harms compared with placebo. Clinicians should consider whether it is appropriate to prescribe IR methylphenidate, given its limited efficacy and increased risk of harms. Future RCTs should explore the long-term efficacy and risks of IR methylphenidate, and the influence of conflicts of interest on reported effects.

Ten principles for generating accessible and useable COVID-19 environmental science and a fit-for-purpose evidence base

1. The 'anthropause', a period of unusually reduced human activity and mobility due to COVID-19 restrictions, has serendipitously opened up unique opportunities for research on how human activities impact the environment. 2. In the field of health, COVID-19 research has led to concerns about the quality of research papers and the underlying research and publication processes due to accelerated peer review and publication schedules, increases in pre-prints and retractions. 3. In the field of environmental science, framing the pandemic and associated global lockdowns as an unplanned global human confinement experiment with urgency should raise the same concerns about the rigorousness and integrity of the scientific process. Furthermore, the recognition of an 'infodemic', an unprecedented explosion of research, risks research waste and duplication of effort, although how information is used is as important as the quality of evidence. This highlights the need for an evidence base that is easy to find and use - that is discoverable, curated, synthesizable, synthesized. 4. We put forward a list of 10 key principles to support the establishment of a reproducible, replicable, robust, rigorous, timely and synthesizable COVID-19 environmental evidence base that avoids research waste and is resilient to the pressures to publish urgently. These principles focus on engaging relevant actors (e.g. local communities, rightsholders) in research design and production, statistical power, collaborations, evidence synthesis, research registries and protocols, open science and transparency, data hygiene (cleanliness) and integrity, peer review transparency, standardized keywords and controlled vocabularies.

Reported outcomes for planned caesarean section versus planned vaginal delivery: A systematic review

BACKGROUND

There is currently no consensus regarding the outcomes and outcome measures that should be reported in studies assessing planned mode of birth.

OBJECTIVE

To develop an inventory for reported outcomes for studies comparing planned caesarean section (CS) and planned vaginal delivery (VD) for women age 18-45.

METHODS

A systematic review of outcomes reported in prospective studies investigating planned CS and planned VD was conducted. Three online databases, Ovid SP version of MEDLINE and EMBASE and the Cochrane Pregnancy and Childbirth Group's Trials Register, were searched from 2011 to June 2019. The inclusion criteria were: prospective studies evaluating planned mode of birth, age 18-45, singleton pregnancy, gestational age 37-40 weeks, >100 participants, middle or high income countries. No language restrictions were applied. Two researchers independently screened titles and abstracts, and subsequently reviewed the full text of each selected study to assess for eligibility. Discrepancies were resolved by discussion with a third author. The selected studies were evaluated using the MOMENT criteria. Outcomes and outcome measures were systematically extracted and organised into an inventory.

RESULTS

63 prospective studies comparing planned CS versus planned VD including data from 6,397,310 women were included. 37 studies (59%) investigating planned CS versus planned VD fulfilled four or more MOMENT criteria. In total, 43 different primary outcomes and 79 different primary outcome measures, and 12 different secondary outcomes and 31 secondary outcome measures were identified from studies investigating planned CS versus planned VD.

CONCLUSION

The findings of this study will contribute to the development of a core outcome set for planned mode of birth in the future. Standardising outcomes will aid comparison and interpretation of data pertaining to planned CS versus planned VD. PROSPERO registration: CRD42019133104.

Inferences About Drug Safety in Phase III Trials in Oncology: Examples From Advanced Prostate Cancer

BACKGROUND

Safety is a central consideration when choosing between multiple medications with similar efficacy. We aimed to evaluate whether adverse event (AE) profiles of 3 such drugs in advanced prostate cancer could be distinguished based on published literature.

METHODS

We assessed consistency in AE reporting, AE risk in placebo arms, and methodology used for risk estimates and quantification of statistical uncertainty in randomized placebo-controlled phase III trials of apalutamide, enzalutamide, and darolutamide in advanced prostate cancer.

RESULTS

Seven included clinical trials enrolled a total of 9215 participants (range = 1051-1715 per trial) across 3 prostate cancer disease states. Within disease states, baseline patient characteristics appeared similar between trials. Of 54 distinct AE types in total, only 3 (fatigue, hypertension, and seizure) were reported by all 7 trials. Absolute risks of AEs in the placebo arms differed systematically and more than twofold between trials, which was associated with visit frequency and resulted in different degrees of uncertainty in AE profiles between trials. No trial used inferential methodology to quantify statistical uncertainty in AE risks, but 6 of 7 trials drew overall conclusions. Two trials concluded that there was no elevated AE risk because of the intervention, including the trial of darolutamide, which had the greatest statistical uncertainty.

CONCLUSIONS

Rigorous comparison of drug safety was precluded by heterogeneity in AE reporting, variation in AE risks in the placebo arms, and lack of inferential statistical methodology, underscoring considerable opportunities to improve how AE data are collected, analyzed, and interpreted in oncology trials.

Adverse effects information in clinical guidelines on pharmacological treatment of depression in children and adolescents: a systematic review

OBJECTIVES

To analyse to what extent clinical practice guidelines on drug treatment of depression in children and adolescents mention the risk of adverse effects, to characterise the citations in the guidelines and to assess to what extent data from a major study (Treatment for Adolescents With Depression Study, TADS) was used as basis for information about adverse effects.

DESIGN

Systematic review of clinical guidelines and clinical decision support tools.

DATA SOURCES

PubMed, EMBASE, guideline collections, Health libraries.

ELIGIBILITY CRITERIA

We included national guidelines on depression in children and adolescents from European and/or English-speaking countries, published in English, German, French or any Scandinavian language since 2008. We also included well-known, international clinical decision support tools.

DATA EXTRACTION AND SYNTHESIS

Guidelines were examined by all authors to identify and classify information on adverse effects. Citations for statements on adverse effects were extracted and classified by category. The extent of citations about suicidality risk versus other adverse effects was assessed.

RESULTS

19 guidelines were assessed. All guidelines discussed risk of suicidal behaviour connected with use of antidepressants. Most guidelines mentioned some other psychiatric adverse effects. Several guidelines did not include information on well-known and common somatic adverse effects. Most references concerned risk of suicidality. Adverse effects identified in underlying studies were not always presented. The TADS study was referred to, directly or indirectly, by 18/19 guidelines, but some only referred to TADS with regard to suicidality without citing the study's findings of somatic adverse effects. No guideline commented on the lack of long-term adverse effects data from TADS.

CONCLUSIONS

Guidelines for treatment of depression in children and adolescents vary widely regarding information on adverse effects. Many guidelines do not provide information on common somatic adverse effects. There is no consensus as to what extent risks of adverse effects connected with use of antidepressants should be described in guidelines.

Evidence-based medicine and COVID-19: what to believe and when to change

The COVID-19 pandemic has led to a surge of information being presented to clinicians regarding this novel and deadly disease. There is a clear urgency to collate, review, appraise and act on this information if we are to do the best for clinicians and patients. However, the speed of the pandemic is a threat to traditional models of knowledge translation and practice change. In this concepts paper, we argue that clinicians need to be agile in their thinking and practice in order to find the right time to change. Adoption of new methods should be based on clinical judgement, the weight of evidence and the balance of probabilities that any new technique, test or treatment might work. The pandemic requires all of us to reach a new level of evidence-based medicine characterised by scepticism, thoughtfulness, responsiveness and clinically agility in practice.

Assessing the quality of meta-analyses in systematic reviews in pharmaceutical research in Iran by 2016: A systematic review

Background: Meta-analyses, like all other studies, may be poorly designed and implemented. This study was designed to determine the quality of meta-analyses in systematic reviews in the field of pharmaceutical research in Iran. Methods: Web of Science Core Collection, EMBASE, Ovid Medline, CINAHL, Scopus, and PubMed were systematically searched on June 4, 2017. The search was limited to the researches in the field of pharmaceutical studies. Based on inclusion criteria, 104 systematic reviews with meta-analysis (SRMA) were selected and assessed using quality assessment tools introduced by Higgins. Results: Participants, experimental interventions, and outcomes were reported in all the articles. Comparator intervention and study design were correctly reported in 103 (99.04%) and 101 (97.12%) articles, respectively. The comprehensive search strategy was available only in 48 articles (46.16%), and there was no evidence of a comprehensive search in 56 articles (53.84%). Risk of bias was investigated in 78 articles (75%). Also, funnel plots were the most commonly used method for reporting the bias in 64 articles (46.42%). Conclusion: In many of the meta-analyses, several items of the tool that represented a high-quality meta-analysis were absent. According to the findings, the comprehensive search and quality assessment were not at an appropriate level. Thus, the importance of reproducibility of information and quality assessment of included studies should be emphasized.

Clinicians' attitudes and perceived barriers and facilitators to cancer treatment clinical practice guideline adherence: a systematic review of qualitative and quantitative literature

Background

Clinical Practice Guidelines (CPGs) synthesize the best available evidence to guide clinician and patient decision making. There are a multitude of barriers and facilitators to clinicians adhering to CPGs; however, little is known about active cancer treatment CPG adherence specifically. This systematic review sought to identify clinician attitudes, and perceived barriers and facilitators to active cancer treatment CPG adherence.

Methods

A systematic search was undertaken of five databases; Ovid Medline, PsychInfo, Embase, Scopus, CINAHL, and PROQUEST. The retrieved abstracts were screened for eligibility against inclusion criteria, and a full text review was conducted of all eligible studies. Data were extracted, and a quality assessment was conducted of all included studies. The qualitative papers were thematically analyzed. Attitudes, barriers, and facilitating factors extracted from the quantitative papers were categorized within the qualitative thematic framework.

Results

The search resulted in the identification of 9676 titles. After duplicates were removed, abstracts screened, and full texts reviewed, 15 studies were included. Four themes were identified which related to negative clinician attitudes and barriers to active cancer treatment CPG adherence: (1) concern over CPG content and currency of CPGs; (2) concern about the evidence underpinning CPGs; (3) clinician uncertainty and negative perceptions of CPGs; and (4) organizational and patient factors. The review also identified four themes related to positive attitudes and facilitators to active cancer treatment CPG adherence: (5) CPG accessibility and ease of use; (6) endorsement and dissemination of CPGs and adequate access to treatment facilities and resources; (7) awareness of CPGs and belief in their relevance; and (8) belief that CPGs support decision making, improve patient care, reduce clinical variation, and reduce costs.

Conclusion

These results highlight that adherence to active cancer treatment CPG recommendations by oncology clinicians is influenced by multiple factors such as attitudes, practices, and access to resources. The review has also revealed many similarities and differences in the factors associated with general CPG, and active cancer treatment CPG, adherence. These findings will inform tailored implementation strategies to increase adherence to cancer treatment CPGs.

Trial registration

PROSPERO (2019) CRD42019125748.

Outcome reporting bias in Cochrane systematic reviews: a cross-sectional analysis

BACKGROUND

Discrepancies in outcome reporting (DOR) between protocol and published studies include inclusions of new outcomes, omission of prespecified outcomes, upgrade and downgrade of secondary and primary outcomes, and changes in definitions of prespecified outcomes. DOR can result in outcome reporting bias (ORB) when changes in outcomes occur after knowledge of results. This has potential to overestimate treatment effects and underestimate harms. This can also occur at the level of systematic reviews when changes in outcomes occur after knowledge of results of included studies. The prevalence of DOR and ORB in systematic reviews is unknown in systematic reviews published post-2007.

OBJECTIVE

To estimate the prevalence of DOR and risk of ORB in all Cochrane reviews between the years 2007 and 2014.

METHODS

A stratified random sampling approach was applied to collect a representative sample of Cochrane systematic reviews from each Cochrane review group. DOR was assessed by matching outcomes in each systematic review with their respective protocol. When DOR occurred, reviews were further assessed if there was a risk of ORB (unclear, low or high risk). We classified DOR as a high risk for ORB if the discrepancy occurred after knowledge of results in the systematic review.

RESULTS

150 of 350 (43%) review and protocol pairings contained DOR. When reviews were further scrutinised, 23% (35 of 150) of reviews with DOR contained a high risk of ORB, with changes being made after knowledge of results from individual trials.

CONCLUSIONS

In our study, we identified just under a half of Cochrane reviews with at least one DOR. Of these, a fifth were at high risk of ORB. The presence of DOR and ORB in Cochrane reviews is of great concern; however, a solution is relatively simple. Authors are encouraged to be transparent where outcomes change and to describe the legitimacy of changing outcomes in order to prevent suspicion of bias.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

BACKGROUND

There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.

METHODS

We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.

RESULTS

We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews).

CONCLUSIONS

Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Undergraduate and postgraduate pharmacovigilance education: A proposal for appropriate curriculum content

BACKGROUND

Adverse drug reactions (ADRs) are common, often preventable, and a leading cause of morbidity and mortality. Pharmacovigilance (PV) involves detection, assessment, understanding, and prevention of adverse effects or any other drug-related problem. Education of healthcare professionals (HCPs) involved in drug prescription, dispensing and administration is essential to help prevent and mitigate both ADRs and medication errors and has to be focused on 3 pivotal aspects: •Awareness: All medicines can produce adverse effects. ADRs should always be considered as part of the differential diagnosis if any new adverse condition, symptoms or signs appear after a drug administration or during or after pharmacological treatment. •Knowledge: HCPs must have a sound understanding of the most frequently prescribed drugs and over-the-counter medications, factors that make patients more likely to benefit or more susceptible to harm, as well as of causes of medication errors. •Reporting: HCPs should know how to report ADRs and the role of reporting on regulatory aspects and scientific knowledge. Undergraduate curricula must provide, at a minimum, sufficient skills that warrant the appropriate and safe prescription/dispensing/administration of medications in clinical practice, focusing both on therapeutic effects and prevention of harm. Clinical appraisal skills must include ADRs as differential diagnosis, taking accurate medication history, basic individual causality assessment, identification and proper management of ADRs, and informing patients of possible ADRs. Postgraduate periodic PV training should be mandatory as part of continuing education. Specialised postgraduate education should include advanced contents.

Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews

Background

Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called “spin”. In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin.

Methods

Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of “spin” related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups.

Discussion

We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.

Cancer cases detected in the prevention and control service of a private cancer clinic in Peru

Purpose

Describe the characteristics of patients seen at the Cancer Prevention and Control Service at a Peruvian private cancer clinic in 2014.

Patients and methods

This retrospective clinical study analyzed the prevalence of 10 cancers and characteristics of patients seen at a private cancer center located in Lima, Peru. The study sample included 7680 adults, and data were collected from de-identified medical records.

Results

The average age of the patients was 44.71 years and 98,82% of them had private insurance. The majority of patients were women (67.69%). Our gross incidence rate of cancer was 35.16 per 100,000 in the Cancer Prevention and Control Service in 2014. Only 0.35% had cancer, and most of those diagnosed with cancer (77.78%) were diagnosed in the early stages, stages I and II. The two most common cancers observed were breast and thyroid cancer.

Conclusions

The high rates of early, rather than late-stage diagnoses at this clinic are dramatically different than national rates. This difference may be because we are analyzing data from a prevention service seeing mainly patients with private insurance as opposed to national data, which consists primarily of patients seen in oncologic services with national insurance.