The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

Prioritisation of clinical trial learning needs of musculoskeletal researchers: an inter-disciplinary modified Delphi study by the Australia & New Zealand musculoskeletal clinical trials network

BACKGROUND

There is a need to increase the capacity and capability of musculoskeletal researchers to design, conduct, and report high-quality clinical trials. The objective of this study was to identify and prioritise clinical trial learning needs of musculoskeletal researchers in Australia and Aotearoa New Zealand. Findings will be used to inform development of an e-learning musculoskeletal clinical trials course.

METHODS

A two-round online modified Delphi study was conducted with an inter-disciplinary panel of musculoskeletal researchers from Australia and Aotearoa New Zealand, representing various career stages and roles, including clinician researchers and consumers with lived experience of musculoskeletal conditions. Round 1 involved panellists nominating 3-10 topics about musculoskeletal trial design and conduct that they believe would be important to include in an e-learning course about musculoskeletal clinical trials. Topics were synthesised and refined. Round 2 asked panellists to rate the importance of all topics (very important, important, not important), as well as select and rank their top 10 most important topics. A rank score was calculated whereby higher scores reflect higher rankings by panellists.

RESULTS

Round 1 was completed by 121 panellists and generated 555 individual topics describing their musculoskeletal trial learning needs. These statements were grouped into 37 unique topics for Round 2, which was completed by 104 panellists. The topics ranked as most important were: (1) defining a meaningful research question (rank score 560, 74% of panellists rated topic as very important); (2) choosing the most appropriate trial design (rank score 410, 73% rated as very important); (3) involving consumers in trial design through to dissemination (rank score 302, 62% rated as very important); (4) bias in musculoskeletal trials and how to minimise it (rank score 299, 70% rated as very important); and (5) choosing the most appropriate control/comparator group (rank score 265, 65% rated as very important).

CONCLUSIONS

This modified Delphi study generated a ranked list of clinical trial learning needs of musculoskeletal researchers. Findings can inform training courses and professional development to improve researcher capabilities and enhance the quality and conduct of musculoskeletal clinical trials.

Developments in the Design, Conduct, and Reporting of Child Health Trials

To identify priority areas to improve the design, conduct, and reporting of pediatric clinical trials, the international expert network, Standards for Research (StaR) in Child Health, was assembled and published the first 6 Standards in Pediatrics in 2012. After a recent review summarizing the 247 publications by StaR Child Health authors that highlight research practices that add value and reduce research "waste," the current review assesses the progress in key child health trial methods areas: consent and recruitment, containing risk of bias, roles of data monitoring committees, appropriate sample size calculations, outcome selection and measurement, and age groups for pediatric trials. Although meaningful change has occurred within the child health research ecosystem, measurable progress is still disappointingly slow. In this context, we identify and review emerging trends that will advance the agenda of increased clinical usefulness of pediatric trials, including patient and public engagement, Bayesian statistical approaches, adaptive designs, and platform trials. We explore how implementation science approaches could be applied to effect measurable improvements in the design, conducted, and reporting of child health research.

The methodological and reporting quality of randomized controlled trials of tyrosine kinase inhibitors for advanced differentiated thyroid cancer: Meta-research study

INTRODUCTION

Clinical trials on tyrosine kinase inhibitors (TKI) treatment have shown an improvement in overall and progression-free survival in patients with advanced differentiated thyroid cancer. However, it is necessary to evaluate these studies to assess methodological biases and inconsistencies that may impact the effects.

OBJECTIVE

To map and assess the methodological quality of randomized clinical trials (RCTs) regarding randomization, allocation concealment, blinding, and selective reporting bias.

METHODS

RCTs assessing the efficacy and safety of TKI for the treatment of advanced differentiated thyroid cancer were included. The search was performed in the MEDLINE database. The included RCTs were assessed for the adequacy of the methodological steps, as recommended by the Cochrane Risk of Bias tool.

RESULTS

Nine studies were analyzed, of which 77.7% were classified as low risk of bias regarding selective reporting and 33.3% as high risk of reporting bias. The mean time between protocol registration and study publication was approximately 5.11 years. Moreover, 66.7% were classified as low risk of bias for randomization and allocation concealment, and 33.3% did not specify the randomization process and allocation concealment in a way that would allow the identification of occurrences of bias. Concerning blinding of participants and outcome assessors, 77.8% of the RCTs reported adequate blinding and were classified as having a low risk of bias, 11.1% had a high risk of bias, and 11.1% had insufficient information and were classified as having unclear risk of bias. Regarding the blinding of the outcome assessors, 33.3% did the blinding correctly, 11.1% did not blind, and 55.6% did not provide enough information.

CONCLUSION

Overall, the assessed RCTs were predominantly at low risk of bias. The critical evaluation of these studies is essential to have confidence in the treatment estimated effect that will support clinical decision-making and provide information to preclude future clinical study flaws.

A methodological quality review of citations of randomized controlled trials of diabetes type2 in leading clinical practice guidelines and systematic reviews

Objective

Evaluate methodological quality of type 2 diabetes RCTs conducted in Iran and cited in clinical practice guidelines and systematic reviews and meta-analyses.

Methods

We conducted a descriptive methodological quality review, analyzing 286 Randomized Controlled Trials (RCTs) on diabetes mellitus published in Iran from July 2004 to 2021. We searched six databases systematically and evaluated eligible articles using the CONSORT 2010 checklist for abstracts. Two investigators assessed the data using a 17-item checklist derived from CONSORT. Additionally, we examined the citations of each RCT in 260 clinical practice guidelines, with a specific focus on the adequate reporting of outcomes.

Results

Out of 6667 articles, 286 analyzed. Poor reporting and failure to meet criteria observed. Only 3.8% cited in guidelines. Reporting rates: primary outcomes (41.9%), randomization (61.8%), trial recruitment (12.6%), blinding (50.8%). 27.9% cited in systematic reviews, 50.34% in systematic reviews and meta-analyses, 26.57% in meta-analyses. 67.8% of papers cited in systematic reviews. Adherence highest for participants, objective, randomization, intervention, outcome; lowest for recruitment, trial design, funding source, harms, and reporting primary outcomes.

Conclusions

Poor methodological reporting and adherence to CONSORT checklist in evaluated RCTs, especially in methodological sections. Improvements needed for reliable and applicable results in guidelines, reviews, and meta-analyses. Inadequate outcome reporting challenges researchers, clinicians, and policymakers, impacting evidence-based decision-making. Urgent improvements in RCT registration necessary.

Randomized controlled trials in nursing conducted by Latin American research teams: A scoping review

INTRODUCTION

Randomized controlled trials (RCTs) are the cornerstone of systematic reviews and other evidence synthesis. RCT identification remains challenging because of limitations in their indexation in major databases and potential language bias. Scientific production in Latin American nursing is steadily increasing, but little is known about its design or main features. We aimed to identify the extent of evidence from RCTs in nursing conducted by Latin American research teams and evaluate their main characteristics, including potential risk of bias.

DESIGN

Scoping review with risk of bias assessment.

METHODS

We conducted a scoping review including a comprehensive electronic search in five relevant databases. We completed a descriptive data analysis and a risk of bias assessment of eligible studies using Cochrane's guidance.

RESULTS

We identified 1784 references of which 47 were RCTs published in 40 journals. Twenty (42.6%) RCTs were published in journals in English. Chronic diseases were the most common health conditions studied (29.7%). Fifteen (31.9%) RCTs had a high risk of bias. Thirty (75%) journals were included in the Journal Citation Report (JCR) catalog and 5 (16.7%) were journals classified under nursing category. Twenty-one (52.5%) journals explicitly required CONSORT checklist recommendations for RCTs reporting.

CONCLUSION

Publication of RCTs in nursing by Latin American authors has increased. Most journals where RCTs are published are in English and not specific to nursing. Searches in journals of other disciplines may be necessary to facilitate identification of RCTs in nursing. CONSORT statements need to be actively promoted to facilitate rigorous methodology and reporting of RCTs.

CLINICAL RELEVANCE STATEMENT

This study highlights the need for an increased research focus on RCTs in nursing in Latin America, and the importance of enhancing the reporting quality of these studies to support evidence-based nursing practice.

Assessment of description and implementation fidelity of clinical trials involving exercise-based treatment in individuals with rotator cuff tears: a scoping review

BACKGROUND

For evidence-based practice, clinicians and researchers can rely on well-conducted randomized clinical trials that exhibit good methodological quality, provide adequate intervention descriptions, and implementation fidelity.

OBJECTIVE

To assess the description and implementation fidelity of exercise-based interventions in clinical trials for individuals with rotator cuff tears.

METHODS

A systematic search was conducted in PubMed, Embase, CINAHL, LILACS, Cochrane Library, Web of Science, SCOPUS and SciELO. Randomized clinical trials that assessed individuals with rotator cuff tears confirmed by imaging exam were included. All individuals must have received an exercise-based treatment. The methodological quality was scored with the Physiotherapy Evidence Database (PEDro) scale. The Template for Intervention Description and Replication (TIDieR) checklist and the National Institutes of Health Behaviour Change Consortium (NIHBCC) were used to assess intervention description and implementation fidelity, respectively.

RESULTS

A total of 13 studies were included. Despite their adequate methodological quality, the description of the intervention was poor with TIDieR scores ranging from 6 to 15 out of 24 total points. The TIDieR highest-scoring item was item 1 (brief name) that was reported in all studies. Considering fidelity, only one of the five domains of NIHBCC (i.e., treatment design) reached just over 50%.

CONCLUSION

Exercise-based interventions used in studies for individuals with rotator cuff tears are poorly reported. The description and fidelity of the intervention need to be better reported to assist clinical decision-making and support evidence-based practice.

Overview of randomized controlled trials of moderate to severe traumatic brain injury: A systematic review

BACKGROUND

Given the complexity of post-TBI medical, surgical, and rehabilitative care, research is critical to optimize interventions across the continuum of care and improve outcomes for persons with moderate to severe TBI.

OBJECTIVE

To characterize randomized controlled trials (RCTs) of moderate to severe traumatic brain injury (TBI) in the literature.

METHOD

Systematic searches of MEDLINE, PubMed, Scopus, CINAHL, EMBASE and PsycINFO for RCTs up to December 2022 inclusive were conducted in accordance with PRISMA guidelines.

RESULTS

662 RCTs of 91,946 participants published from 1978 to 2022 met inclusion criteria. The number of RCTs published annually has increased steadily. The most reported indicator of TBI severity was the Glasgow Coma Scale (545 RCTs, 82.3%). 432 (65.3%) RCTs focused on medical/surgical interventions while 230 (34.7%) addressed rehabilitation. Medical/surgical RCTs had larger sample sizes compared to rehabilitation RCTs. Rehabilitation RCTs accounted for only one third of moderate to severe TBI RCTs and were primarily conducted in the chronic phase post-injury relying on smaller sample sizes.

CONCLUSION

Further research in the subacute and chronic phases as well as increasing rehabilitation focused TBI RCTs will be important to optimizing the long-term outcomes and quality of life for persons living with TBI.

Change over Five Years in Important Measures of Methodological Quality and Reporting in Randomized Cardiovascular Clinical Trials

OBJECTIVES

The aim of our current study was to analyze whether the use of important measures of methodological quality and reporting of randomized clinical trials published in the field of cardiovascular disease research haschanged over time. A furtheraim was to investigate whether there was an improvement over time in the ability of these trials to provide a good estimate of the true intervention effect.

METHODS

We conducted two searches in the Cochrane Central Register of Controlled Trials (CENTAL) database to identify randomized cardiovascular clinical trials published in either 2012 or 2017. Randomized clinical trials (RCTs) trials in cardiovascular disease research with adult participants were eligible to be included. We randomly selected 250 RCTs for publication years 2012 and 2017. Trial characteristics, data on measures of methodological quality, and reporting were extracted and the risk of bias for each trial was assessed.

RESULTS

As compared to 2012, in 2017 there were significant improvements in the reporting of the presence of a data monitoring committee (42.0% in 2017 compared to 34.4% in 2012; p < 0.001), and a positive change in registering randomized cardiovascular disease research in clinical trial registries (78.4% in 2017 compared to 68.9% in 2012; p = 0.03). We also observed that significantly more RCTs reported sample size calculation (60.4% in 2017 compared to 49.6% in 2012; p < 0.01) in 2017 as compared to 2012. RCTs in 2017 were more likely to have a low overall risk of bias (RoB) than in 2012 (29.2% in 2017 compared to 21.2% in 2012; p < 0.01). However, fewer 2017 RCTs were rated low (50.8% compared to 65.6%; p < 0.001) risk for blinding of participants and personnel, for blinding of outcome assessors (82.4% compared to 90.8%; p < 0.001), and selective outcome reporting (62.8% compared to 80.0%; <0.001).

CONCLUSIONS

As compared to 2012, in 2017 there were significant improvements in some, but not all, the important measures of methodological quality. Although more trials in the field of cardiovascular disease research had a lower overall RoB in 2017, the improvement over time was not consistently perceived in all RoB domains.

Perspectives on systematic review protocol registration: a survey amongst stakeholders in the clinical research publication process

BACKGROUND

As systematic reviews (SRs) inform healthcare decisions, it is key that they address relevant questions and use rigorous methodology. Registration of SR protocols helps researchers identify relevant topics for future reviews and aims to prevent bias and duplication of effort. However, most SRs protocols are currently not registered, despite its significance. To guide future recommendations to enhance preregistration of SRs, it is important to gain a comprehensive understanding of the perspectives within the research community. Therefore, this study aims to examine the experiences with and factors of influence (barriers and facilitators) on prospective SR registration amongst researchers, peer reviewers and journal editors.

METHODS

Two different surveys were distributed to two groups: researchers and journal editors both identified from an existing sample of SRs. Researchers who indicated to have peer reviewed a SR were surveyed on their perspectives as peer reviewers as well. Survey design and analysis were informed by the Consolidated Framework for Implementation Research (CFIR). Shared and unique subthemes from the perspectives of researchers, peer reviewers and journal editors were identified and linked to the SR registration process (Innovation), to team, organisation (Inner setting) and (inter)national research community (Outer setting), and to characteristics of researchers, peer reviewers or journal editors (Individuals).

RESULTS

The survey's response rates were 65/727 (9%) for researchers, of which 37 were peer reviewers, and 22/308 (7%) for journal editors. Most respondents (n = 76, 94%) were familiar with SR protocol registration and 81% of researchers had registered minimally one SR protocol. Shared SR registration process subthemes were the importance and advantages of SR protocol registration, as well as barriers such as a high administrative burden. Shared subthemes regarding the inner and outer setting centred on journal processes, external standards and time. Shared individual factors were knowledge, skills and awareness.

CONCLUSIONS

The majority of the respondents were familiar with SR protocol registration and had a positive attitude towards it. This study identified suboptimal registration process, administrative burden and lack of mandatory SR protocol registration as barriers. By overcoming these barriers, SR protocol registration could contribute more effectively to the goals of open science.

SYSTEMATIC REVIEW REGISTRATION

osf.io/gmv6z.

A re-analysis of 150 women's health trials to investigate how the Bayesian approach may offer a solution to the misinterpretation of statistical findings

OBJECTIVE

To investigate whether a Bayesian interpretation might help prevent misinterpretation of statistical findings and support authors to differentiate evidence of no effect from statistical uncertainty.

DESIGN

A Bayesian re-analysis to determine posterior probabilities of clinically important effects (e.g., a large effect is set at a 4 percentage point difference and a trivial effect to be within a 0.5 percentage point difference). Posterior probabilities greater than 95% are considered as strong statistical evidence, and less than 95% as inconclusive.

SAMPLE

150 major women's health trials with binary outcomes.

MAIN OUTCOME MEASURES

Posterior probabilities of large, moderate, small and trivial effects.

RESULTS

Under frequentist methods, 48 (32%) were statistically significant (p-value ≤ 0.05) and 102 (68%) statistically non-significant. The frequentist and Bayesian point estimates and confidence intervals showed strong concordance. Of the statistically non-significant trials (n = 102), the Bayesian approach classified the majority (94, 92%) as inconclusive, neither able to confirm or refute effectiveness. A small number of statistically non-significant findings (8, 8%) were classified as having strong statistical evidence of an effect.

CONCLUSIONS

Whilst almost all trials report confidence intervals, in practice most statistical findings are interpreted on the basis of statistical significance, mostly concluding evidence of no effect. Findings here suggest the majority are likely uncertain. A Bayesian approach could help differentiate evidence of no effect from statistical uncertainty.

Chronic pain trials often exclude people with comorbid depressive symptoms: A secondary analysis of 346 randomized controlled trials

BACKGROUND

Chronic pain and depression are common comorbid conditions, but there is limited evidence-based guidance for management of the two conditions together. In recent years, there has been an increase in the number of chronic pain randomized controlled trials that collect depression outcomes, but it is unknown how often these trials include people with depression or significant depressive symptoms. If trials do not include participants representative of real-world populations, evidence and guidance generated from these trials risk being inapplicable for large proportions of the target population, or worse, risk harm. Thus, in order to identify pathways to improve the conduct of clinical trials, the aims of this study were to (1) estimate the proportion of randomized controlled trials evaluating chronic pain interventions and reporting depression outcomes that include participants with significant depressive symptoms; and (2) assess the variability of inclusion proportions by pain type, intervention type, gender, country of origin, and publication year.

METHODS

Studies were extracted from an umbrella review of interventions for chronic pain that reported depression outcomes. Screening and data extraction were completed in duplicate and conflicts were resolved by a third author. Randomized controlled trials with at least 50% adult participants and validated depression scales were included, and randomized controlled trials with populations whose mean scores were at or above depression thresholds at baseline were considered to have included participants with depression.

RESULTS

Of the 346 randomized controlled trials analyzed, 142 (41%) included participants with depression. Eight pain-type groups and nine intervention types were identified. Randomized controlled trials investigating fibromyalgia and mixed chronic pain had the highest proportion of participants with depression, whereas studies of arthritis and axial pain had among the lowest. Randomized controlled trials from the United States had a significantly lower inclusion proportion compared with non-US studies, especially for studies on arthritis. The increase in inclusion proportion by publication year was driven by the increase in fibromyalgia studies.

DISCUSSION AND CONCLUSION

This study highlights opportunities to improve the conduct of chronic pain clinical trials. The majority of randomized controlled trials s analyzed evaluated participants without significant depressive symptoms at baseline, thus the findings synthesized in systematic reviews and subsequent guidelines are most applicable to the subset of real-world populations that do not have significant depressive symptoms. As well, systemic biases around psychological conditions and gender may be important contributors to differences in the study of depression in fibromyalgia compared with common conditions such as arthritis and axial pain. In order to better inform clinical practice, future research must intentionally include individuals with comorbid depression in trials of common chronic pain conditions, and consider methods to mitigate biases that may distort study design.

Supporting translation of research evidence into practice-the use of Normalisation Process Theory to assess and inform implementation within randomised controlled trials: a systematic review

BACKGROUND

The status of randomised controlled trials (RCTs) as the 'gold standard' for evaluating efficacy in healthcare interventions is increasingly debated among the research community, due to often insufficient consideration for implementation. Normalisation Process Theory (NPT), which focuses on the work required to embed processes into practice, offers a potentially useful framework for addressing these concerns. While the theory has been deployed in numerous RCTs to date, more work is needed to consolidate understanding of if, and how, NPT may aid implementation planning and processes within RCTs. Therefore, this review seeks to understand how NPT contributes to understanding the dynamics of implementation processes within RCTs. Specifically, this review will identify and characterise NPT operationalisation, benefits and reported challenges and limitations in RCTs.

METHODS

A qualitative systematic review with narrative synthesis of peer-reviewed journal articles from eight databases was conducted. Studies were eligible for inclusion if they reported sufficient detail on the use of NPT within RCTs in a healthcare domain. A pre-specified data extraction template was developed based on the research questions of this review. A narrative synthesis was performed to identify recurrent findings.

RESULTS

Searches identified 48 articles reporting 42 studies eligible for inclusion. Findings suggest that NPT is primarily operationalised prospectively during the data collection stage, with limited sub-construct utilisation overall. NPT is beneficial in understanding implementation processes by aiding the identification and analysis of key factors, such as understanding intervention fidelity in real-world settings. Nearly three-quarters of studies failed to report the challenges and limitations of utilising NPT, though coding difficulties and data falling outside the NPT framework are most common.

CONCLUSIONS

NPT appears to be a consistent and generalisable framework for explaining the dynamics of implementation processes within RCTs. However, operationalisation of the theory to its full extent is necessary to improve its use in practice, as it is currently deployed in varying capacities. Recommendations for future research include investigation of NPT alongside other frameworks, as well as earlier operationalisation and greater use of NPT sub-constructs.

TRIAL REGISTRATION

The protocol for this systematic review was accepted for public registration on PROSPERO (registration number: CRD42022345427) on 26 July 2022.

What is the quality of reporting in randomized controlled trials in spinal conditions

Purpose

Substandard quality across published randomized controlled trials (RCTs) is a major concern. Imperfect reporting has the potential to distort the evidence landscape and waste valuable health-care resources. In this study, we aim to assess the current quality of reporting in the field of spine using a modified version of the Consolidated Standards of Reporting Trials (CONSORT) checklist.

Materials and Methods

A list of published RCTs in the field of spine disease from January 1, 2013, to December 31, 2020, was built. Two reviewers scored the published RCTs against a modified CONSORT checklist. The mean adjusted CONSORT scores for each study, reporting category, and checklist item were calculated.

Results

The mean and median scores across all of the RCTs were 0.72 and 0.74 out of 1.00, respectively. The spectrum of scores was wide, ranging from 0.45 to 0.94. The reporting categories with the lowest score included randomization, blinding, and abstract. The items which were most under-reported included allocation sequence generation, type of randomization used, full trial protocol details, and abstract methodology. The inter-rater reliability between our reviewers was substantial (κ = 0.7, κ = 0.71).

Conclusion

Our findings correlate with only a moderate level of compliance to the CONSORT criteria on the quality of reporting for RCTs in spinal conditions. This is in line with previous reports on compliance, both within and outside the field of spinal conditions. Further continued and sustained efforts are still required to enhance the quality and consistency of RCT reporting, ultimately reducing health-care resource wastage and improving patient safety.

Methodology reporting improved over time in 176,469 randomized controlled trials

OBJECTIVES

To describe randomized controlled trial (RCT) methodology reporting over time.

STUDY DESIGN AND SETTING

We used a deep learning-based sentence classification model based on the Consolidated Standards of Reporting Trials (CONSORT) statement, considered minimum requirements for reporting RCTs. We included 176,469 RCT reports published between 1966 and 2018. We analyzed the reporting trends over 5-year time periods, grouping trials from 1966 to 1990 in a single stratum. We also explored the effect of journal impact factor (JIF) and medical discipline.

RESULTS

Population, Intervention, Comparator, Outcome (PICO) items were commonly reported during each period, and reporting increased over time (e.g., interventions: 79.1% during 1966-1990 to 87.5% during 2010-2018). Reporting of some methods information has increased, although there is room for improvement (e.g., sequence generation: 10.8-41.8%). Some items are reported infrequently (e.g., allocation concealment: 5.1-19.3%). The number of items reported and JIF are weakly correlated (Pearson's r (162,702) = 0.16, P < 0.001). The differences in the proportion of items reported between disciplines are small (<10%).

CONCLUSION

Our analysis provides large-scale quantitative support for the hypothesis that RCT methodology reporting has improved over time. Extending these models to all CONSORT items could facilitate compliance checking during manuscript authoring and peer review, and support metaresearch.

Efficacy and safety of traditional Chinese medicine adjuvant therapy for severe pneumonia: evidence mapping of the randomized controlled trials, systematic reviews, and meta-analyses

Background and Objective: Severe pneumonia is a critical respiratory disease with high mortality. There is insufficient evidence on the efficacy and safety of traditional Chinese medicine (TCM) adjuvant therapy for severe pneumonia. This study aims to identify, describe, assess, and summarize the currently available high-quality design evidence on TCM adjuvant therapy for severe pneumonia to identify evidence gaps using the evidence mapping approach. Methods: Systematic searches were performed on English and Chinese online databases (PubMed, EMBASE, Cochrane Library, Web of Science, CNKI, WanFang Data, CQVIP, and SinoMed) to identify papers from inception until August 2023 for inclusion into the review. Randomized controlled trials (RCTs), systematic reviews (SRs), and meta-analyses concerning TCM adjuvant therapy for severe pneumonia or its complications in adults were included. The risk of bias in RCTs was evaluated by using the Cochrane Handbook ROB tool. The Assessment of Multiple Systematic Reviews 2 (AMSTAR-2), the Risk of Bias in Systematic Review (ROBIS) tool, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system were used to assess the methodological quality, risk of bias, and evidence quality of SRs or meta-analyses, respectively. Then, a bubble plot was designed to visually display information in four dimensions. Results: A total of 354 RCTs and 17 SRs or meta-analyses met the inclusion criteria. The published RCTs had several flaws, such as unreasonable design, limited sample size, insufficient attention to non-drug therapy studies and syndrome differentiation, improper selection or use of outcome indicators, and failure to provide high-quality evidence. Sixteen SRs or meta-analyses of methodological quality scored "Critically Low" confidence. Twelve SRs or meta-analyses were rated as "High Risk." Most outcomes were rated as "Low" evidence quality. We found that TCM combined with conventional treatment could improve the clinical total effective rate and the TCM syndromes efficacy. The combined approach could also shorten mechanical ventilation time, infection control time, and length of hospital and ICU stay; significantly reduce temperature, respiratory rate, heart rate, white blood cell counts, levels of C-reactive protein, procalcitonin, blood inflammatory factors, bacteriological response, and D-dimer; decrease CPIS, APACHE II score, and PSI score; improve pulmonary imaging features, arterial blood gas indicators (including arterial oxygen pressure, arterial oxygen saturation, and oxygen index), and lung function (including forced vital capacity and forced expiratory volume in the first second) for severe pneumonia compared with conventional treatment only (p < 0.05). There was no significant difference in adverse reactions and incidence of adverse events (p > 0.05). In addition, compared with conventional treatment only, most SRs or meta-analyses concluded that TCM combined with conventional treatment was "Beneficial" or "Probably beneficial." Conclusion: TCM combined with conventional treatment had advantages in efficacy, clinical signs, laboratory results, and life quality outcomes of severe pneumonia, with no difference in safety outcomes compared with conventional treatment only. QingJin Huatan decoction is the most promising target, and Xuanbai Chengqi decoction has a "Probably beneficial" conclusion. XueBiJing injection and TanReQing injection are two commonly used Chinese herbal injections for treating severe pneumonia, and both are "Probably beneficial." However, there was a need for multicenter RCTs with large sample sizes and high methodological quality in the future. In addition, the methodological design and quality of SRs or meta-analyses should be improved to form high-quality, evidence-based medical evidence and provide evidence for the effectiveness and safety of TCM adjuvant therapy for severe pneumonia.

Two decades of surgical randomized controlled trials: worldwide trends in volume and methodological quality

BACKGROUND

RCTs are essential in guiding clinical decision-making but are difficult to perform, especially in surgery. This review assessed the trend in volume and methodological quality of published surgical RCTs over two decades.

METHODS

PubMed was searched systematically for surgical RCTs published in 1999, 2009, and 2019. The primary outcomes were volume of trials and RCTs with a low risk of bias. Secondary outcomes were clinical, geographical, and funding characteristics.

RESULTS

Some 1188 surgical RCTs were identified, of which 300 were published in 1999, 450 in 2009, and 438 in 2019. The most common subspecialty in 2019 was gastrointestinal surgery (50.7 per cent). The volume of surgical RCTs increased mostly in Asia (61, 159, and 199 trials), especially in China (7, 40, and 81). In 2019, countries with the highest relative volume of published surgical RCTs were Finland and the Netherlands. Between 2009 and 2019, the proportion of RCTs with a low risk of bias increased from 14.7 to 22.1 per cent (P = 0.004). In 2019, the proportion of trials with a low risk of bias was highest in Europe (30.5 per cent), with the UK and the Netherlands as leaders in this respect.

CONCLUSION

The volume of published surgical RCTs worldwide remained stable in the past decade but their methodological quality improved. Considerable geographical shifts were observed, with Asia and especially China leading in terms of volume. Individual European countries are leading in their relative volume and methodological quality of surgical RCTs.

Pilot trials may improve the quality of full-scale trials: a meta-research study

OBJECTIVES

Evidence on the value of pilot trials for subsequent trial's quality is scarce. This study aims to determine if a pilot trial improves the quality of the full-scale trial.

STUDY DESIGN AND SETTING

We searched PubMed for pilot trials and their subsequent full-scale trials. The meta-analysis of the full-scale trials was used to identify other full-scale trials on the same research topic but without a pilot trial. Markers of trial quality included publication outcomes and Cochrane Risk of Bias (RoB) assessment.

RESULTS

Fifty-eight full-scale trials with a pilot trial and 151 full-scale trials without were identified from 47 meta-analyses. Trials with a pilot trial were published 0.9 years sooner (mean ± standard deviation: 1.7 ± 1.0 vs. 2.6 ± 2.0, P = 0.005) and in peer-reviewed journals with higher impact factors (60.9 ± 75.0 vs. 24.8 ± 50.3, P < 0.001). A pilot trial's presence was associated with lower risk of bias in full-scale trial random sequence generation (OR [95% CI]: 4.05 [1.27-12.91]), allocation concealment (2.89 [1.07-7.83]), and participants/researchers masking (4.31 [1.37-13.50]), but not outcome assessment masking (1.03 [0.49-2.18]), incomplete outcome data (1.27 [0.47-3.42]), and selective reporting (1.23 [0.44-3.46]).

CONCLUSION

Conducting a pilot trial may enhance the quality of the subsequent full-scale trial.

Analysis of predictive performance and reliability of classifiers for quality assessment of medical evidence revealed important variation by medical area

OBJECTIVES

A major obstacle in deployment of models for automated quality assessment is their reliability. To analyze their calibration and selective classification performance.

STUDY DESIGN AND SETTING

We examine two systems for assessing the quality of medical evidence, EvidenceGRADEr and RobotReviewer, both developed from Cochrane Database of Systematic Reviews (CDSR) to measure strength of bodies of evidence and risk of bias (RoB) of individual studies, respectively. We report their calibration error and Brier scores, present their reliability diagrams, and analyze the risk-coverage trade-off in selective classification.

RESULTS

The models are reasonably well calibrated on most quality criteria (expected calibration error [ECE] 0.04-0.09 for EvidenceGRADEr, 0.03-0.10 for RobotReviewer). However, we discover that both calibration and predictive performance vary significantly by medical area. This has ramifications for the application of such models in practice, as average performance is a poor indicator of group-level performance (e.g., health and safety at work, allergy and intolerance, and public health see much worse performance than cancer, pain, and anesthesia, and Neurology). We explore the reasons behind this disparity.

CONCLUSION

Practitioners adopting automated quality assessment should expect large fluctuations in system reliability and predictive performance depending on the medical area. Prospective indicators of such behavior should be further researched.

Assessing the Integrity of Clinical Trials Included in Evidence Syntheses

Evidence syntheses of randomized clinical trials (RCTs) offer the highest level of scientific evidence for informing clinical practice and policy. The value of evidence synthesis itself depends on the trustworthiness of the included RCTs. The rising number of retractions and expressions of concern about the authenticity of RCTs has raised awareness about the existence of problematic studies, sometimes called "zombie" trials. Research integrity, i.e., adherence to ethical and professional standards, is a multi-dimensional concept that is incompletely evaluated for the RCTs included in current evidence syntheses. Systematic reviewers tend to rely on the editorial and peer-review system established by journals as custodians of integrity of the RCTs they synthesize. It is now well established that falsified and fabricated RCTs are slipping through. Thus, RCT integrity assessment becomes a necessary step in systematic reviews going forward, in particular because RCTs with data-related integrity concerns remain available for use in evidence syntheses. There is a need for validated tools for systematic reviewers to proactively deploy in the assessment of integrity deviations without having to wait for RCTs to be retracted by journals or expressions of concern issued. This article analyzes the issues and challenges in conducting evidence syntheses where the literature contains RCTs with possible integrity deficits. The way forward in the form of formal RCT integrity assessments in systematic reviews is proposed, and implications of this new initiative are discussed. Future directions include emphasizing ethical and professional standards, providing tailored integrity-specific training, and creating systems to promote research integrity, as improvements in RCT integrity will benefit evidence syntheses.

The science of child and adolescent mental health in Greece: a nationwide systematic review

Evidence-based information is essential for effective mental health care, yet the extent and accessibility of the scientific literature are critical barriers for professionals and policymakers. To map the necessities and make validated resources accessible, we undertook a systematic review of scientific evidence on child and adolescent mental health in Greece encompassing three research topics: prevalence estimates, assessment instruments, and interventions. We searched Pubmed, Web of Science, PsycINFO, Google Scholar, and IATPOTEK from inception to December 16th, 2021. We included studies assessing the prevalence of conditions, reporting data on assessment tools, and experimental interventions. For each area, manuals informed data extraction and the methodological quality were ascertained using validated tools. This review was registered in protocols.io [68583]. We included 104 studies reporting 533 prevalence estimates, 223 studies informing data on 261 assessment instruments, and 34 intervention studies. We report the prevalence of conditions according to regions within the country. A repository of locally validated instruments and their psychometrics was compiled. An overview of interventions provided data on their effectiveness. The outcomes are made available in an interactive resource online [ https://rpubs.com/camhi/sysrev_table ]. Scientific evidence on child and adolescent mental health in Greece has now been cataloged and appraised. This timely and accessible compendium of up-to-date evidence offers valuable resources for clinical practice and policymaking in Greece and may encourage similar assessments in other countries.

Fracture risk reduction and safety by osteoporosis treatment compared with placebo or active comparator in postmenopausal women: systematic review, network meta-analysis, and meta-regression analysis of randomised clinical trials

OBJECTIVE

To review the comparative effectiveness of osteoporosis treatments, including the bone anabolic agents, abaloparatide and romosozumab, on reducing the risk of fractures in postmenopausal women, and to characterise the effect of antiosteoporosis drug treatments on the risk of fractures according to baseline risk factors.

DESIGN

Systematic review, network meta-analysis, and meta-regression analysis of randomised clinical trials.

DATA SOURCES

Medline, Embase, and Cochrane Library to identify randomised controlled trials published between 1 January 1996 and 24 November 2021 that examined the effect of bisphosphonates, denosumab, selective oestrogen receptor modulators, parathyroid hormone receptor agonists, and romosozumab compared with placebo or active comparator.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomised controlled trials that included non-Asian postmenopausal women with no restriction on age, when interventions looked at bone quality in a broad perspective. The primary outcome was clinical fractures. Secondary outcomes were vertebral, non-vertebral, hip, and major osteoporotic fractures, all cause mortality, adverse events, and serious cardiovascular adverse events.

RESULTS

The results were based on 69 trials (>80 000 patients). For clinical fractures, synthesis of the results showed a protective effect of bisphosphonates, parathyroid hormone receptor agonists, and romosozumab compared with placebo. Compared with parathyroid hormone receptor agonists, bisphosphonates were less effective in reducing clinical fractures (odds ratio 1.49, 95% confidence interval 1.12 to 2.00). Compared with parathyroid hormone receptor agonists and romosozumab, denosumab was less effective in reducing clinical fractures (odds ratio 1.85, 1.18 to 2.92 for denosumab v parathyroid hormone receptor agonists and 1.56, 1.02 to 2.39 for denosumab v romosozumab). An effect of all treatments on vertebral fractures compared with placebo was found. In the active treatment comparisons, denosumab, parathyroid hormone receptor agonists, and romosozumab were more effective than oral bisphosphonates in preventing vertebral fractures. The effect of all treatments was unaffected by baseline risk indicators, except for antiresorptive treatments that showed a greater reduction of clinical fractures compared with placebo with increasing mean age (number of studies=17; β=0.98, 95% confidence interval 0.96 to 0.99). No harm outcomes were seen. The certainty in the effect estimates was moderate to low for all individual outcomes, mainly because of limitations in reporting, nominally indicating a serious risk of bias and imprecision.

CONCLUSIONS

The evidence indicated a benefit of a range of treatments for osteoporosis in postmenopausal women for clinical and vertebral fractures. Bone anabolic treatments were more effective than bisphosphonates in the prevention of clinical and vertebral fractures, irrespective of baseline risk indicators. Hence this analysis provided no clinical evidence for restricting the use of anabolic treatment to patients with a very high risk of fractures.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42019128391.

From Protocol to Definitive Study-The State of Randomized Controlled Trial Evidence in Sports Medicine Research: A Systematic Review and Survey Study

OBJECTIVE

To evaluate the progression, quality, and challenges associated with conducting and publishing randomized controlled trials (RCTs) in sports medicine.

DESIGN

Systematic review and survey.

SETTING

MEDLINE and Embase were searched for all publications before September 17, 2021. A targeted search of clinicaltrials.gov , BMC Musculoskeletal Disorders, PubMed, and Google Scholar were also conducted. The survey was administered to authors using REDCap.

PARTICIPANTS

Where the systematic search revealed no corresponding published definitive trial, authors of the published pilots were surveyed.

INTERVENTIONS

Survey assessing limitations to definitive trials.

MAIN OUTCOME MEASURES

Protocol/method articles, pilot articles, and relevant clinical trial registry records with corresponding definitive trials were pooled.

RESULTS

Our literature search yielded 27 006 studies; of which, we included 208 studies (60 (28.8%) pilot RCTs, 84 (40.4%) protocol/method articles, and 64 (30.8%) trial registry records). From these, 44 corresponding definitive RCTs were identified. Pilot study and definitive RCT methodological quality increased on average most significantly during the duration of this review (30.6% and 8.2%). Of the 176 authors surveyed, 59 (33.5%) responded; 24.6% (14/57) stated that they completed an unpublished definitive trial, while 52.6% (30/57) reported having one underway.

CONCLUSIONS

The quality and number of RCT publications within the field of sports medicine has been increasing since 1999. The number of sports medicine-related protocol and pilot articles preceding a definitive trial publication showed a sharp increase over the past 10 years, although only 5 pilot studies have progressed to a definitive RCT.

Health outcomes in primary care: a 20-year evidence map of randomized controlled trials

OBJECTIVE

To quantify the different types of health outcomes assessed as primary outcomes in randomized controlled trials (RCTs) in the primary care (PC) setting during the last 20 years and identify whether potential gaps exist in specific types of health care and types of intervention.

METHODS

We systematically searched PubMed, Scopus, and Cochrane Central Register of Controlled Trials, from January 2000 to September 2020 for published RCTs in PC. We recorded characteristics of eligible studies and mapped evidence by health outcome category (patient health outcomes, health services outcomes); and for each outcome category, by types of health care (preventive, acute, chronic, palliative), and by types of intervention (drug, behavioural, on structure, and on process). For RCTs assessing patient health outcomes as primary outcomes, we further mapped using the quality-of-care dimensions, that is, effectiveness, safety, and patient-centredness.

RESULTS

Of the 518 eligible RCTs in PC, 357 (68.9%) evaluated a patient health outcome as the primary outcome, and 161 (31.1%) evaluated only health services outcomes as primary outcomes. Many focused on population with chronic illness (224 trials; 43.2%) and evaluated interventions on processes of health care (239 trials; 46.1%). Research gaps identified include preventive and palliative care, behavioural interventions, and safety and patient-centredness outcomes as primary outcomes.

CONCLUSION

Our evidence map showed research gaps in certain types of health care and interventions. It also showed research gaps in assessing safety and measures to place patient at the centre of health care delivery as primary outcomes.

Indicators of questionable research practices were identified in 163,129 randomized controlled trials

OBJECTIVES

To explore indicators of the following questionable research practices (QRPs) in randomized controlled trials (RCTs): (1) risk of bias in four domains (random sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment); (2) modifications in primary outcomes that were registered in trial registration records (proxy for selective reporting bias); (3) ratio of the achieved to planned sample sizes; and (4) statistical discrepancy.

STUDY DESIGN AND SETTING

Full texts of all human RCTs published in PubMed in 1996-2017 were automatically identified and information was collected automatically. Potential indicators of QRPs included author-specific, publication-specific, and journal-specific characteristics. Beta, logistic, and linear regression models were used to identify associations between these potential indicators and QRPs.

RESULTS

We included 163,129 RCT publications. The median probability of bias assessed using Robot Reviewer software ranged between 43% and 63% for the four risk of bias domains. A more recent publication year, trial registration, mentioning of CONsolidated Standards Of Reporting Trials-checklist, and a higher journal impact factor were consistently associated with a lower risk of QRPs.

CONCLUSION

This comprehensive analysis provides an insight into indicators of QRPs. Researchers should be aware that certain characteristics of the author team and publication are associated with a higher risk of QRPs.

Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Reporting guidelines are tools to help improve the transparency, completeness, and clarity of published articles in health research. Specifically, the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statements provide evidence-based guidance on what to include in randomised trial articles and protocols to guarantee the efficacy of interventions. These guidelines are subsequently described and discussed in journal articles and used to produce checklists. Determining the online impact (i.e., number and type of links received) of these articles can provide insights into the dissemination of reporting guidelines in broader environments (web-at-large) than simply that of the scientific publications that cite them. To address the technical limitations of link analysis, here the Debug-Validate-Access-Find (DVAF) method is designed and implemented to measure different facets of the guidelines' online impact. A total of 65 articles related to 38 reporting guidelines are taken as a baseline, providing 240,128 URL citations, which are then refined, analysed, and categorised using the DVAF method. A total of 15,582 links to journal articles related to the CONSORT and SPIRIT initiatives were identified. CONSORT 2010 and SPIRIT 2013 were the reporting guidelines that received most links (URL citations) from other online objects (5328 and 2190, respectively). Overall, the online impact obtained is scattered (URL citations are received by different article URL IDs, mainly from link-based DOIs), narrow (limited number of linking domain names, half of articles are linked from fewer than 29 domain names), concentrated (links come from just a few academic publishers, around 60% from publishers), non-reputed (84% of links come from dubious websites and fake domain names) and highly decayed (89% of linking domain names were not accessible at the time of the analysis). In light of these results, it is concluded that the online impact of these guidelines could be improved, and a set of recommendations are proposed to this end.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11192-022-04542-z.

Survey of quality and clarity of methods and results reporting in 1 year of intervention studies published in high-impact medical and psychiatric journals

OBJECTIVE

We assessed how well articles in major medical and psychiatric journals followed best reporting practices in presenting results of intervention studies.

METHOD

Standardised data collection was used to review studies in high-impact and widely read medical (JAMA, Lancet and New England Journal of Medicine) and psychiatric (American Journal of Psychiatry, JAMA Psychiatry, Journal of Clinical Psychiatry and Lancet Psychiatry) journals, published between 1 September 2018 and 31 August 2019. Two team members independently reviewed each article.

MEASURES

The primary outcome measure was proportion of papers reporting consensus elements required to understand and evaluate the results of the intervention. The secondary outcome measure was comparison of complete and accessible reporting in the major medical versus the major psychiatric journals.

RESULTS

One hundred twenty-seven articles were identified for inclusion. At least 90% of articles in both medical and psychiatric journals included sample size, statistical significance, randomisation method, elements of study flow, and age, sex, and illness severity by randomisation group. Selected elements less frequently reported by either journal type were confidence intervals in the abstract, reported in 93% (95% CI 84% to 97%) of medical journal articles and 58% (95% CI 45% to 69%) of psychiatric journal articles, and sample size method (93%, 95% CI 84% to 97% medical; 69%, 95% CI 57% to 80% psychiatric), race and ethnicity by randomisation group (51%, 95% CI 40% to 63% medical; 73%, 95% CI 60% to 83% psychiatric), and adverse events (94%; 95% CI 86% to 98% medical; 80%, 95% CI 68% to 88% psychiatric) in the main text. CIs were included less often in psychiatric than medical journals (p<0.004 abstract, p=0.04 main text, after multiple-testing correction).

CONCLUSIONS

Recommendations include standard inclusion of a table specifying the outcome(s) designated as primary, and the sample size, effect size(s), CI(s) and p value(s) corresponding to the primary test(s) for efficacy.

A systematic review of N-of-1 trials and single case experimental designs in physiotherapy for musculoskeletal conditions

BACKGROUND

Single Case Experimental Designs (SCEDs) are especially useful for small heterogeneous samples. Their role in evaluation of physiotherapy interventions for musculoskeletal conditions has not been systematically reviewed.

OBJECTIVES

Systematically review use, purpose, and outcomes of SCEDs for physiotherapy interventions for musculoskeletal conditions.

DATA SOURCES

Electronic databases and grey literature, searched using pre-defined terms.

STUDY SELECTION OR ELIGIBILITY CRITERIA

Studies of human participants enrolled in eligible SCEDs (individual or a series).

STUDY APPRAISAL AND SYNTHESIS METHODS

We extracted study characteristics, analytic methods and results, synthesising these descriptively. We used RoBiN-T scale to assess risk of bias.

RESULTS

We included 19 SCEDs comprising 92 participants, with wide variability in design, methodology, analysis and in conditions and interventions evaluated. 95% of participants responded favourably to the tested intervention. Overall risk of bias was high, due to poor internal validity, especially regarding randomisation, blinding, inter-rater agreement and measurement of treatment adherence. Visual analysis alone was performed in 55% of studies. Assessment of provider and participant satisfaction was limited.

CONCLUSIONS AND IMPLICATIONS

of key findings: SCEDs may be well-suited to evaluation of physiotherapy interventions for musculoskeletal conditions, but the risk of bias in studies to date is high. Following SCED guidelines to minimize the risk of bias and maximise clinical usefulness is recommended.

Evidence for the beneficial effect of ketamine in the treatment of patients with post-traumatic stress disorder: A systematic review and meta-analysis

Post-traumatic stress disorder (PTSD) is an anxiety disorder with manifestations somatic resulting from reliving the trauma. The therapy for the treatment of PTSD has limitations, between reduced efficacy and "PTSD pharmacotherapeutic crisis". Scientific evidence has shown that the use of ketamine has benefits for the treatment of depressive disorders and other symptoms present in PTSD compared to other conventional therapies. Therefore, this study aims to analyze the available evidence on the effect of ketamine in the treatment of post-traumatic stress. The systematic review and the meta-analysis were conducted following PRISMA guidelines and RevManager software, using randomized controlled trials and eligible studies of quality criteria for data extraction and analysis. The sample design evaluated included the last ten years, whose search resulted in 594 articles. After applying the exclusion criteria, 35 articles were selected, of which 14 articles were part of the sample, however, only six articles were selected the meta-analysis. The results showed that the ketamine is a promising drug in the management of PTSD with effect more evident performed after 24 h evaluated by MADRS scale. However, the main limitations of the present review demonstrate that more high-quality studies are needed to investigate the influence of therapy, safety, and efficacy.

Differential effects of intermittent energy restriction vs. continuous energy restriction combined high-intensity interval training on overweight/obese adults: A randomized controlled trial

BACKGROUND

Intermittent energy restriction (IER) and continuous energy restriction (CER) are increasingly popular dietary approaches used for weight loss and overall health. These energy restriction protocols combined with exercise on weight loss and other health outcomes could achieve additional effects in a short-term intervention.

OBJECTIVES

To evaluate the effects of a 4-week IER or CER program on weight, blood lipids, and CRF in overweight/obese adults when combined with high-intensity interval training (HIIT).

METHODS

Forty-eight overweight/obese adults [age: 21.3 ± 2.24 years, body mass index (BMI): 25.86 ± 2.64 kg⋅m-2] were randomly assigned to iER, cER, and normal diet (ND) groups (n = 16 per group), each consisting of a 4-week intervention. All of the groups completed HIIT intervention (3 min at 80% of V̇O2max followed by 3 min at 50% of V̇O2max ), 30 min/training sessions, five sessions per week. iER subjects consumed 30% of energy needs on 2 non-consecutive days/week, and 100% of energy needs on another 5 days; cER subjects consumed 70% of energy needs; and ND subjects consumed 100% of energy needs. Body composition, waist circumference (WC) and hip circumference (HC), triglyceride (TG), total cholesterol (TC), low-density lipoprotein-cholesterol (LDL-c), high-density lipoprotein-cholesterol (HDL-c), and cardiorespiratory fitness (CRF) were measured before and after the intervention.

RESULTS

Of the total 57 participants who underwent randomization, 48 (84.2%) completed the 4-week intervention. After intervention body composition and body circumference decreased in three groups, but no significant differences between groups. The iER tends to be superior to cER in the reduction of body composition and body circumference. The mean body weight loss was 4.57 kg (95% confidence interval [CI], 4.1-5.0, p < 0.001) in iER and 2.46 kg (95% CI, 4.1-5.0, p < 0.001) in iER. The analyses of BMI, BF%, WC, and HC were consistent with the primary outcome results. In addition, TG, TC, HDL-c, and CRF improved after intervention but without significant changes (p > 0.05).

CONCLUSION

Both IER and CER could be effective in weight loss and increased CRF when combined with HIIT. However, iER showed greater benefits for body weight, BF%, WC, and HC compared with cER.

Treatment efficacy and effectiveness in adults with major depressive disorder and childhood trauma history: a systematic review and meta-analysis

BACKGROUND

Childhood trauma is a common and potent risk factor for developing major depressive disorder in adulthood, associated with earlier onset, more chronic or recurrent symptoms, and greater probability of having comorbidities. Some studies indicate that evidence-based pharmacotherapies and psychotherapies for adult depression might be less efficacious in patients with a history of childhood trauma than patients without childhood trauma, but findings are inconsistent. Therefore, we examined whether individuals with major depressive disorder, including chronic forms of depression, and a reported history of childhood trauma, had more severe depressive symptoms before treatment, had more unfavourable treatment outcomes following active treatments, and were less likely to benefit from active treatments relative to a control condition, compared with individuals with depression without childhood trauma.

METHODS

We did a comprehensive meta-analysis (PROSPERO CRD42020220139). Study selection combined the search of bibliographical databases (PubMed, PsycINFO, and Embase) from Nov 21, 2013, to March 16, 2020, and full-text randomised clinical trials (RCTs) identified from several sources (1966 up to 2016-19) to identify articles in English. RCTs and open trials comparing the efficacy or effectiveness of evidence-based pharmacotherapy, psychotherapy, or combination intervention for adult patients with depressive disorders and the presence or absence of childhood trauma were included. Two independent researchers extracted study characteristics. Group data for effect-size calculations were requested from study authors. The primary outcome was depression severity change from baseline to the end of the acute treatment phase, expressed as standardised effect size (Hedges' g). Meta-analyses were done using random-effects models.

FINDINGS

From 10 505 publications, 54 trials met the inclusion criteria, of which 29 (20 RCTs and nine open trials) contributed data of a maximum of 6830 participants (age range 18-85 years, male and female individuals and specific ethnicity data unavailable). More than half (4268 [62%] of 6830) of patients with major depressive disorder reported a history of childhood trauma. Despite having more severe depression at baseline (g=0·202, 95% CI 0·145 to 0·258, I2=0%), patients with childhood trauma benefitted from active treatment similarly to patients without childhood trauma history (treatment effect difference between groups g=0·016, -0·094 to 0·125, I2=44·3%), with no significant difference in active treatment effects (vs control condition) between individuals with and without childhood trauma (childhood trauma g=0·605, 0·294 to 0·916, I2=58·0%; no childhood trauma g=0·178, -0·195 to 0·552, I2=67·5%; between-group difference p=0·051), and similar dropout rates (risk ratio 1·063, 0·945 to 1·195, I2=0%). Findings did not significantly differ by childhood trauma type, study design, depression diagnosis, assessment method of childhood trauma, study quality, year, or treatment type or length, but differed by country (North American studies showed larger treatment effects for patients with childhood trauma; false discovery rate corrected p=0·0080). Most studies had a moderate to high risk of bias (21 [72%] of 29), but the sensitivity analysis in low-bias studies yielded similar findings to when all studies were included.

INTERPRETATION

Contrary to previous studies, we found evidence that the symptoms of patients with major depressive disorder and childhood trauma significantly improve after pharmacological and psychotherapeutic treatments, notwithstanding their higher severity of depressive symptoms. Evidence-based psychotherapy and pharmacotherapy should be offered to patients with major depressive disorder regardless of childhood trauma status.

FUNDING

None.

The score after 10 years of registration of systematic review protocols

BACKGROUND

With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status.

METHODS

PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed.

RESULTS

A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%).

CONCLUSIONS

We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders.

SYSTEMATIC REVIEW REGISTRATION

osf.io/9kj7r/.

Randomized controlled trials in patients with COVID-19: a systematic review and critical appraisal

OBJECTIVES

This study aimed to describe the prevalence of risks of bias in randomized trials of therapeutic interventions for COVID-19.

METHODS

Systematic review and risk of bias assessment performed by two independent reviewers of a random sample of 40 randomized trials of therapeutic interventions for moderate-severe COVID-19. We used the RoB 2.0 tool to assess the risk of bias, which evaluates bias under five domains as well as an overall assessment of each trial as high or low risk of bias.

RESULTS

Of the 40 included trials, 19 (47%) were at high risk of bias, and this was particularly frequent in trials from low-middle income countries (11/14, 79%). Potential deviations to intended interventions (i.e., control participants accessing experimental treatments) were considered a potential source of bias in some studies (14, 35%), as was the risk due to selective reporting of results (6, 15%). The randomization process was considered at low risk of bias in most studies (34, 95%), as were missing data (36, 90%) and measurement of the outcome (35, 87%).

CONCLUSION

Many randomized trials evaluating COVID-19 interventions are at risk of bias, particularly those conducted in low-middle income countries. Biases are mostly due to deviations from intended interventions and partly due to the selection of reported results. The use of placebo control and publicly available protocol can mitigate many of these risks.

The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies

Background and Aims

Journal impact factor has historically been taken as a proxy for quality. However, this is open to significant manipulation and bias. There is currently not widely adopted, robust journal and paper ranking metric which is focused solely on risk of bias.

Methods

Risk of bias data was extracted from all Cochrane database systematic reviews in Child Health, Lungs, and Airways for the years 2017–2019. A novel paper quality score, the Clinical Research Bias Index (CRBI), was applied. Individual paper data were pooled for each journal. A comparison was made to journal impact factors, individual paper citations, reads, and altmetric scores.

Results

927 papers were analyzed for risk of bias. 119 (12·8%) scored a CRBI of 100%, with a mean score of 70%. A journal's overall CRBI risk of bias score was poorly correlated with impact factor (r 0.25). Citations (r 0.02), and reads (r 0.01) of individual papers showed very little association with the paper's risk of bias. Likewise, reads were not correlated with citations (r 0.03). H‐index and Altmetric scores were similarly poorly correlated with CRBI.

Conclusion

The novel research quality tool CRBI demonstrates the poor correlation between journal impact factor, citations, and risk of bias. Journal and paper ranking metrics should ensure that they are fit for purpose, and enable the dissemination of high‐quality research for the benefit of patients. We propose the CRBI as a potential solution which is resistant to manipulation and will reward the creation and publication of bias‐free research.

Evaluating how clear the questions being investigated in randomised trials are: systematic review of estimands

OBJECTIVES

To evaluate how often the precise research question being addressed about an intervention (the estimand) is stated or can be determined from reported methods, and to identify what types of questions are being investigated in phase 2-4 randomised trials.

DESIGN

Systematic review of the clarity of research questions being investigated in randomised trials in 2020 in six leading general medical journals.

DATA SOURCE

PubMed search in February 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Phase 2-4 randomised trials, with no restrictions on medical conditions or interventions. Cluster randomised, crossover, non-inferiority, and equivalence trials were excluded.

MAIN OUTCOME MEASURES

Number of trials that stated the precise primary question being addressed about an intervention (ie, the primary estimand), or for which the primary estimand could be determined unambiguously from the reported methods using statistical knowledge. Strategies used to handle post-randomisation events that affect the interpretation or existence of patient outcomes, such as intervention discontinuations or uses of additional drug treatments (known as intercurrent events), and the corresponding types of questions being investigated.

RESULTS

255 eligible randomised trials were identified. No trials clearly stated all the attributes of the estimand. In 117 (46%) of 255 trials, the primary estimand could be determined from the reported methods. Intercurrent events were reported in 242 (95%) of 255 trials; but the handling of these could only be determined in 125 (49%) of 255 trials. Most trials that provided this information considered the occurrence of intercurrent events as irrelevant in the calculation of the treatment effect and assessed the effect of the intervention regardless (96/125, 77%)-that is, they used a treatment policy strategy. Four (4%) of 99 trials with treatment non-adherence owing to adverse events estimated the treatment effect in a hypothetical setting (ie, the effect as if participants continued treatment despite adverse events), and 19 (79%) of 24 trials where some patients died estimated the treatment effect in a hypothetical setting (ie, the effect as if participants did not die).

CONCLUSIONS

The precise research question being investigated in most trials is unclear, mainly because of a lack of clarity on the approach to handling intercurrent events. Clear reporting of estimands is necessary in trial reports so that all stakeholders, including clinicians, patients and policy makers, can make fully informed decisions about medical interventions.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42021238053.

Modelado de tópicos aplicado al análisis del papel del aprendizaje automático en revisiones sistemáticas

El objetivo de la investigación fue analizar el papel del aprendizaje automático de datos en las revisiones sistemáticas de literatura. Se aplicó la técnica de Procesamiento de Lenguaje Natural denominada modelado de tópicos, a un conjunto de títulos y resúmenes recopilados de la base de datos Scopus. Especificamente se utilizó la técnica de Asignación Latente de Dirichlet (LDA), a partir de la cual se lograron descubrir y comprender las temáticas subyacentes en la colección de documentos. Los resultados mostraron la utilidad de la técnica utilizada en la revisión exploratoria de literatura, al permitir agrupar los resultados por temáticas. Igualmente, se pudo identificar las áreas y actividades específicas donde más se ha aplicado el aprendizaje automático, en lo referente a revisiones de literatura. Se concluye que la técnica LDA es una estrategia fácil de utilizar y cuyos resultados permiten abordar una amplia colección de documentos de manera sistemática y coherente, reduciendo notablemente el tiempo de la revisión.

Efficacy of Traditional Chinese Exercise in Improving Gait and Balance in Cases of Parkinson's Disease: A Systematic Review and Meta-analysis

Background

The efficacy of traditional Chinese exercise (TCE)-based intervention in the improvement of motor function in patients with Parkinson's disease (PD) is controversial. The present study aimed to assess the effects of TCE on balance and gait outcomes, as well as motor symptoms in individuals with PD, and evaluate potential discrete moderators such as TCE dosage-related variables.

Method

PubMed, Embase, Cochrane's Library, Web of Science, Medline, and Scopus were systematically searched from their dates of inception to February 2022. All studies were randomized controlled trials (RCTs) of TCE-based interventions for PD. The treatment effects were estimated using a random-effect meta-analysis model with standardized mean differences (Hedges' g). The Physiotherapy Evidence Database was used to evaluate the methodological quality of the study.

Result

Fifteen studies involving a total of 873 participants were included in the final analysis. The meta-analytic findings revealed significant improvements in balance outcomes [Berg Balance Scales (BBS) (g = 0.83, 95% CI = 0.37–1.29, p = 0.000, I² = 84%), time up and go (TUG) (g = −0.80, 95% CI = −1.13– −0.47, p = 0.000, I² = 81%), and the one legged blind balance test (g = 0.49, 95% CI = 0.13–0.86, p = 0.01, I² = 10%)], as well as gait outcomes [gait velocity (g = 0.28, 95% CI = 0.02–0.54, p = 0.04, I² = 64%), 6-min walking test (6MWT) (g = 0.32, 95% CI 0.01–0.62, p = 0.04, I²= 15%), stride length (g = 0.25, 95% CI = 0.08–0.41, p = 0.003, I² = 42%)], and motor symptoms [Unified Parkinson's Disease Rating Scale part III (UPDRS-III) (g = −0.77, 95% CI = −1.06– −0.48, p = 0.000, I² = 76%)]. However, cadence (g = −0.03) and step length (g = 0.02) did not differ significantly. The moderator shows that the effects of TCE on BBS and gait velocity were moderated by Pedro score, exercise type, control group type, and number of sessions. Meta-regression found that TCE (exercise duration, number of sessions, and session duration) was significantly associated with improved UPDRS-III and BBS scores.

Conclusion

These findings provide evidence for the therapeutic benefits of TCE as an adjunct therapy for patients with PD. TEC dosage (high-intensity long sessions) may moderate some favorable effects.

Systematic Review Registration

PROSPERO, identifier CRD42022314686.

Risk of Bias in Iranian Randomized Trials Included in Cochrane Reviews

BACKGROUND

Among interventional studies, randomized controlled trials (RCTs) provide the highest level of evidence. However, RCTs can be susceptible to the risk of bias (RoB). Systematic reviews can be performed to appraise RoB in the included articles using evaluative tools. This study aimed to describe the main characteristics and focus on the RoB of RCTs conducted in Iran and included in Cochrane Reviews (CRs).

METHODS

We searched "Iran" by selecting the "Search All Text" and "Review" fields in the Cochrane Database of Systematic Reviews within Ovid. CRs that included the RCTs conducted in Iran were retrieved. A trial was selected only if it was included in CRs, described as a controlled clinical trial, involved human subjects and CR authors assessed its RoB. The trials were characterized by investigating the relevant articles and the table "Characteristics of included studies" in each CR. The RoB was investigated by collecting the review authors' judgments based on RoB assessment tables in the CRs.

RESULTS

Out of 1166 Iranian RCTs included by 571 CRs, low RoB was found in 44.9% for random sequence generation, 20.8% for allocation concealment, 32.3% for blinding of participants/personnel, 36.5% for blinding of outcome assessors, 56.3% for incomplete outcome data, 41.3% for selective outcome reporting and 53.8% for other sources of bias.

CONCLUSION

The RoB in Iranian RCTs was found to be mostly high or unclear. While this is similar to the global situation, it is recommended that the methodological quality of conducting and reporting RCTs be addressed in Iran.

Effects of Endurance Training on Motor Signs of Parkinson's Disease: A Systematic Review and Meta-Analysis

BACKGROUND

Evidence has demonstrated that endurance training (ET) reduces the motor signs of Parkinson's disease (PD). However, there has not been a comprehensive meta-analysis of studies to date.

OBJECTIVE

The aim of this study was to compare the effect of ET versus nonactive and active control conditions on motor signs as assessed by either the Unified Parkinson's Disease Rating Scale part III (UPDRS-III) or Movement Disorder Society-UPDRS-III (MDS-UPDRS-III).

METHODS

A random-effect meta-analysis model using standardized mean differences (Hedges' g) determined treatment effects. Moderators (e.g., combined endurance and physical therapy training [CEPTT]) and meta-regressors (e.g., number of sessions) were used for sub-analyses. Methodological quality was assessed by the Physiotherapy Evidence Database.

RESULTS

Twenty-seven randomized controlled trials (RCTs) met inclusion criteria (1152 participants). ET is effective in decreasing UPDRS-III scores when compared with nonactive and active control conditions (g =  - 0.68 and g =  - 0.33, respectively). This decrease was greater (within- and between-groups average of - 8.0 and - 6.8 point reduction on UPDRS-III scores, respectively) than the moderate range of clinically important changes to UPDRS-III scores (- 4.5 to - 6.7 points) suggested for PD. Although considerable heterogeneity was observed between RCTs (I² = 74%), some moderators that increased the effect of ET on motor signs decreased the heterogeneity of the analyses, such as CEPTT (I² = 21%), intensity based on treadmill speed (I² = 0%), self-perceived exertion rate (I² = 33%), and studies composed of individuals with PD and freezing of gait (I² = 0%). Meta-regression did not produce significant relationships between ET dosage and UPDRS-III scores.

CONCLUSIONS

ET is effective in decreasing UPDRS-III scores. Questions remain about the dose-response relationship between ET and reduction in motor signs.

Assessment and correlates of autistic symptoms in Schizophrenia Spectrum Disorders measured with the PANSS Autism Severity Score: A systematic review

Schizophrenia Spectrum Disorders (SSD) and Autism Spectrum Disorders (ASD) are considered separate entities, but the two spectra share important similarities, and the study of these areas of overlap represents a field of growing scientific interest. The PANSS Autism Score (PAUSS) was recently developed specifically to assess autistic symptoms in people living with SSD reliably and quickly. The aims of the present systematic review were to provide a comprehensive assessment of the use of the PAUSS scale in available literature and to systematically analyze cognitive, functional and neurobiological correlates of autistic symptoms measured with this instrument in SSD. The systematic literature search included three electronic databases (PubMed, Scopus and PsycINFO) as well as a manual search in Google Scholar and in reference lists of included papers. Screening and extraction were conducted by at least two independent reviewers. Out of 213 identified records, 22 articles referring to 15 original studies were included in the systematic review. Studies were conducted in several different countries by independent groups, showing consistent scientific interest in the use of the scale; most works focused on cognitive and functional correlates of ASD symptoms, but some also considered neurobiological features. Results of included studies showed that autistic symptoms in people with SSD are consistently associated with worse cognitive performance, especially in the social cognition domain, and with worse psychosocial functioning. However, the presence of autistic symptoms appears to also have a protective role, particularly on functioning, in subjects with more severe psychotic symptoms. Further exploring the impact of autistic symptoms could be of significant scientific and clinical interest, allowing the development of tailored interventions to improve treatment for people living with SSDs.