Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews

Enhancing Orthodontic Renewal and Retention Techniques: A Systematic Review

Orthodontists have a variety of options available for retainers. Research in Orthodontics focuses on assessing outcomes important to clinicians; however, there is inconsistency in how these outcomes are selected and evaluated. This review sought to assess the effects of different orthodontic retainers on patients' quality of life (QoL). Various approaches were employed in this systematic review, and a thorough search was conducted across six databases. The review involved a comprehensive evaluation of six included studies, highlighting changes in dental structure post-treatment, emphasizing the role of extraction procedures and the quality of debonding in improving retention. The study identified key outcomes for orthodontic clinical trials, highlighting orthodontists' preferences for specific retainer types. Moreover, it discussed the impact of sociocultural influences on retention care. Involving patients actively in discussions about whether to end or extend the retention phase was deemed essential. Noteworthy improvements in occlusal outcomes were linked to extraction treatments. Gender and malocclusion severity influenced QoL before and after orthodontic treatment. The degree of improvement observed in the Class III malocclusion group was comparatively lower than that in the Class I and Class II groups. Orthodontic treatment was found to yield favorable psychological outcomes, as evidenced by notable enhancements in self-esteem and social engagement among individuals. Fixed appliances were shown to negatively affect oral health-related quality of life (OHRQoL), particularly for those with aesthetic and functional concerns. A consensus has been reached on the essential themes and outcomes that should be incorporated in clinical trials related to orthodontic retention for non-cleft and non-surgical cases.

Trials and tribulations of transparency related to inconsistencies between plan and conduct in peer-reviewed physiotherapy publications: A methodology review

RATIONALE

The physiotherapy profession strives to be a leader in providing quality care and strongly recognizes the value of research to guide clinical practice. Adherence to guidelines for research reporting and conduct is a significant step towards high-quality, transparent and reproducible research.

AIM/OBJECTIVE

Assess integrity between planned and conducted methodology in randomized controlled trials (RCTs) and systematic reviews (SRs) published in physiotherapy journals.

METHODS

Eighteen journals were manually searched for RCTs and SRs published from 1 July 2021 through 31 December 2021. Studies were included if the journal or specific study was indexed in PubMed and published/translated in English. Descriptive statistics determined congruence between preregistration data and publication.

RESULTS

Forty RCTs and 68 SRs were assessed. Forty-three SRs included meta-analysis (MA). Of the 34 registered RCTs, 7 (20.6%) had no discrepancy between the registration and publication. Two trials (5.9%) addressed all discrepancies, 4 (11.8%) addressed some and 21 (61.8%) did not address any discrepancies. Of the 36 registered MAs, 33 (91.7%) had discrepancies between the registration and publication. Two (5.6%) addressed all discrepancies and three (8.3%) had no discrepancies. Eight SRs without MA published information not matching their registration, and none provided justification for the discrepancies.

CONCLUSION

Most RCTs/SRs were registered; the majority had discrepancies between preregistration and publication, potentially influencing the outcomes and interpretations of findings. Journals should require preregistration and compare the submission with the registration information when assessing publication suitability. Readers should be aware of these inconsistencies and their implications when interpreting and translating results into practice.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Assessing the magnitude of changes from protocol to publication-a survey on Cochrane and non-Cochrane Systematic Reviews

Objective

To explore differences between published reviews and their respective protocols in a sample of 97 non-Cochrane Systematic Reviews (non-CSRs) and 97 Cochrane Systematic Reviews (CSRs) in terms of PICOS (Patients/Population, Intervention, Comparison/Control, Outcome, Study type) elements and the extent to which they were reported.

Study Design and Setting

We searched PubMed and Cochrane databases to identify non-CSRs and CSRs that were published in 2018. We then searched for their corresponding Cochrane or PROSPERO protocols. The published reviews were compared to their protocols. The primary outcome was changes from protocol to review in terms of PICOS elements.

Results

We identified a total of 227 changes from protocol to review in PICOS elements, 1.11 (Standard Deviation (SD), 1.22) changes per review for CSRs and 1.23 (SD, 1.12) for non-CSRs per review. More than half of each sub-sample (54.6% of CSRs and 67.0% of non-CSRs) (Absolute Risk Reduction (ARR) 12.4% [-1.3%; 26.0%]) had changes in PICOS elements. For both subsamples, approximately a third of all changes corresponded to changes related to primary outcomes. Marked differences were found between the sub-samples for the reporting of changes. 95.8% of the changes in PICOS items were not reported in the non-CSRs compared to 42.6% in the CSRs (ARR 53.2% [43.2%; 63.2%]).

Conclusion

CSRs showed better results than non-CSRs in terms of the reporting of changes. Reporting of changes from protocol needs to be promoted and requires general improvement. The limitations of this study lie in its observational design. Registration: https://osf.io/6j8gd/.

Factors associated with successful publication for systematic review protocol registration: an analysis of 397 registered protocols

BACKGROUND

Meta-analyses are on top of the evidence-based medicine pyramid, yet many of them are not completed after they are begun. Many factors impacting the publication of meta-analysis works have been discussed, and their association with publication likelihood has been investigated. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication. In our current review, we aim to investigate these various factors and their impact on the likelihood of publication. A comprehensive review of 397 registered protocols retrieved from five databases was performed to investigate the different factors that might affect the likelihood of publication. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication.

RESULTS

We found that corresponding authors in developed countries and English-speaking countries had higher likelihoods of publication: 206/320 (p = 0.018) and 158/236 (p = 0.006), respectively. Factors affecting publications are the countries of corresponding author (p = 0.033), whether they are from developed countries (OR: 1.9, 95% CI: 1.2-3.1, p = 0.016), from English-speaking countries (OR: 1.8, 95% CI: 1.2-2.7, p = 0.005), update status of the protocol (OR: 1.6, 95% CI: 1.0-2.6, p = 0.033), and external funding (OR: 1.7, 95% CI: 1.1-2.7, p = 0.025). Multivariable regression retains three variables as significant predictors for the publication of a systematic review: whether it is the corresponding author from developed countries (p = 0.013), update status of the protocol (p = 0.014), and external funding (p = 0.047).

CONCLUSION

Being on top of the evidence hierarchy, systematic review and meta-analysis are the keys to informed clinical decision-making. Updating protocol status and external funding are significant influences on their publications. More attentions should be paid to the methodological quality of this type of publication.

Interventions to facilitate the successful eruption of impacted maxillary incisor teeth due to the presence of a supernumerary: A systematic review and meta-analysis

INTRODUCTION

A failure of maxillary incisor eruption is commonly attributed to the presence of a supernumerary tooth. This systematic review aimed to assess the percentage of impacted maxillary incisors that successfully erupt after surgical removal of supernumerary teeth with or without other interventions.

METHODS

Systematic literature searches without restrictions were undertaken in 8 databases for studies reporting any intervention aimed at facilitating incisor eruption, including surgical removal of the supernumerary alone or in conjunction with additional interventions published up to September 2022. After duplicate study selection, data extraction, and risk of bias assessment according to the risk of bias in nonrandomized studies of interventions and Newcastle-Ottawa scale, random-effects meta-analyses of aggregate data were conducted.

RESULTS

Fifteen studies (14 retrospective and 1 prospective) were included with 1058 participants (68.9% male; mean age, 9.1 years). The pooled eruption prevalence for removal of the supernumerary tooth with space creation or removal of the supernumerary tooth with orthodontic traction was significantly higher at 82.4% (95% confidence interval [CI], 65.5-93.2) and 96.9% (95% CI, 83.8-99.9) respectively, compared with removal of an associated supernumerary only (57.6%; 95% CI, 47.8-67.0). The odds of successful eruption of an impacted maxillary incisor after removal of a supernumerary were more favorable if the obstruction was removed in the deciduous dentition (odds ratio [OR], 0.42; 95% CI, 0.20-0.90; P = 0.02); if the supernumeraries were conical (OR, 2.91; 95% CI, 1.98-4.28; P <0.001); if the incisor was in the correct position (OR, 2.19; 95% CI, 1.14-4.20; P = 0.02), at the level of the gingival third (OR 0.07; 95% CI, <0.01-0.97; P = 0.04) and had incomplete root formation (OR, 9.02; 95% CI, 2.04-39.78; P = 0.004). Delaying removal of the supernumerary tooth 12 months after the expected eruption time of the maxillary incisor (OR, 0.33; 95% CI, 0.10-1.03; P = 0.05) and waiting >6 months for spontaneous eruption after removal of the obstacle (OR, 0.13; 95% CI, 0.03-0.50; P = 0.003) was associated with worse odds for eruption.

CONCLUSIONS

Limited evidence indicated that the adjunctive use of orthodontic measures and removal of supernumerary teeth might be associated with greater odds of successfull impacted incisor eruption than removal of the supernumerary tooth alone. Certain characteristics related to supernumerary type and the position or developmental stage of the incisor may also influence successful eruption after removal of the supernumerary. However, these findings should be viewed with caution as our certainty is very low to low because of bias and heterogeneity. Further well-conducted and reported studies are required. The results of this systematic review have been used to inform and justify the iMAC Trial.

Selective outcome reporting in root coverage randomized clinical trials

BACKGROUND

Outcome discrepancies between protocols and respective publications represent a concerning bias. The purpose of this study was to assess the prevalence of selective outcome reporting (SOR) in root coverage randomized clinical trials (RCTs).

METHODS

Published root coverage RCTs (July 2005 to March 2020) were included if a corresponding protocol could be identified in a public registry. Discrepancies between protocol and its correspondent publication(s) were compared regarding primary and secondary outcomes and other study characteristics. Associations between trial characteristics and SOR were evaluated.

RESULTS

Forty four studies (54 publications) were included. The majority of studies (77.3%) were retrospectively registered. SOR was frequent (40.9% of trials) and consisted of primary outcome downgrade (22.7%); secondary outcome upgrade (11.4%); new primary outcome introduced in publication (25%); protocol primary outcome omitted from publication (13.6%) and discrepancy in primary outcome timing (18.2%). SOR was unclear in 20.5% of studies and favoured statistical significance in 12 studies (27.3%). SOR was significantly associated with study significance (p < 0.001) and unclear outcome definition in the publication (p < 0.001). Only a third (32.8%) of primary outcomes were completely defined.

CONCLUSIONS

The present study identified high prevalence of SOR in root coverage RCTs.

Development of a core outcome set for use in routine orthodontic clinical trials

INTRODUCTION

A diverse range of outcomes is used in orthodontic research with a focus on measuring outcomes important to clinicians and little consistency in outcome selection and measurement. We aimed to develop a core outcome set for use in clinical trials of orthodontic treatment not involving cleft or orthognathic patient groups.

METHODS

A list of outcomes measured in previous orthodontic research was identified through a scoping literature review. Additional outcomes of importance to patients were obtained using qualitative interviews and focus groups with adolescents aged 10-16 years. Rating of outcomes was carried out in a 2-round electronic Delphi process involving health care professionals and patients using a 9-point scale. A face-to-face meeting was subsequently held with stakeholders to discuss the results before refining the core outcome set.

RESULTS

After triangulation, a final list of 34 outcomes grouped under 10 domains was obtained for rating in the e-Delphi surveys. Fifteen outcomes were voted "in" after the second Delphi round involving 274 participants with a further outcome being included after the consensus meeting. These were subsequently refined into a final set of 7 core outcomes, including the impact of self-perceived esthetics, alignment and/or occlusion, skeletal relationship, stability, patient-related adherence, breakages, and adverse effects on teeth or teeth-supporting structures.

CONCLUSIONS

A bespoke orthodontic core outcome set encompassing both clinician- and patient-focused outcomes was developed. Incorporating this is the first step into providing a more holistic assessment of the impact of treatment while allowing for meaningful comparisons and synthesis of results from individual trials.

Selective outcome reporting in randomized clinical trials of dental implants

AIM

Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants.

MATERIALS AND METHODS

Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols.

RESULTS

A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04).

CONCLUSIONS

Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.

Evolution of international collaborative research efforts to develop non-Cochrane systematic reviews

This research-on-research study describes efforts to develop non-Cochrane systematic reviews (SRs) by analyzing demographical and time-course collaborations between international institutions using protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO) or published in scientific journals. We have published an a priori protocol to develop this study. Protocols published in scientific journals were searched using the MEDLINE and Embase databases; the query terms "Systematic review" [Title] AND "protocol" [Title] were searched from February 2011 to December 2017. Protocols registered at PROSPERO during the same period were obtained by web scraping all non-Cochrane records with a Python script. After excluding protocols that had a fulfillment or duplication rate of less than 90%, they were classified as published "only in PROSPERO", "only in journals", or in "journals and PROSPERO". Results of data and metadata extraction using text mining processes were curated by two reviewers. These Datasets and R scripts are freely available to facilitate reproducibility. We obtained 20,814 protocols of non-Cochrane SRs. While "unique protocols" by reviewers' institutions from 60 countries were the most frequent, a median of 6 (2-150) institutions from 130 different countries were involved in the preparation of "collaborative protocols". The highest Ranked countries involved in overall protocol production were the UK, the U.S., Australia, Brazil, China, Canada, the Netherlands, Germany, and Italy. Most protocols were registered only in PROSPERO. However, the number of protocols published in scientific journals (924) or in both PROSPERO and journals (807) has increased over the last three years. Syst Rev and BMJ Open published more than half of the total protocols. While the more productive countries were involved in "unique" and "collaborative protocols", less productive countries only participated in "collaborative protocols" that were mainly published in PROSPERO. Our results suggest that, although most countries were involved in solitary production of protocols for non-Cochrane SRs during the study period, it would be useful to develop new strategies to promote international collaborations, especially with less productive countries.

Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality

OBJECTIVES

A priori registration of systematic review protocols in international prospective register of systematic reviews (PROSPERO) can help reduce selective reporting of outcomes. The aim of this study was to assess the association between registration of orthodontic systematic reviews in PROSPERO and review quality with the Assessment of Multiple Systematic Reviews (AMSTAR) tool.

STUDY DESIGN AND SETTING

Seven databases were searched for systematic reviews with/without meta-analysis in orthodontics published between 2012 and 2016. After duplicate study selection and data extraction, the quality of identified reviews was assessed in duplicate with the AMSTAR tool. Descriptive statistics of medians and interquartile ranges (IQRs) and chi-square/Fisher exact tests were calculated. Univariable/multivariable linear regression modeling was implemented to assess the effect of review registration on %AMSTAR score at α of 5%.

RESULTS

A total of 182 orthodontic systematic reviews were identified, 37 (20.3%) of which were registered. Considerable differences were seen in AMSTAR between registered (median = 86.4%; IQR = 77.3-95.5%) and nonregistered reviews (median = 72.7%; IQR = 59.1-81.8%). After adjustment, registration in PROSPERO was associated with an average increase in %AMSTAR score of 6.6% (95% confidence interval = 1.0-12.3%).

CONCLUSION

Although only a small percentage of orthodontic systematic reviews was registered a priori in PROSPERO, registered reviews were of higher quality than nonregistered reviews.

A low proportion of systematic reviews in physical therapy are registered: a survey of 150 published systematic reviews

BACKGROUND

Systematic reviews provide the best evidence about the effectiveness of healthcare interventions. Although systematic reviews are conducted with explicit and transparent methods, discrepancies might occur between the protocol and the publication.

OBJECTIVES

To estimate the proportion of systematic reviews of physical therapy interventions that are registered, the methodological quality of (un)registered systematic reviews and the prevalence of outcome reporting bias in registered systematic reviews.

METHODS

A random sample of 150 systematic reviews published in 2015 indexed on the PEDro database. We included systematic reviews written in English, Italian, Portuguese and Spanish. A checklist for assessing the methodological quality of systematic reviews tool was used. Relative risk was calculated to explore the association between meta-analysis results and the changes in the outcomes.

RESULTS

Twenty-nine (19%) systematic reviews were registered. Funding and publication in a journal with an impact factor higher than 5.0 were associated with registration. Registered systematic reviews demonstrated significantly higher methodological quality (median=8) than unregistered systematic reviews (median=5). Nine (31%) registered systematic reviews demonstrated discrepancies between protocol and publication with no evidence that such discrepancies were applied to favor the statistical significance of the intervention (RR=1.16; 95% CI: 0.63-2.12).

CONCLUSION

A low proportion of systematic reviews in the physical therapy field are registered. The registered systematic reviews showed high methodological quality without evidence of outcome reporting bias. Further strategies should be implemented to encourage registration.

Majority of systematic reviews published in high-impact journals neglected to register the protocols: a meta-epidemiological study

OBJECTIVES

To describe the registration of systematic review (SR) protocols and examine whether or not registration reduced the outcome reporting bias in high-impact journals.

STUDY DESIGN AND SETTING

We searched MEDLINE via PubMed to identify SRs of randomized controlled trials of interventions. We included SRs published between August 2009 and June 2015 in the 10 general and internal medicinal journals with the highest impact factors in 2013. We examined the proportion of SR protocol registration and investigated the relationship between registration and outcome reporting bias using multivariable logistic regression.

RESULTS

Among the 284 included reviews, 60 (21%) protocols were registered. The proportion of registration increased from 5.6% in 2009 to 27% in 2015 (P for trend <0.001). Protocol registration was not associated with outcome reporting bias (adjusted odds ratio [OR] 0.85, 95% confidence interval [CI] 0.39-1.86). The association between Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) adherence and protocol registration was not statistically significant (OR 1.09, 95% CI 0.59-2.01).

CONCLUSIONS

Six years after the launch of the PRISMA statement, the proportion of protocol registration in high-impact journals has increased some but remains low. The present study found no evidence suggesting that protocol registration reduced outcome reporting bias.

Evaluating benefits and harm of therapies

OBJECTIVE

This third article in a series of 4 aims to inform dental practitioners on the importance of weighing benefits and harm when making clinical treatment decisions.

METHODS

This article describes some examples of important consequences to patients' health when potential side effects of interventions are not taken into consideration in the decision-making process. Some information to help dental practitioners search for data on negative effects of therapies is also presented, and the need to weigh benefits and harm of therapies is discussed.

RESULTS

Therapeutic interventions in dentistry may cause several side effects, such as nerve damage caused by dental implant placement. Adequate evaluation of potential side effects before performing a procedure and comprehensive communication with the patient about these side effects are sine qua non conditions for any clinical decision.

CONCLUSIONS

The information provided in this article may encourage dental practitioners to obtain more information regarding negative aspects of clinical interventions, to more comprehensively inform patients and to improve clinical decision-making.