The Unknown Prevalence of Postrandomization Bias in 15 Physical Therapy Journals: A Methods Review

Does manual therapy meaningfully change quantitative sensory testing and patient reported outcome measures in patients with musculoskeletal impairments related to the spine?: A 'trustworthy' systematic review and meta-analysis

OBJECTIVES

To perform a 'trustworthy' systematic review (SR) with meta-analysis on the potential mechanisms of manual therapy used to treat spinal impairments.

DESIGN

SR with meta-analysis.

LITERATURE SEARCH

Articles published between January 2010 and October 2022 from CENTRAL, CINAHL, MEDLINE, PubMed, ProQuest, and PEDro.

METHODS

This SR included English-language randomized clinical trials (RCTs) involving manual therapy to treat spinal impairments in adults. The primary outcome was pressure pain thresholds (PPTs). To synthesize RCTs with high confidence in estimated effects using the GRADE, RCTs with questionable prospective, external, and internal validity, and high risk of bias (RoB) were excluded.

RESULTS

Following title and abstract screening, 89 full-text RCTs were reviewed. Twenty-two studies included the criteria of interest. Sixteen were not prospectively registered, two contained discussion/conclusions judged to be inconsistent with the registry, and one was rated as having a high RoB. Three studies met the inclusion criteria; heterogeneous interventions and locations for PPT testing prevented synthesis into practice recommendations. The two studies with high confidence in estimated effects had small effect sizes, and one study had confidence intervals that crossed zero for the outcome measures of interest.

DISCUSSION

Standardized PPT testing, as a potential measure of centrally mediated pain, could provide clues regarding the mechanisms of manual therapy or help identify/refine research questions.

CONCLUSION

High-quality RCTs could not be synthesized into strong conclusions secondary to the dissimilarity in research designs. Future research regarding quantitative sensory testing should develop RCTs with high confidence in estimated effects that can be translated into strong recommendations.

Is there 'trustworthy' evidence for using manual therapy to treat patients with shoulder dysfunction?: A systematic review

The primary objective of this review was to create a 'trustworthy,' living systematic review and meta-analysis for the application of manual therapy interventions in treating patients with shoulder dysfunction. Included studies were English-language randomized controlled trials published between 1/1/2010 and 8/3/2023, with searches performed in: PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), CINHAL, ProQuest Nursing & Allied Health, EBSCO Medline, and PEDro. The population of focus included adults 18 years and older with musculoskeletal impairments related to shoulder dysfunction. Our primary outcomes included pain and region-specific outcome measures. We excluded trials, including participants having shoulder dysfunction resulting from surgery, radicular pain, instability/dislocation, fracture, lymphedema, and radiation. Our screening methodology was based upon a previously published 'trustworthy' systematic review protocol. This included the application of our PICOTS criteria in addition to screening for prospective clinical trial registration and following of prospective intent, as well as assessment of PEDro scores, risk-of-bias ratings, GRADE scoring, and examination of confidence in estimated effects. Twenty-six randomized controlled trials met our PICOTS criteria; however, only 15 of these were registered. Only three were registered prospectively. Two of these did not have discussions and conclusions that aligned with their primary outcome. The remaining single study was found to have a high risk-of-bias, meaning the remainder of the protocol could not be employed and that no randomized controlled trials could undergo further assessment or meta-analysis. The results of this systematic review indicate there are no 'trustworthy' randomized controlled trials examining the effectiveness of manual therapy interventions for the treatment of patients with shoulder dysfunction, as defined by the prospectively established methodology. Therefore, these findings signal that creating a 'trustworthy,' living systematic review on this clinically relevant topic is not yet possible due to a lack of 'trustworthy' randomized controlled trials.

Evaluation of Transparency and Openness Guidelines in Physical Therapist Journals

OBJECTIVE

The goals of this study were to evaluate the extent that physical therapist journals support open science research practices by adhering to the Transparency and Openness Promotion (TOP) guidelines and to assess the relationship between journal scores and their respective journal impact factor (JIF).

METHODS

Scimago, mapping studies, the National Library of Medicine, and journal author guidelines were searched to identify physical therapist journals for inclusion. Journals were graded on 10 standards (29 available total points) related to transparency with data, code, research materials, study design and analysis, preregistration of studies and statistical analyses, replication, and open science badges. The relationship between journal transparency and openness scores and their JIF was determined.

RESULTS

Thirty-five journals' author guidelines were assigned transparency and openness factor scores. The median score (interquartile range) across journals was 3.00 out of 29 (3.00) points (for all journals the scores ranged from 0 to 8). The 2 standards with the highest degree of implementation were design and analysis transparency (reporting guidelines) and study preregistration. No journals reported on code transparency, materials transparency, replication, and open science badges. TOP factor scores were a significant predictor of JIF scores.

CONCLUSION

There is low implementation of the TOP standards by physical therapist journals. TOP factor scores demonstrated predictive abilities for JIF scores. Policies from journals must improve to make open science practices the standard in research. Journals are in an influential position to guide practices that can improve the rigor of publication which, ultimately, enhances the evidence-based information used by physical therapists.

IMPACT

Transparent, open, and reproducible research will move the profession forward by improving the quality of research and increasing the confidence in results for implementation in clinical care.

Clinical trial registration has become more prevalent in physical therapy but it is still inadequate: A meta-research study

BACKGROUND

A study using data from 2009 showed low prevalence and inadequate trial registration in physiotherapy. In 2013, a joint editorial recommended prospective registration in physiotherapy journals. Ten years later it is unclear whether the joint editorial achieved its intended benefit.

OBJECTIVES

To investigate the proportion of randomized trials adequately registered and the extent of selective reporting of outcomes in trials of physiotherapy interventions published in 2019 and to compare these data with equivalent published data from 2009.

DESIGN

Meta-research study.

METHOD

A random sample of 200 trials published in 2019 was used. Evidence of registration was sought on trial registers and by contacting authors. Data from the article was compared with data from the trial registration. Data from this sample of trial published in 2019 were compared with equivalent published data from 2009.

RESULTS

In 2019, the proportion of trials that were registered was 63% versus 34% in 2009 (absolute difference 29%). In 2019, 18% of the trials were prospectively registered compared to 6% in 2009 (absolute difference 12%). Unambiguous primary outcomes (i.e., method and timepoints of measurement clearly defined in the trial registry entry) were registered for 30% in 2019. Registration was adequate (i.e., prospective with unambiguous primary outcomes) for 8%, compared with 3% in 2009 (absolute difference 5%). Selective outcome reporting occurred in 73% of the trials in which it was assessable; in 2009 this proportion was 47% (absolute difference 26%).

CONCLUSIONS

Registration of randomized trials in physiotherapy increased in the past decade, but it is still inadequate. More effort is still required to implement and enforce adequate registration.

Protocol for the development of a 'trustworthy' living systematic review and meta analyses of manual therapy interventions to treat neuromusculoskeletal impairments

BACKGROUND

Preprocessed research resources are believed to be highly 'trustworthy' when translating research to clinical practice. However, the overall 'trustworthiness' is unknown if this evidence contains randomized clinical trials (RCTs) where prospective has not been/cannot be verified, has low confidence in estimated effects, and if they are not up to date.

OBJECTIVES

This protocol will be used to create a baseline benchmark for a series of trustworthy living systematic reviews (SRs) regarding manual therapy interventions.

METHODS

Data will originate from RCTs related to manual therapy neuromusculoskeletal interventions, indexed in 6 search engines in English from 1 January 2010, to the present. Two blinded reviewers will identify the RCTs and extract data using Covidence. The data will be synthesized based on consensus and analyzed using the Cochrane collaboration's Review Manager.

EXPECTED OUTCOMES

It is expected that there will be a shortage of RCTs with at least a moderate confidence in estimated effects that will allow for strong practice recommendations.

DISCUSSION

Identifying evidence that can be translated into strong practice recommendations is essential to identify beneficial and harmful interventions, decrease practice variability, and identify neuromusculoskeletal manual therapy interventions that require further disciplined methodological focus.

Manual therapy for non-radicular cervical spine related impairments: establishing a 'Trustworthy' living systematic review and meta-analysis

OBJECTIVES

To establish a 'trustworthy' living systematic review (SR) with a meta-analysis of manual therapy for treating non-radicular cervical impairments.

DESIGN

SR with meta-analysis.

LITERATURE SEARCH

Articles published between January 2010 and September 2022 were included from: Cochrane Central Register of Controlled Trials (CENTRAL); CINAHL; MEDLINE; PubMed; PEDro, and ProQuest Nursing and Allied Health.

METHODS

This SR included English-language randomized clinical trials (RCTs) of manual therapy involving adults used to treat non-radicular cervical impairments. The primary outcomes were pain and region-specific outcome measures. Cervicogenic headaches and whiplash were excluded to improve homogeneity. Two reviewers independently assessed RCTs. The prospective plan was to synthesize results with high confidence in estimated effects using GRADE.

RESULTS

Thirty-five RCTs were screened for registration status. Twenty-eight were not registered or registered prospectively. In 5 studies, the discussion and conclusion did not match the registry, or this could not be determined. One study did not meet the external validity criterion, and another was rated as having a high risk of bias. One study met the inclusion and exclusion criteria, so practice recommendations could not be made. The remaining study did not identify any clinically meaningful group differences.

DISCUSSION

Only one prospectively registered RCT met this SR's strict, high-quality standards. The single identified paper provides initial high-quality evidence on this topic.

CONCLUSION

This SR establishes a foundation of trustworthiness and can be used to generate research agendas to determine the potential clinical utility of manual therapy directed at the cervical spine for non-radicular cervical complaints.

Is the quality of systematic reviews influenced by prospective registration: a methods review of systematic musculoskeletal physical therapy reviews

INTRODUCTION

It is unknown if verified prospective registration of systematic reviews (SRs) and the randomized clinical trials (RCTs) that they use affect an SR's methodological quality on A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2).

METHODS

Data originated from interventional SRs published in International Society of Physiotherapy Journals Editors (ISPJE) member journals, indexed in MEDLINE, between 1 January 2018 and 18 August 2021. Blinded reviewers identified the SRs and extracted the data for the variables of interest for the SRs and the RCTs.

RESULTS

Two of 14 ISPJE member journals required prospective SR registration. Twenty SRs were identified, and 169 unique, retrievable RCTs were included within those SRs. One (5.0%) of the 20 SRs and 15 of the 169 (8.9%) RCTs were prospectively registered and published consistent with this intent. Nineteen (95.0%) of the 20 identified SRs was categorized as 'critically low' on the AMSTAR 2.

DISCUSSION

SRs and the RCTs identified within them were infrequently prospectively registered, prospectively verifiable, or prospectively verified based on the established research record.

CONCLUSIONS

Ensuring that SRs and RCTs have fidelity with the research record from conception to publication may help rule out low-value interventions, decrease variability in physical therapy practice, and solidify evidence-based physical therapy practice.

"Trustworthiness," confidence in estimated effects, and confidently translating research into clinical practice

Trustworthy, preprocessed sources of evidence, such as systematic reviews and clinical practice guidelines, are crucial for practicing clinicians. Confidence in estimated effects is related to how different the outcome data were between the two groups. Factors including the effect size, variability of the effect, research integrity, research methods, and selected outcome measures impact confidence in the estimated effect. The current evidence suggests that post-randomization biases cannot be ruled out with a high degree of certainty in published research, limiting the utility of preprocessed sources for clinicians. Research should be prospectively registered to improve this situation, and fidelity with prospective intent should be verified to minimize biases and strengthen confidence in estimated effects. Otherwise, discussions related to preprocessed literature, including P-values, point estimates of effect, confidence intervals, post-randomization biases, external and internal validity measures, and the confidence in estimated effects required to translate research into practice confidently, are all moot points.