A methodological review revealed that reporting of trials in manual therapy has not improved over time

Waste not, want not: call to action for spinal manipulative therapy researchers

BACKGROUND

Research waste is defined as research outcomes with no or minimal societal benefits. It is a widespread problem in the healthcare field. Four primary sources of research waste have been defined: (1) irrelevant or low priority research questions, (2) poor design or methodology, (3) lack of publication, and (4) biased or inadequate reporting. This commentary, which was developed by a multidisciplinary group of researchers with spinal manipulative therapy (SMT) research expertise, discusses waste in SMT research and provides suggestions to improve future research.

MAIN TEXT

This commentary examines common sources of waste in SMT research, focusing on design and methodological issues, by drawing on prior research and examples from clinical and mechanistic SMT studies. Clinical research is dominated by small studies and studies with a high risk of bias. This problem is compounded by systematic reviews that pool heterogenous data from varying populations, settings, and application of SMT. Research focusing on the mechanisms of SMT often fails to address the clinical relevance of mechanisms, relies on very short follow-up periods, and has inadequate control for contextual factors.

CONCLUSIONS

This call to action is directed to researchers in the field of SMT. It is critical that the SMT research community act to improve the way research is designed, conducted, and disseminated. We present specific key action points and resources, which should enhance the quality and usefulness of future SMT research.

Automatic categorization of self-acknowledged limitations in randomized controlled trial publications

OBJECTIVE

Acknowledging study limitations in a scientific publication is a crucial element in scientific transparency and progress. However, limitation reporting is often inadequate. Natural language processing (NLP) methods could support automated reporting checks, improving research transparency. In this study, our objective was to develop a dataset and NLP methods to detect and categorize self-acknowledged limitations (e.g., sample size, blinding) reported in randomized controlled trial (RCT) publications.

METHODS

We created a data model of limitation types in RCT studies and annotated a corpus of 200 full-text RCT publications using this data model. We fine-tuned BERT-based sentence classification models to recognize the limitation sentences and their types. To address the small size of the annotated corpus, we experimented with data augmentation approaches, including Easy Data Augmentation (EDA) and Prompt-Based Data Augmentation (PromDA). We applied the best-performing model to a set of about 12K RCT publications to characterize self-acknowledged limitations at larger scale.

RESULTS

Our data model consists of 15 categories and 24 sub-categories (e.g., Population and its sub-category DiagnosticCriteria). We annotated 1090 instances of limitation types in 952 sentences (4.8 limitation sentences and 5.5 limitation types per article). A fine-tuned PubMedBERT model for limitation sentence classification improved upon our earlier model by about 1.5 absolute percentage points in F1 score (0.821 vs. 0.8) with statistical significance (p<.001). Our best-performing limitation type classification model, PubMedBERT fine-tuning with PromDA (Output View), achieved an F1 score of 0.7, improving upon the vanilla PubMedBERT model by 2.7 percentage points, with statistical significance (p<.001).

CONCLUSION

The model could support automated screening tools which can be used by journals to draw the authors' attention to reporting issues. Automatic extraction of limitations from RCT publications could benefit peer review and evidence synthesis, and support advanced methods to search and aggregate the evidence from the clinical trial literature.

The Development, Validity, and Responsiveness of a Patient-Centred Outcome Measurement Tool for Evaluating Integrative Medicine Interventions

Background

The paper sets out the development, validity, and responsiveness of the Integrative Medicine Treatment Evaluation Form (IMTEF), which has been designed to measure the effects of complementary and integrative therapy (CIT) interventions in cancer and palliative care (PC) patients in a National Health Service (NHS) hospital setting. Treatment evaluation is essential for ensuring safety and quality of services, for meeting NHS governance requirements. It also helps to add to the evidence base for complementary and integrative therapies through collecting data about treatments.

Methods

A number of different Patient Reported Outcome Measures (PROMs) tools were reviewed in order to design the IMTEF, which details questions that captures both quantitative and qualitative data. The IMTEF was reviewed by patients and a range of health care practitioners.

Results

IMTEF's validity is supported by feedback from health care practitioners and patients, by its ability to detect different degrees of change in relation to change scores, and by its correlations with Visual Analog Scale (VAS) scores.

Conclusion

The IMTEF can be used to assess the effects of therapeutic bodywork and CITs when many of the patients do not have the capacity or the time to answer many questions, and when therapists do not know in advance the number of treatments that patients will be able to receive. Because of the way it is structured, it can also assess the effects after a number of sessions.

"Do Not Mix Apples with Oranges" to Avoid Misinterpretation of Placebo Effects in Manual Therapy: The Risk Is Resulting in a Fruit Basket. Comment on Molina-Àlvarez et al. Manual Therapy Effect in Placebo-Controlled Trials: A Systematic Review and Meta-Analysis. Int. J. Environ. Res. Public Health 2022, 19, 14021

We read with interest the systematic review with the meta-analysis by Miguel Molina-Álvarez et al. [...].

Blinding and sham control methods in trials of physical, psychological, and self-management interventions for pain (article I): a systematic review and description of methods

ABSTRACT

Blinding is challenging in randomised controlled trials of physical, psychological, and self-management therapies for pain, mainly because of their complex and participatory nature. To develop standards for the design, implementation, and reporting of control interventions in efficacy and mechanistic trials, a systematic overview of currently used sham interventions and other blinding methods was required. Twelve databases were searched for placebo or sham-controlled randomised clinical trials of physical, psychological, and self-management treatments in a clinical pain population. Screening and data extraction were performed in duplicate, and trial features, description of control methods, and their similarity to the active intervention under investigation were extracted (protocol registration ID: CRD42020206590). The review included 198 unique control interventions, published between 2008 and December 2021. Most trials studied people with chronic pain, and more than half were manual therapy trials. The described control interventions ranged from clearly modelled based on the active treatment to largely dissimilar control interventions. Similarity between control and active interventions was more frequent for certain aspects (eg, duration and frequency of treatments) than others (eg, physical treatment procedures and patient sensory experiences). We also provide an overview of additional, potentially useful methods to enhance blinding, as well as the reporting of processes involved in developing control interventions. A comprehensive picture of prevalent blinding methods is provided, including a detailed assessment of the resemblance between active and control interventions. These findings can inform future developments of control interventions in efficacy and mechanistic trials and best-practice recommendations.

Differences between experimental and placebo arms in manual therapy trials: a methodological review

BACKGROUND

To measure the specific effectiveness of a given treatment in a randomised controlled trial, the intervention and control groups have to be similar in all factors not distinctive to the experimental treatment. The similarity of these non-specific factors can be defined as an equality assumption. The purpose of this review was to evaluate the equality assumptions in manual therapy trials.

METHODS

Relevant studies were identified through the following databases: EMBASE, MEDLINE, SCOPUS, WEB OF SCIENCE, Scholar Google, clinicaltrial.gov, the Cochrane Library, chiloras/MANTIS, PubMed Europe, Allied and Complementary Medicine (AMED), Physiotherapy Evidence Database (PEDro) and Sciencedirect. Studies investigating the effect of any manual intervention compared to at least one type of manual control were included. Data extraction and qualitative assessment were carried out independently by four reviewers, and the summary of results was reported following the PRISMA statement.

RESULT

Out of 108,903 retrieved studies, 311, enrolling a total of 17,308 patients, were included and divided into eight manual therapy trials categories. Equality assumption elements were grouped in three macro areas: patient-related, context-related and practitioner-related items. Results showed good quality in the reporting of context-related equality assumption items, potentially because largely included in pre-existent guidelines. There was a general lack of attention to the patient- and practitioner-related equality assumption items.

CONCLUSION

Our results showed that the similarity between experimental and sham interventions is limited, affecting, therefore, the strength of the evidence. Based on the results, methodological aspects for planning future trials were discussed and recommendations to control for equality assumption were provided.

Do manual therapies have a specific autonomic effect? An overview of systematic reviews

Background

The impact of manual therapy interventions on the autonomic nervous system have been largely assessed, but with heterogeneous findings regarding the direction of these effects. We conducted an overview of systematic reviews to describe if there is a specific autonomic effect elicited by manual therapy interventions, its relation with the type of technique used and the body region where the intervention was applied.

Methods

We conducted an overview according to a publicly registered protocol. We searched the Cochrane Database of Systematic Reviews, MEDLINE, EPISTEMONIKOS and SCOPUS, from their inception to march 2021. We included systematic reviews for which the primary aim of the intervention was to assess the autonomic effect elicited by a manual therapy intervention in either healthy or symptomatic individuals. Two authors independently applied the selection criteria, assessed risk of bias from the included reviews and extracted data. An established model of generalisation guided the data analysis and interpretation.

Results

We included 12 reviews (5 rated as low risk of bias according the ROBIS tool). The findings showed that manual therapies may have an effect on both sympathetic and parasympathetic systems. However, the results from included reviews were inconsistent due to differences in their methodological rigour and how the effects were measured. The reviews with a lower risk of bias could not discriminate the effects depending on the body region to which the technique was applied.

Conclusion

The magnitude of the specific autonomic effect elicited by manual therapies and its clinical relevance is uncertain. We point out some specific recommendations in order to improve the quality and relevance of future research in this field.

Evidence-based practice among Italian osteopaths: a national cross-sectional survey

BACKGROUND

While evidence-based practice (EBP) is widely accepted across healthcare professions, research investigating its implementation in manual therapy professions such as osteopathy is limited. The primary aim of this study was to investigate Italian osteopaths' attitudes, skills, and use of EBP. A secondary purpose was to understand the obstacles and enablers to EBP adoption in the Italian osteopathic context.

METHODS

A cross-sectional national survey was conducted (April to June 2020) among a sample of Italian osteopaths. Eligible participants were invited to complete the Italian-translated Evidence-Based practice Attitude and Utilization Survey (EBASE) anonymously online using various recruitment strategies, including email and social media campaigns. In addition to the three EBASE sub-scores (attitudes, skills and use), the demographic characteristics of the sample were considered.

RESULTS

A total of 473 osteopaths responded to the survey. The sample appeared to represent the Italian osteopathic profession. The majority of participants had a favorable attitude toward EBP. Eighty-eight percent of respondents agreed that EBP was necessary for osteopathy practice and that scientific literature and research findings were beneficial to their clinical scenario (95%). Perceived skill levels in EBP were rated as moderate, with the lowest scores for items relating to clinical research and systematic review conduct. Apart from reading/reviewing scientific literature and using online search engines to locate relevant research papers, participant engagement in all other EBP-related activities was generally low. Clinical practice was perceived to be based on a very small proportion of clinical research evidence. The primary obstacles to EBP implementation were a dearth of clinical evidence in osteopathy, and poor skills in applying research findings. The primary enablers of EBP adoption were access to full-text articles, internet connectivity at work, and access to online databases.

CONCLUSIONS

Italian osteopaths were largely supportive of evidence-based practice but lacked basic skills in EBP and rarely engaged in EBP activities. The updating of osteopathic training curriculum and professional formal regulation in Italy could provide a suitable framework to improve EBP skills and use.

Manual Therapy: Always a Passive Treatment?

The use of manual therapy as an intervention has garnered intense debate, one that is often mired in a straw-man argument that manual therapy is a purely passive intervention. When passive interventions are equated with low-value care, it is easy to deride manual therapy as low-value care. However, manual therapy describes a wide variety of treatments, some with passive components and some that are primarily passive in certain scenarios. But manual therapy can be an integral part of highly active treatment strategies. We implore investigators to describe manual therapy interventions in sufficient detail so that they can be reproduced and thereby help the end users of research (including clinicians) to better assess the value of such interventions. This Viewpoint challenges the assumption that manual therapy is always a passive treatment of low value. J Orthop Sports Phys Ther 2021;51(10):474-477. Epub 1 Jul 2021. doi:10.2519/jospt.2021.10330.

Has Reporting on Physical Therapy Interventions Improved in 2 Decades? An Analysis of 140 Trials Reporting on 225 Interventions

OBJECTIVES

To investigate the completeness of reporting of physical therapy interventions in randomized controlled trials before and after publication of the Template for Intervention Description and Replication (TIDieR) checklist (a reporting guideline for interventions).

DESIGN

Meta-research.

METHODS

We searched 6 journals for trials using physical therapy interventions that were published in 2000 and 2018. Two independent assessors scored the TIDieR checklist and extracted descriptive information, including Physiotherapy Evidence Database (PEDro) scale scores. We identified control or treatment interventions, exercise-based interventions, and area of physical therapy. We performed a descriptive analysis and defined a priori a 20% difference between studies published in 2000 and 2018 as meaningful. We assessed correlations between TIDieR and PEDro scale scores for all interventions.

RESULTS

In total, 140 articles that met selection criteria evaluated 225 interventions (2000, n = 61; 2018, n = 164). Mean ± SD TIDieR score (2000, 7.52 ± 2.62; 2018, 8.26 ± 2.26) did not show a meaningful difference between years for all interventions (+5%), controls (+6%), treatment interventions (+6%), exercise-based interventions (+9%), or musculoskeletal (+4%) or neurological (+7%) physical therapy. For exercise interventions, number of sessions was reported more (+21%) in 2018 than in 2000. For musculoskeletal trials, 2 items were reported more completely in 2018 than in 2000 (materials, +29%; individual versus group, +22%) and 3 items were reported more completely in neurological trials (mode of delivery, +20%, [item 8.1] when +45%, and assessment of fidelity, +20%). The item "Who delivered the intervention?" was reported less completely (-23%) in 2018 than in 2000 in neurological trials. We found no correlation (r = 0.12) between PEDro scale score and TIDieR score.

CONCLUSION

There were few meaningful improvements in how physical therapy interventions were described after publication of the TIDieR reporting guideline. J Orthop Sports Phys Ther 2021;51(10):503-509. doi:10.2519/jospt.2021.10642.

Evidence based social science in China paper 2: The quality of social science RCTs published from 2000-2020

OBJECTIVE

This study collected randomized controlled trials (RCTs) published in the social sciences in China and assessed their risk of bias and reporting quality.

STUDY DESIGN AND SETTING

Three databases were systematically searched for publications from January 2000 to June 2020 for RCTs in the social sciences published by Chinese researchers. The risk of bias was assessed using the Cochrane Collaboration Risk of Bias Tool, and reporting quality was assessed using the Consolidated Standards of Reporting Trials for Social and Psychological Interventions (CONSORT-SPI).

RESULTS

A total of 316 RCTs were identified, including 204 articles in English and 112 articles in Chinese. The most frequently researched interventions focused on education (33.9%), and the most frequently studied population were students (32.9%). Eighty-seven percent of RCTs had intermediate reporting quality. Twenty-four of the 43 CONSORT-SPI sub-items had a compliance rate of less than 50%. Most RCTs had an unclear risk of bias for blinding outcome assessors (84.5%), blinding participants and personnel (82.9%), allocation concealment (73.1%), and random sequence generation (68.0%). A low proportion of CONSORT-SPI items were reported and, high proportion of the papers had unclear risk of bias.

CONCLUSION

The quality and reporting of RCTs in the social sciences needs improvement in China, especially for reporting methods and results. Most studies had an unclear risk of bias as they lacked important methodological information.

Declaration of use and appropriate use of reporting guidelines in high-impact rehabilitation journals is limited: a meta-research study

OBJECTIVES

The main aims of this metaresearch study conducted among high-impact rehabilitation journals were: 1) to evaluate if the use of reporting guidelines (RGs) was declared and 2) to categorize the declared use as appropriate or inappropriate.

STUDY DESIGN AND SETTING

Cross-sectional analysis of a random sample of 200 studies published in the period 2010-2019 in five generic rehabilitation journals with the highest 5-year impact factor. Randomized controlled trials, systematic reviews, observational studies, and diagnostic studies were included. Prevalence with 95% confidence intervals (CIs) was estimated for the main outcomes.

RESULTS

Among the 200 selected studies, 17.5% (95% CI: 12.2-22.8%) declared using RGs. Among these studies, 48.6% (95% CI: 32-65.1%) declared an appropriate use. There was an increasing trend over time for authors to report the use of RGs (OR: 1.31; 95% CI: 1.13-1.53). Systematic reviews (n = 54) reported more frequently the use of RGs than other study designs (35.2%).

CONCLUSION

In high-impact rehabilitation journals, a small minority of article authors declared the use of RGs. In approximately half of these studies, RGs were used inappropriately. There is an urgent need to improve the use of RGs in this field of research.

Sample size, study length, and inadequate controls were the most common self-acknowledged limitations in manual therapy trials: A methodological review

OBJECTIVES

The aim of this study was to quantify and analyze the presence and type of self-acknowledged limitations (SALs) in a sample of manual therapy (MT) randomized controlled trials.

STUDY DESIGN AND SETTING

We randomly selected 120 MT trials. We extracted data related to SALs from the original reports and classified them into 12 categories. After data extraction, specific limitations within each category were identified. A descriptive analysis was performed using frequencies and percentages for qualitative variables.

RESULTS

The number of SALs per trial article ranged from 0 to 8, and more than two-thirds of trials acknowledged at least two different limitations. Despite its small proportion, 9% of trials did not report SALs. The most common limitation declared, in almost half of our sample, related to sample size (47.5%) followed by limitations related to study length and follow-up (33.3%) and inadequate controls (32.5%).

CONCLUSION

Our results indicate that at least two different limitations are consistently acknowledged in MT trial reports, the most common being those related to sample size, study length, follow-up, and inadequate controls. Analysis of the reasons behind the SALs gives some insights about the main difficulties in conducting research in this field and may help develop strategies to improve future research.