Evaluation of randomized controlled trials: a primer and tutorial for mental health researchers

Guided web app intervention for reducing symptoms of depression in postpartum women: Results of a feasibility randomized controlled trial

Background

Chile faces a significant postpartum depression prevalence and treatment gap, necessitating accessible interventions. While cognitive-behavioral internet-based interventions have proven effective in high-income countries, this field is underdeveloped in Chile. Based on the country's widespread use of digital technology, a guided 8-week cognitive-behavioral web app intervention named "Mamá, te entiendo" was developed.

Objective

This study aimed to assess the acceptability and feasibility of "Mamá, te entiendo", for reducing depressive symptomatology in postpartum women.

Methods

Sixty-five postpartum women with minor or major depression were randomly assigned to either intervention or waitlist. Primary outcomes centered on study feasibility, intervention feasibility, and acceptability. Semi-structured interviews with a sub-sample enriched the understanding of participants' experiences. Secondary outcomes included mental health variables assessed at baseline, post-intervention, and 1-month follow-up.

Results

Chilean women displayed great interest in the intervention. 44.8 % of participants completed the intervention. Participants reported high satisfaction and engagement levels, with interviewees highlighting the value of the intervention's content, exercises, and therapist's feedback. However, preliminary efficacy analysis didn't reveal a significant interaction between group and time for outcome measures.

Discussion

This research represents a pioneering effort in Chile to evaluate an internet-based intervention for postpartum depression symptoms. The demonstrated feasibility and acceptability highlight the potential of integrating technology-driven approaches into mental health interventions. However, the intervention did not demonstrate superiority, as both groups exhibited similar positive progress in several outcomes. Therefore, the following research phase should involve a larger and more diverse sample to assess the intervention's effectiveness, identify influencing factors, and determine the individuals who benefit the most.

Advancing randomized controlled trial methodologies: The place of innovative trial design in eating disorders research

Randomized controlled trials can be used to generate evidence on the efficacy and safety of new treatments in eating disorders research. Many of the trials previously conducted in this area have been deemed to be of low quality, in part due to a number of practical constraints. This article provides an overview of established and more innovative clinical trial designs, accompanied by pertinent examples, to highlight how design choices can enhance flexibility and improve efficiency of both resource allocation and participant involvement. Trial designs include individually randomized, cluster randomized, and designs with randomizations at multiple time points and/or addressing several research questions (master protocol studies). Design features include the use of adaptations and considerations for pragmatic or registry-based trials. The appropriate choice of trial design, together with rigorous trial conduct, reporting and analysis, can establish high-quality evidence to advance knowledge in the field. It is anticipated that this article will provide a broad and contemporary introduction to trial designs and will help researchers make informed trial design choices for improved testing of new interventions in eating disorders. PUBLIC SIGNIFICANCE: There is a paucity of high quality randomized controlled trials that have been conducted in eating disorders, highlighting the need to identify where efficiency gains in trial design may be possible to advance the eating disorder research field. We provide an overview of some key trial designs and features which may offer solutions to practical constraints and increase trial efficiency.

Every mistake is a treasure: Lessons learned from the TRIANGLE trial for anorexia nervosa

BACKGROUND

Despite several decades of treatment research for anorexia nervosa (AN), many of the same questions remain: how to boost enrollment, engage participants, prevent attrition, and meet the needs of a diverse patient population within the rigorous framework of a randomized controlled trial (RCT).

METHODS

In this research forum, we highlight some of the challenges and opportunities observed over the course of TRIANGLE, the largest RCT for severe AN treatment in the UK to date. We discuss strategies for addressing common challenges and avoiding common pitfalls and propose solutions to future researchers seeking to conduct treatment research in AN.

RESULTS

Our experience underscores the value of involving people with lived experience at every stage of intervention research. We offer additional recommendations for treatment researchers, including, (1) early qualitative research to identify patient barriers and obstacles, (2) clear, systematic collaboration with clinical sites for patient recruitment and passive data collection, (3) careful consideration of assessment metrics, including repeated measurement of quality of life, (4) adopting a flexible, patient-centered approach to clinical trial research, and (5) considering the unique needs and obstacles that might impact carer participation in research and their ability to provide support to their loved ones.

DISCUSSION

We hope that these lessons learned will prove fruitful for the next generation of researchers embarking on treatment research for AN.

PUBLIC SIGNIFICANCE

Using the TRIANGLE trial as an illustrative case study, we highlight the value of lived experience and codesign for developing and testing interventions for AN. We offer several lessons learned over the course of the trial, pertaining to trial enrollment, retention and engagement, measurement of outcomes, and research adaptations for real-world settings, and hope that these recommendations facilitate future treatment research for AN.

Evaluating the Practicality of Causal Inference From Non-randomized Observational Data in Small-Scale Clinical Settings: A Study on the Effects of Ninjin'yoeito

Objective The primary goal of this study was to demonstrate the practical application of causal inference using non-randomized observational data, adapting this approach to smaller populations, such as those in hospitals or community healthcare. This adaptation seeks a more effective and practical research method than randomized controlled trials (RCTs), with the goal of revealing novel insights unexplored by traditional research and enhancing understanding within the realm of causal inference. Methods This study evaluated the effects of Ninjin'yoeito (NYT), a traditional Japanese Kampo medicine, on Overactive Bladder Symptom Score (OABSS) and the frailty scores. Employing new statistical methods, this study sought to illustrate the efficacy of estimating causal relationships from non-randomized data in a clinical setting. The database included 985 women aged 65-90 years who visited a clinic between November 2016 and November 2022. By utilizing various statistical techniques, including regression analysis, inverse probability of treatment weighting (IPTW), instrumental variable (IV), and difference-in-differences (DiD) analysis, this study aimed to provide insights beyond traditional methods, attempting to bridge the gap between theory and practice in causal inference. Results After applying propensity score matching, the NYT treatment group (220 participants) and non-treatment group (182 participants) were each adjusted to two groups of 159 individuals. NYT significantly improved OABSS and frailty scores. IPTW analysis highlighted that on average, the NYT treatment group showed an improvement of 0.8671 points in OABSS and 0.1339 points in the frailty scores, surpassing the non-treatment group (p<0.05). IV analysis indicated that NYT treatment is predicted to increase ΔOABSS by an average of approximately 4.86 points, highlighting its significant positive impact on OABSS improvement. The DiD analysis showed that the NYT treatment group demonstrated an average improvement of 0.5457 points in OABSS, which was significantly higher than that of the control group. The adjusted R² value for the model is 0.025. Conclusion This study successfully implemented a practical application of causal inference using non-randomized observational data in a relatively small population. NYT showed a significant improvement in OABSS and vulnerability, and this result was confirmed using a new statistical method. The relatively low adjusted R² of the model suggests the existence of other unmeasured variables that influence OABSS and vulnerability improvement. In particular, the use of diverse statistical techniques, including IPTW, IV, and DiD analysis, is an important step toward revealing the effectiveness of inferring causal relationships from non-randomized data and narrowing the gap between theory and practice. This study provides a valid and practical alternative to RCTs and reveals new insights that have not been explored in traditional research.

Additive effects of adjunctive app-based interventions for mental disorders - A systematic review and meta-analysis of randomised controlled trials

Background

It is uncertain whether app-based interventions add value to existing mental health care.

Objective

To examine the incremental effects of app-based interventions when used as adjunct to mental health interventions.

Methods

We searched PubMed, PsycINFO, Scopus, Web of Science, and Cochrane Library databases on September 15th, 2023, for randomised controlled trials (RCTs) on mental health interventions with an adjunct app-based intervention compared to the same intervention-only arm for adults with mental disorders or respective clinically relevant symptomatology. We conducted meta-analyses on symptoms of different mental disorders at postintervention. PROSPERO, CRD42018098545.

Results

We identified 46 RCTs (4869 participants). Thirty-two adjunctive app-based interventions passively or actively monitored symptoms and behaviour, and in 13 interventions, the monitored data were sent to a therapist. We found additive effects on symptoms of depression (g = 0.17; 95 % CI 0.02 to 0.33; k = 7 comparisons), anxiety (g = 0.80; 95 % CI 0.06 to 1.54; k = 3), mania (g = 0.2; 95 % CI 0.02 to 0.38; k = 4), smoking cessation (g = 0.43; 95 % CI 0.29 to 0.58; k = 10), and alcohol use (g = 0.23; 95 % CI 0.08 to 0.39; k = 7). No significant effects were found on symptoms of depression within a bipolar disorder (g = -0.07; 95 % CI -0.37 to 0.23, k = 4) and eating disorders (g = -0.02; 95 % CI -0.44 to 0.4, k = 3). Studies on depression, mania, smoking, and alcohol use had a low heterogeneity between the trials. For other mental disorders, only single studies were identified. Only ten studies had a low risk of bias, and 25 studies reported insufficient statistical power.

Discussion

App-based interventions may be used to enhance mental health interventions to further reduce symptoms of depression, anxiety, mania, smoking, and alcohol use. However, the effects were small, except for anxiety, and limited due to study quality. Further high-quality research with larger sample sizes is warranted to better understand how app-based interventions can be most effectively combined with established interventions to improve outcomes.