Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression

Computerized Cognitive and Skills Training in Older People With Mild Cognitive Impairment: Using Ecological Momentary Assessment to Index Treatment-Related Changes in Real-World Performance of Technology-Dependent Functional Tasks

OBJECTIVES

Cognitive and functional skills training improves skills and cognitive test performance, but the true test of efficacy is real-world transfer. We trained participants with mild cognitive impairment (MCI) or normal cognition (NC) for up to 12 weeks on six technology-related skills using remote computerized functional skills assessment and training (FUNSAT) software. Using ecological momentary assessment (EMA), we measured real-world performance of the technology-related skills over 6 months and related EMA-identified changes in performance to training gains.

DESIGN

Randomized clinical trial with post-training follow-up.

SETTING

A total of 14 Community centers in New York City and Miami.

PARTICIPANTS

Older adults with normal cognition (n = 72) or well-defined MCI (n = 92), ranging in age from 60 to 90, primarily female, and racially and ethnically diverse.

INTERVENTION

Computerized cognitive and skills training.

MEASUREMENTS

EMA surveys measuring trained and untrained functional skills 3 or more days per week for 6 months and training gains from baseline to end of training.

RESULTS

Training gains in completion times across all 6 tasks were significant (p <0.001) for both samples, with effect sizes more than 1.0 SD for all tasks. EMA surveys detected increases in performance for both trained (p <0.03) and untrained (p <0.001) technology-related skills for both samples. Training gains in completion times predicted increases in performance of both trained and untrained technology-related skills (all p <0.001).

CONCLUSIONS

Computerized training produces increases in real-world performance of important technology-related skills. These gains continued after the end of training, with greater gains in MCI participants.

Test-retest reliability of a smartphone-based approach-avoidance task: Effects of retest period, stimulus type, and demographics

The approach-avoidance task (AAT) is an implicit task that measures people's behavioral tendencies to approach or avoid stimuli in the environment. In recent years, it has been used successfully to help explain a variety of health problems (e.g., addictions and phobias). Unfortunately, more recent AAT studies have failed to replicate earlier promising findings. One explanation for these replication failures could be that the AAT does not reliably measure approach-avoidance tendencies. Here, we first review existing literature on the reliability of various versions of the AAT. Next, we examine the AAT's reliability in a large and diverse sample (N = 1077; 248 of whom completed all sessions). Using a smartphone-based, mobile AAT, we measured participants' approach-avoidance tendencies eight times over a period of seven months (one measurement per month) in two distinct stimulus sets (happy/sad expressions and disgusting/neutral stimuli). The mobile AAT's split-half reliability was adequate for face stimuli (r = .85), but low for disgust stimuli (r = .72). Its test-retest reliability based on a single measurement was poor for either stimulus set (all ICC1s < .3). Its test-retest reliability based on the average of all eight measurements was moderately good for face stimuli (ICCk = .73), but low for disgust stimuli (ICCk = .5). Results suggest that single-measurement AATs could be influenced by unexplained temporal fluctuations of approach-avoidance tendencies. These fluctuations could be examined in future studies. Until then, this work suggests that future research using the AAT should rely on multiple rather than single measurements.

Brief self-affirmation intervention for adults with psoriasis for reducing anxiety and depression and boosting well-being: Evidence from a randomized controlled trial

BACKGROUND

There are relatively few studies to address mental health implications of self-affirming, especially across groups experiencing a chronic health condition. In this study, short- and longer-term effects of a brief self-affirmation intervention framed in terms of implementation intentions (if-then plans with self-affirming cognitions; S-AII) were evaluated against an active control group (non-affirming implementation intentions; N-AII), matched to the target condition, and mere goal intention condition (a non-active control) in adults with psoriasis. The three pre-registered primary outcomes captured depression, anxiety, and well-being.

METHODS

Adults with psoriasis (N = 175; M_age = 36.53, s.d. = 11.52) were randomized into S-AII, N-AII, or control. Participants' mental health outcomes were assessed prior to randomization (at baseline), at week 2 (post-intervention), and at a 1-month follow-up.

RESULTS

Linear mixed models were used and results were reported on the intention-to-treat principle. Analyses revealed that S-AII exerted significantly more improvement in the course of well-being (ds > 0.25), depressive symptoms (ds > -0.40), and anxiety (ds > -0.45) than the N-AII and control group at 2-week post-intervention. Though the differences between groups faded at 1-month follow-up, the within-group changes over time for S-AII in all mental health outcomes remained significant.

CONCLUSIONS

Brief and low-intensity S-AII intervention exerted in the short-term a considerable impact on mental health outcomes. The S-AII shows promising results as a relevant public mental health strategy for enhancing well-being and reducing psychological distress. Future studies could consider whether these effects can be further enhanced with booster interventions.

Real-time Symptom Assessment in Patients With Endometriosis: Psychometric Evaluation of an Electronic Patient-Reported Outcome Measure, Based on the Experience Sampling Method

BACKGROUND

The experience sampling method (ESM) holds advantages over traditional retrospective questionnaires including a high ecological validity, no recall bias, the ability to assess fluctuation of symptoms, and the ability to analyze the temporal relationship between variables.

OBJECTIVE

This study aimed to evaluate the psychometric properties of an endometriosis-specific ESM tool.

METHODS

This is a short-term follow-up prospective study, including patients with premenopausal endometriosis aged ≥18 years who reported dysmenorrhea, chronic pelvic pain, or dyspareunia between December 2019 and November 2020. An ESM-based questionnaire was sent out by a smartphone application 10 times a day during 1 week on randomly chosen moments. Additionally, patients completed questionnaires concerning demographics, end-of-day pain scores, and end-of-week symptom scores. The psychometric evaluation included compliance, concurrent validity, and internal consistency.

RESULTS

Twenty-eight patients with endometriosis completed the study. Compliance for answering the ESM questions was as high as 52%. End-of-week pain scores were higher than ESM mean scores and showed peak reporting. ESM scores showed strong concurrent validity when compared with symptoms scored by the Gastrointestinal Symptom Rating Scale-Irritable Bowel Syndrome, 7-item Generalized Anxiety Disorders Scale, 9-question Patient Health Questionnaire, and the majority of questions of the 30-item Endometriosis Health Profile. Cronbach α coefficients demonstrated a good internal consistency for abdominal symptoms, general somatic symptoms, and positive affect, and an excellent internal consistency for negative affect.

CONCLUSIONS

This study supports the validity and reliability of a newly developed electronic instrument for the measurement of symptoms in women with endometriosis, based on momentary assessments. This ESM patient-reported outcome measure has the advantage of providing a more detailed view on individual symptom patterns and offers the possibility for patients to have insight in their symptomatology, leading to more individualized treatment strategies that can improve the quality of life of women with endometriosis.

Patient-specific affect-abdominal pain interactions in endometriosis: an experience sampling method (ESM) study

OBJECTIVES

Cross-sectional studies show that endometriosis-related pain is associated with affect. Measuring these symptoms in real-time in a longitudinal perspective yields the ability to analyze the temporal relationship between variables. The aim was to evaluate the association between affect and abdominal pain, using the Experience Sampling Method (ESM) as a real-time, randomly repeated assessment.

METHODS

Thirty-four endometriosis patients and 31 healthy subjects completed up to 10 real-time self-assessments concerning abdominal pain and affective symptoms during seven consecutive days.

RESULTS

Endometriosis patients experienced more abdominal pain and negative affective symptoms, and scored lower on positive affect compared to healthy controls. A significant association was found between abdominal pain and both positive and negative affect in endometriosis patients. For healthy controls, less strong or non-significant associations were found. When looking at abdominal pain as a predictor for affect and vice versa, we found that only in endometriosis patients, pain was subsequently accompanied by negative affect, and positive affect may alleviate pain in these patients.

CONCLUSIONS

This study confirms a concurrent and temporal relationship between affect and abdominal pain in endometriosis patients and supports the use of real-time symptom assessment to interpret potential influencers of abdominal complaints in patients with endometriosis.

Enhancing well-being and alleviating depressive symptoms in people with HIV/AIDS: An intervention based on if-then plans with self-affirming cognitions

Effective antiretroviral treatment has increased the life expectancy of people living with HIV, and currently, the challenges of prominent importance appear to be mental health issues. This preregistered study among adults living with HIV/AIDS investigated the effectiveness of a brief self‐affirmation intervention framed in terms of if–then plans (i.e. self‐affirming implementation intentions [S‐AII]) against both active and non‐active control conditions, forming non‐affirming implementation intentions and mere goal intentions, respectively. The primary outcomes were defined as a reduction of depressive symptoms and enhancement of well‐being, along with secondary outcomes as positive other‐ and self‐directed feelings. A total of 162 individuals were assessed for eligibility, and 130 (aged 18–74 years) were randomized to the study conditions. Intervention effects were estimated through intention‐to‐treat analysis, using linear mixed models. The S‐AII intervention yielded improvements in overall well‐being over 2 weeks (d = .23), primarily driven by positive changes in emotional (d = .24) and social (d = .30) dimensions of well‐being. There were no significant differences in depression or secondary outcomes. Based on a minimal clinically important difference index, the S‐AII intervention resulted in improvement in well‐being in approximately 40 percent of participants. Nevertheless, further systematic research is needed to optimize self‐affirmation‐interventions, before their application in real‐life contexts.

A Factorial Randomized Controlled Trial of Implementation-Intention-Based Self-Affirmation Interventions: Findings on Depression, Anxiety, and Well-being in Adults With Psoriasis

This study builds on growing evidence on implementation-intention-based self-affirmation intervention effects on mental health. Using a factorial design, this pre-registered study aimed to further investigate whether (1) strengthening the element of specificity within body-related self-affirming implementation intention (BS-AII) intervention compared to general self-affirming implementation intention (S-AII) would provide greater improvements in mental health outcomes for adults with psoriasis, and (2) whether the addition of a booster component would result in enhancing effectiveness at follow-up. A total of 306 adults with psoriasis were assessed for eligibility and 222 (aged 18-71 years) were randomized and received S-AII, BS-AII, or MGI (mere goal intention-control condition). Within each group, participants were again randomized to booster (B) or no-booster condition in a 3 × 2 factorial design, resulting in six groups: S-AII; S-AII + B; BS-AII; BS-AII + B; MGI; and MGI + B. Data were collected over three-time points, at baseline, 2 weeks post-intervention, and at 1-month later. Three primary outcomes were defined as a reduction of anxiety and depressive symptoms and enhancement of well-being. In terms of secondary outcomes, positive other- and self-directed feelings and also an emotional attitude toward the body were evaluated. To fully estimate intervention effects through intention-to-treat analysis, linear mixed models were used. A significant effect of time was observed, but no evidence of time-by-group interactions and no three-way interactions were detected. Exploratory analyses revealed two significant moderating effects of age and self-esteem, pointing to boundary conditions of the interventions. These findings offer to gain deeper insights on null (or negative) effects also reported in past works and highlight that self-affirmation interventions should be more thoroughly investigated and optimized before they can be broadly implemented in real-life contexts, especially to prevent backfiring and negative-enhancing effects.

Does Day-to-Day Variability in Stool Consistency Link to the Fecal Microbiota Composition?

Introduction

Stool consistency has been associated with fecal microbial composition. Stool consistency often varies over time, in subjects with and without gastrointestinal disorders, raising the question whether variability in the microbial composition should be considered in microbiota studies. We evaluated within-subject day-to-day variability in stool consistency and the association with the fecal microbiota in irritable bowel syndrome (IBS) and healthy subjects, over seven days.

Methods

Twelve IBS patients and 12 healthy subjects collected fecal samples during seven consecutive days. Stool consistency was determined by the patient-reported Bristol Stool Scale (BSS) and fecal dry weight percentage. 16S rRNA V4 gene sequencing was performed and microbial richness (alpha diversity; Chao1 index, observed number of species, effective Shannon index) and microbial community structure (beta diversity; Bray-Curtis distance, generalized UniFrac, and taxa abundance on family level) were determined.

Results

Linear mixed-effects models showed significant associations between stool consistency and microbial richness, but no time effect. This implies that between-subject but not within-subject variation in microbiota over time can partially be explained by variation in stool consistency. Redundancy analysis showed a significant association between stool consistency and microbial community structure, but additional linear mixed-effects models did not demonstrate a time effect on this.

Conclusion

This study supports an association between stool consistency and fecal microbiota, but no effect of day-to-day fluctuations in stool consistency within seven days. This consolidates the importance of considering stool consistency in gut microbiota research, though confirms the validity of single fecal sampling to represent an individual's microbiota at a given time point. NCT00775060.

Sample size, sample size planning, and the impact of study context: systematic review and recommendations by the example of psychological depression treatment

BACKGROUND

Sample size planning (SSP) is vital for efficient studies that yield reliable outcomes. Hence, guidelines, emphasize the importance of SSP. The present study investigates the practice of SSP in current trials for depression.

METHODS

Seventy-eight randomized controlled trials published between 2013 and 2017 were examined. Impact of study design (e.g. number of randomized conditions) and study context (e.g. funding) on sample size was analyzed using multiple regression.

RESULTS

Overall, sample size during pre-registration, during SSP, and in published articles was highly correlated (r's ≥ 0.887). Simultaneously, only 7-18% of explained variance related to study design (p = 0.055-0.155). This proportion increased to 30-42% by adding study context (p = 0.002-0.005). The median sample size was N = 106, with higher numbers for internet interventions (N = 181; p = 0.021) compared to face-to-face therapy. In total, 59% of studies included SSP, with 28% providing basic determinants and 8-10% providing information for comprehensible SSP. Expected effect sizes exhibited a sharp peak at d = 0.5. Depending on the definition, 10.2-20.4% implemented intense assessment to improve statistical power.

CONCLUSIONS

Findings suggest that investigators achieve their determined sample size and pre-registration rates are increasing. During study planning, however, study context appears more important than study design. Study context, therefore, needs to be emphasized in the present discussion, as it can help understand the relatively stable trial numbers of the past decades. Acknowledging this situation, indications exist that digital psychiatry (e.g. Internet interventions or intense assessment) can help to mitigate the challenge of underpowered studies. The article includes a short guide for efficient study planning.

More Light? Opportunities and Pitfalls in Digitalized Psychotherapy Process Research

While the evidence on the effectiveness of different psychotherapies is often strong, it is not settled whereby and how these therapies work. Knowledge on the causal factors and change mechanisms is of high clinical and public relevance, as it contributes to the empirically informed advancement of psychotherapeutic interventions. Here, digitalized research approaches might possess the potential to generate new insights into human behavior change, contributing to augmented interventions and mental healthcare practices with better treatment outcomes. In this perspective article, we describe recent findings of research into change mechanisms that were only feasible with digital tools and outline important future directions for this rather novel branch of research. Furthermore, we indicate several challenges and pitfalls that are to be solved, in order to advance digitalized psychotherapy process research, both methodologically and technologically.

A Blended Cognitive-Behavioral Intervention for the Treatment of Postpartum Depression: Study Protocol for a Randomized Controlled Trial

Despite the existence of effective treatment for postpartum depression, few women seek professional help, indicating the need for a new and innovative format of treatment that can overcome help-seeking barriers. This article presents the study protocol for a blended cognitive–behavioral intervention for the treatment of postpartum depression, by integrating face-to-face sessions with a web-based program (Be a Mom) into one treatment protocol. This study will be a two-arm, noninferiority randomized controlled trial comparing blended intervention to usual treatment for postpartum depression provided in healthcare centers. Portuguese postpartum adult women diagnosed with postpartum depression (according to the DSM-5 diagnostic criteria for major depressive disorder) will be recruited during routine care appointments in local healthcare centers and will be eligible to participate. Measures will be completed at baseline, postintervention, and at three- and six-month follow-ups. The primary outcome will be depressive symptoms. Secondary outcomes will include anxiety symptoms, fatigue, quality of life, marital satisfaction, maternal self-efficacy, and mother–child bonding. Cost-effectiveness analysis and mediator and moderator analysis will be conducted. This study will provide insight into the efficacy and cost-effectiveness of a blended psychological intervention in the Portuguese context and increase the empirically validated treatment options for postpartum depression.

Immediate and long-term effectiveness of adding an Internet intervention for depression to routine outpatient psychotherapy: Subgroup analysis of the EVIDENT trial

OBJECTIVE

To examine immediate and long-term effectiveness of an adjunctive Internet intervention for depression in a large sample of patients undergoing routine psychotherapy.

METHOD

The current study evaluated a subgroup of patients from the Evident trial, a randomized investigation of a 12-week minimally guided Internet intervention (Deprexis) for the treatment of mild to moderate depression. 340 adults (mean age = 43.3 years; 71.7 % female) of the original sample received routine outpatient psychotherapy during the trial period, resulting in a standard psychotherapy group (n = 174) and an augmented therapy group (n = 166). Outcomes were assessed at baseline, post-treatment and 6-month follow-up.

RESULTS

Intention-to-treat analyses indicated that combined treatment led to a greater reduction in symptoms of depression (effect size d = 0.32; p = .002), improved therapeutic progress (d = 0.36; p = .003), and higher mental health-related quality of life (d = 0.34; p = .004). There was no intervention effect on physical health-related quality of life. The same pattern was found at 6-month follow-up, and adjunctive treatment also resulted in increased rates of clinical improvement. Treatment success was independent from therapeutic orientation of combined face-to-face therapy.

CONCLUSION

Results indicate that the adjunctive use of the investigated intervention can produce additional and lasting effects in routine outpatient psychotherapy for mild to moderate levels of depression. The study adds to the ongoing evidence on augmented effects of blended treatment. Future studies should investigate different types of blends in diverse populations by means of change-sensitive assessment strategies.

Introducing the DizzyQuest: an app-based diary for vestibular disorders

Background

Most questionnaires currently used for assessing symptomatology of vestibular disorders are retrospective, inducing recall bias and lowering ecological validity. An app-based diary, administered multiple times in daily life, could increase the accuracy and ecological validity of symptom measurement. The objective of this study was to introduce a new experience sampling method (ESM) based vestibular diary app (DizzyQuest), evaluate response rates, and to provide examples of DizzyQuest outcome measures which can be used in future research.

Methods

Sixty-three patients diagnosed with a vestibular disorder were included. The DizzyQuest consisted of four questionnaires. The morning- and evening-questionnaires were administered once each day, the within-day-questionnaire 10 times a day using a semi-random time schedule, and the attack questionnaire could be completed after the occurrence of a vertigo or dizziness attack. Data were collected for 4 weeks. Response rates and loss-to-follow-up were determined. Reported symptoms in the within-day-questionnaire were compared within and between patients and subgroups of patients with different vestibular disorders.

Results

Fifty-one patients completed the study period. Average response rates were significantly higher than the desired response rate of > 50% (p < 0.001). The attack-questionnaire was used 159 times. A variety of neuro-otological symptoms and different disease profiles were demonstrated between patients and subgroups of patients with different vestibular disorders.

Conclusion

The DizzyQuest is able to capture vestibular symptoms within their psychosocial context in daily life, with little recall bias and high ecological validity. The DizzyQuest reached the desired response rates and showed different disease profiles between subgroups of patients with different vestibular disorders. This is the first time ESM was used to assess daily symptoms and quality of life in vestibular disorders, showing that it might be a useful tool in this population.

Electronic supplementary material

The online version of this article (10.1007/s00415-020-10092-2) contains supplementary material, which is available to authorized users.