How to guarantee finding a statistically significant difference: the use and abuse of subgroup analyses

Agreeableness and adolescents' cyberbullying perpetration: A longitudinal moderated mediation model of moral disengagement and empathy

OBJECTIVE

The current study explored whether agreeableness predicted cyberbullying perpetration across 3 years and extended previous studies by exploring the mediating effect of moral disengagement and the moderating effects of empathy and gender.

METHOD

The participants included 2407 adolescents from 7 middle schools in China. They were recruited to complete the Big Five Personality Inventory, Bullying Scale and Empathy Scale at Time 1, Moral Disengagement Scale at Time 1 and Time 2, and Cyberbullying Perpetration Scale at Time 1, Time 2, and Time 3.

RESULTS

Agreeableness at Time 1 predicted cyberbullying perpetration at Time 3 and moral disengagement at Time 2 mediated this relationship. The relationship between moral disengagement at Time 2 and cyberbullying perpetration at Time 3 was stronger for low cognitive empathy adolescents than high cognitive empathy adolescents at Time 1. The relationship between agreeableness at Time 1 and cyberbullying perpetration adolescents at Time 3 was stronger for low affective empathy than high affective empathy adolescents at Time 1. The link between moral disengagement at Time 2 and cyberbullying perpetration at Time 3 was weaker for females than males.

CONCLUSIONS

Low agreeableness adolescents are more likely to use moral disengagement, which in turn leads to more cyberbullying perpetration.

Creating falseness-How to establish statistical evidence of the untrue

Null hypothesis significance testing is the typical statistical approach in search of the truthfulness of hypotheses. This method does not formally consider the prior credence in the hypothesis, which affects the chances of reaching correct conclusions. When scientifically implausible or empirically weakly supported hypotheses are tested, there is an increased risk that a positive finding in a test in fact is false positive. This article argues that when scientifically weakly supported hypotheses are tested repeatedly-such as when studying the clinical effects of homeopathy-the accumulation of false positive study findings will risk providing false evidence also in systematic reviews and meta-analyses. False positive findings are detrimental to science and society, as once published, they accumulate persistent untrue evidence, which risks giving rise to nonpurposive research programmes, policy changes, and promotion of ineffective treatments. The problems with false positive findings are discussed, and advice is given on how to minimize the problem. The standard of evidence of a hypothesis should depend not only on the results of statistical analyses but also on its a priori support. Positive findings from studies investigating hypotheses with poor theoretical and empirical foundations should be viewed as tentative until the results are replicated and/or the hypothesis gains more empirical evidence supporting it as likely to be true.

Effectiveness of a Web-Based Tailored Intervention With Virtual Assistants Promoting the Acceptability of HPV Vaccination Among Mothers of Invited Girls: Randomized Controlled Trial

BACKGROUND

In 2010, the human papillomavirus (HPV) vaccination was introduced in the Dutch National Immunization Program for 12-year-old girls, aiming to reduce the incidence of cervical cancer in women. HPV vaccination uptake turned out to be lower than expected: 61% versus 70%, respectively. Mothers were shown to play the most important role in the immunization decision about this vaccination. They had also expressed their need for interactive personal information about the HPV vaccination over and above the existing universal general information. To improve the effectiveness of the existing education about the HPV vaccination, we systematically developed a Web-based tailored intervention with virtual assistants providing mothers of girls to be invited with tailored feedback on their decision making about the HPV vaccination.

OBJECTIVE

The aim of this study was to evaluate the effectiveness of the Web-based tailored intervention for promoting HPV vaccination acceptance by means of a randomized controlled trial (RCT).

METHODS

Mothers were recruited via the Dutch vaccination register (Praeventis) (n=36,000) and three Web-based panels (n=2483). Those who gave informed consent (N=8062) were randomly assigned to the control (n=4067) or intervention condition (n=3995). HPV vaccination uptake, as registered by Praeventis once the HPV vaccination round was completed, was used as the primary outcome. Secondary outcomes were differential scores across conditions between baseline (before the provided access to the new tailored intervention) and follow-up (just before the first vaccination) regarding the mothers' degree of informed decision making (IDM), decisional conflict, and critical determinants of HPV vaccination uptake among which are intention, attitude, risk perception, and outcome beliefs.

RESULTS

Intention-to-treat analysis (N=8062) showed a significant positive effect of the intervention on IDM, decisional conflict, and nearly all determinants of HPV vaccination uptake (P<.001). No effect was found on uptake (P=.60). This may be attributed to the overall high uptake rates in both conditions. Mothers evaluated the intervention as highly positive, including the website as well as the virtual assistants that were used to deliver the tailored feedback.

CONCLUSIONS

This computer-tailored intervention has the potential to improve HPV vaccination acceptability and IDM and to decrease decisional conflict among mothers of invited girls. Implications for future research are discussed.

TRIAL REGISTRATION

Trialregister.nl NTR4935; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4935 (Archived by WebCite at http://www.webcitation.org/6srT7l9EM).

A Subgroup Analysis Found no Diminished Response to Buprenorphine Transdermal System Treatment for Chronic Low Back Pain Patients Classified with Depression

BACKGROUND

Chronic pain (CP) patients with depression typically exhibit worse post-treatment outcomes than nondepressed CP patients. The cause is often assumed to reflect a differential response to treatment, neglecting other potential explanations, such as the continuation of differences in pretreatment outcomes. This post hoc analysis examines whether worse post-treatment outcomes for depressed patients with chronic low back pain (CLBP) are driven by reduced treatment efficacy.

METHODS

Data were from opioid-naïve adult patients with moderate-to-severe CLBP who participated in a randomized, placebo-controlled, double-blind clinical trial of Butrans(®) (buprenorphine) Transdermal System (BTDS) for pain relief. Depression screening was based on baseline SF-36v2 Mental Health subscale scores. Patient-reported measures of pain severity, pain interference, quality of life, sleep problems, and functional disability were administered at screening and during the study. Differential treatment efficacy for each outcome was examined using analysis of covariance models that included interaction terms between treatment arm and depression status.

RESULTS

At baseline, patients classified as depressed showed greater pain interference, lower quality of life, more sleep problems, and greater functional disability than nondepressed patients; the two groups did not differ in pain severity. No statistically significant interactions between treatment arm and depression status were observed. The direction of improvement post-treatment favored the depressed group on nine of seventeen outcomes.

CONCLUSIONS

Results do not support a differential response to BTDS treatment between depressed and nondepressed CLBP patients across a variety of patient-reported outcomes. These findings raise the question of whether depressed mood actually moderates the effectiveness of treatment in CP patients.

Interaction of treatment with a continuous variable: simulation study of significance level for several methods of analysis

Interactions between treatments and covariates in RCTs are a key topic. Standard methods for modelling treatment-covariate interactions with continuous covariates are categorisation or linear functions. Both approaches are easily criticised, but for different reasons. Multivariable fractional polynomial interactions, an approach based on fractional polynomials with the linear interaction model as the simplest special case, was proposed. Four variants of multivariable fractional polynomial interaction (FLEX1-FLEX4), allowing varying flexibility in functional form, were suggested. However, their properties are unknown, and comparisons with other procedures are unavailable. Additionally, we consider various methods based on categorisation and on cubic regression splines. We present the results of a simulation study to determine the significance level (probability of a type 1 error) of various tests for interaction between a binary covariate ('treatment effect') and a continuous covariate in univariate analysis. We consider a simplified setting in which the response variable is conditionally normally distributed, given the continuous covariate. We consider two main cases with the covariate distribution well behaved (approximately symmetric) or badly behaved (positively skewed). We construct nine scenarios with different functional forms for the main effect. In the well-behaved case, significance levels are in general acceptably close to nominal and are slightly better for the larger sample size (n = 250 and 500 were investigated). In the badly behaved case, departures from nominal are more pronounced for several approaches. For a final assessment of these results and recommendations for practice, a study of power is needed.

Prognosis research strategy (PROGRESS) 4: stratified medicine research

In patients with a particular disease or health condition, stratified medicine seeks to identify those who will have the most clinical benefit or least harm from a specific treatment. In this article, the fourth in the PROGRESS series, the authors discuss why prognosis research should form a cornerstone of stratified medicine, especially in regard to the identification of factors that predict individual treatment response

Impact of MMX® mesalamine on improvement and maintenance of health-related quality of life in patients with ulcerative colitis

BACKGROUND

Ulcerative colitis (UC) substantially reduces patients' health-related quality of life (HRQoL). The current study examined the burden of disease and the impact of daily multimatrix (MMX®) mesalamine treatment on HRQoL for patients with active or quiescent mild-to-moderate UC.

METHODS

Data were from a two-phase, multicenter, open-label study with mild-to-moderate UC patients. In the acute phase, 132 patients with active disease received MMX mesalamine 2.4-4.8 g/day QD for 8 weeks. In the maintenance phase, 207 patients with quiescent disease received MMX mesalamine 2.4 g/day QD for 12 months. The Short Form-12 (version 2) (SF-12v2) measured HRQoL during each phase. Disease burden was examined by comparing acute-phase baseline scores with a U.S. general population sample. Repeated-measures analyses assessed change in SF-12v2 scores for each phase. Correspondence between HRQoL and disease activity was examined through correlations between SF-12v2 scores with patient-reported symptom measures.

RESULTS

Baseline SF-12v2 scores for patients with UC were generally much lower than for the general population sample, indicating a broad disease burden. In the acute phase, significant improvement was observed for most SF-12v2 scores at week 8; a comparison with the matched norms showed a complete elimination of burden. No changes in SF-12v2 scores were observed during the maintenance phase. Changes in symptom measures and SF-12v2 scores were moderately correlated.

CONCLUSIONS

The sizeable burden of active mild-to-moderate UC on HRQoL was eliminated following 8 weeks' treatment with MMX mesalamine 2.4-4.8 g/day. HRQoL remained stable over 12 months of maintenance treatment in patients with quiescent UC.

The impact of MMX mesalazine on disease-specific health-related quality of life in ulcerative colitis patients

BACKGROUND

Past studies with ulcerative colitis (UC) patients indicate that disease activity strongly predicts health-related quality of life (HRQL).

AIM

To examine the degree to which daily treatment with MMX mesalazine predicts improved HRQL for patients with active UC and with stable HRQL for patients with quiescent UC.

METHODS

Data from two phases of a multicentre open-label trial were examined. In the acute phase, 132 patients with mild-to-moderate active UC received MMX mesalazine 2.4-4.8 g/day for 8 weeks, while 206 patients with quiescent UC received MMX mesalazine 2.4 g/day for a 12-month maintenance phase. Disease-specific HRQL was measured at baseline and endpoint of each phase using the Short Inflammatory Bowel Disease Questionnaire (SIBDQ). Repeated-measures anova models examined baseline-endpoint changes in SIBDQ, stool frequency (SF), and rectal bleeding severity (RBS). Correlations assessed the associations between SIBDQ and SF/RBS scores, while ancova techniques tested the sensitivity of SIBDQ to disease recurrence.

RESULTS

SIBDQ scores significantly increased for active mild-to-moderate UC patients following 8 weeks of treatment, while SIBDQ scores remained stable for quiescent UC patients following 12 months of treatment. Changes in SIBDQ scores correlated significantly with changes in SF and RBS scores. Patients with recurrent UC at maintenance phase endpoint had significantly lower SIBDQ scores than nonrecurrent patients.

CONCLUSIONS

Daily MMX mesalazine therapy was associated with significant improvement in disease-specific HRQL for patients with mild-to-moderate active UC and with the maintenance of HRQL for patients with quiescent UC. In both patient groups, HRQL was significantly associated with disease activity.

A comparison of subgroup analyses in grant applications and publications

In this paper, the authors compare subgroup analyses as outlined in grant applications and their related publications. Grants awarded by the Netherlands Organization for Health Research and Development (ZonMw) from 2001 onward that were finalized before March 1, 2010, were studied. Of the 79 grant proposals, 50 (63%) were intervention studies, 18 (23%) were diagnostic studies, and 6 (8%) were prognostic studies. Subgroups were mentioned in 49 (62%) grant applications and in 53 (67%) publications. In 20 of the 79 projects (25%), the publications were completely in agreement with the grant proposal; that is, subgroups that were prespecified in the grant proposal were reported and no new subgroup analyses were introduced in the publications. Of the 149 prespecified subgroups, 46 (31%) were reported in the final report or scientific publications, and 143 of the 189 (76%) reported subgroups were based on post-hoc findings. For 77% of the subgroup analyses in the publications, there was no mention of whether these were prespecified or post hoc. Justification for subgroup analysis and methods to study subgroups were rarely reported. The authors conclude that there is a large discrepancy between grant applications and final publications regarding subgroup analyses. Both nonreporting prespecified subgroup analyses and reporting post-hoc subgroup analyses are common. More guidance is clearly needed.

Exploring the efficacy and moderators of two computer-tailored physical activity interventions for older adults: a randomized controlled trial

BACKGROUND

Important health benefits can be derived when low-cost (e.g., computer-tailored) physical activity interventions for older adults demonstrate sustained effects.

PURPOSE

The purpose of the study was to conduct in-depth analysis on the long-term efficacy of two tailored physical activity interventions for older adults.

METHODS

A randomized controlled trial (n = 1,971) with two computer-tailored interventions and a no-intervention control group was conducted. The two tailored interventions consisted of three tailored letters, delivered during 4 months. The basic tailored intervention targeted psychosocial determinants alone, while the environmentally tailored intervention additionally targeted environmental determinants, by providing tailored environmental information. Self-reported behaviors (i.e., total physical activity, transport walking and cycling, leisure walking and cycling, and sports) were measured at baseline and 12 months. Additionally, potential personal, health-related, and psychosocial moderators of the intervention effects were examined.

RESULTS

The environmentally tailored intervention was effective in changing total physical activity, leisure cycling, and sports compared with the basic intervention and control group. No intervention effects were found for the basic intervention. Moderation analysis revealed that participants with a higher age, lower body mass index, and higher intention were unresponsive to the interventions.

CONCLUSIONS

Providing environmental information is an effective intervention strategy for increasing physical activity behaviors among older adults, especially among certain "at-risk" subgroups such as lower educated, overweight, or insufficiently active participants. Moderation analysis was perceived as a promising method for identifying meaningful subgroups that are unaffected by an intervention, which should receive special attention in future interventions.