Adjustment for baseline measurement error in randomized controlled trials induces bias

Calibrating validation samples when accounting for measurement error in intervention studies

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers.

Digital training for non-specialist health workers to deliver a brief psychological treatment for depression in India: Protocol for a three-arm randomized controlled trial

BACKGROUND

Training non-specialist health workers (NSHWs) at scale is a major barrier to increasing the coverage of depression care in India. This trial will test the effectiveness of two forms of digital training compared to conventional face-to-face training in changing the competence of NSHWs to deliver a brief evidence-based psychological treatment for depression.

METHODS

This protocol is for a three-arm, parallel group randomized controlled trial comparing three ways of training NSHWs to deliver the Healthy Activity Program (HAP), a brief manualized psychotherapy for depression, in primary care. The arms are: digital training (DGT); digital training combined with individualized coaching support (DGT+); and conventional face-to-face training (F2F). The target sample comprises N = 336 government contracted NSHWs in Madhya Pradesh, India. The primary outcome is change of competence to deliver HAP; secondary outcomes include cost-effectiveness of the training programs, change in participants' mental health knowledge, attitudes and behavior, and satisfaction with the training. Assessors blind to participant allocation status will collect outcomes pre- (baseline) and post- (endpoint) training to ascertain differences in outcomes between arms. Training program costs will be collected to calculate incremental costs of achieving one additional unit on the competency measure in the digital compared to face-to-face training programs. Health worker motivation, job satisfaction, and burnout will be collected as exploratory outcome variables.

DISCUSSION

This trial will determine whether digital training is an effective, cost-effective, and scalable approach for building workforce capacity to deliver a brief evidence-based psychological treatment for depression in primary care in a low-resource setting.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04157816.

Text-mining in electronic healthcare records can be used as efficient tool for screening and data collection in cardiovascular trials: a multicenter validation study

OBJECTIVE

This study aimed to validate trial patient eligibility screening and baseline data collection using text-mining in electronic healthcare records (EHRs), comparing the results to those of an international trial.

STUDY DESIGN AND SETTING

In three medical centers with different EHR vendors, EHR-based text-mining was used to automatically screen patients for trial eligibility and extract baseline data on nineteen characteristics. First, the yield of screening with automated EHR text-mining search was compared with manual screening by research personnel. Second, the accuracy of extracted baseline data by EHR text mining was compared to manual data entry by research personnel.

RESULTS

Of the 92,466 patients visiting the out-patient cardiology departments, 568 (0.6%) were enrolled in the trial during its recruitment period using manual screening methods. Automated EHR data screening of all patients showed that the number of patients needed to screen could be reduced by 73,863 (79.9%). The remaining 18,603 (20.1%) contained 458 of the actual participants (82.4% of participants). In trial participants, automated EHR text-mining missed a median of 2.8% (Interquartile range [IQR] across all variables 0.4-8.5%) of all data points compared to manually collected data. The overall accuracy of automatically extracted data was 88.0% (IQR 84.7-92.8%).

CONCLUSION

Automatically extracting data from EHRs using text-mining can be used to identify trial participants and to collect baseline information.

Different ways to estimate treatment effects in randomised controlled trials

Background

Regarding the analysis of RCT data there is a debate going on whether an adjustment for the baseline value of the outcome variable should be made. When an adjustment is made, there is a lot of misunderstanding regarding the way this should be done. Therefore, the aims of this educational paper are: 1) to explain different methods used to estimate treatment effects in RCTs, 2) to illustrate the different methods with a real life example and 3) to give an advise on how to analyse RCT data.

Methods

Longitudinal analysis of covariance, repeated measures analysis in which also the baseline value is used as outcome and the analysis of changes were theoretically explained and applied to an example dataset investigating a systolic blood pressure lowering treatment.

Results

It was shown that differences at baseline should be taken into account and that regular repeated measures analysis and regular analysis of changes did not adjust for the baseline differences between the groups and therefore lead to biased estimates of the treatment effect. In the real life example, due to the differences at baseline between the treatment and control group, the different methods lead to different estimates of the treatment effect.

Conclusion

Regarding the analysis of RCT data, it is advised to use longitudinal analysis of covariance or a repeated measures analysis without the treatment variable, but with the interaction between treatment and time in the model.

Empirical comparison of four baseline covariate adjustment methods in analysis of continuous outcomes in randomized controlled trials

Background

Although seemingly straightforward, the statistical comparison of a continuous variable in a randomized controlled trial that has both a pre- and posttreatment score presents an interesting challenge for trialists. We present here empirical application of four statistical methods (posttreatment scores with analysis of variance, analysis of covariance, change in scores, and percent change in scores), using data from a randomized controlled trial of postoperative pain in patients following total joint arthroplasty (the Morphine COnsumption in Joint Replacement Patients, With and Without GaBapentin Treatment, a RandomIzed ControlLEd Study [MOBILE] trials).

Methods

Analysis of covariance (ANCOVA) was used to adjust for baseline measures and to provide an unbiased estimate of the mean group difference of the 1-year postoperative knee flexion scores in knee arthroplasty patients. Robustness tests were done by comparing ANCOVA with three comparative methods: the posttreatment scores, change in scores, and percentage change from baseline.

Results

All four methods showed similar direction of effect; however, ANCOVA (−3.9; 95% confidence interval [CI]: −9.5, 1.6; P=0.15) and the posttreatment score (−4.3; 95% CI: −9.8, 1.2; P=0.12) method provided the highest precision of estimate compared with the change score (−3.0; 95% CI: −9.9, 3.8; P=0.38) and percent change (−0.019; 95% CI: −0.087, 0.050; P=0.58).

Conclusion

ANCOVA, through both simulation and empirical studies, provides the best statistical estimation for analyzing continuous outcomes requiring covariate adjustment. Our empirical findings support the use of ANCOVA as an optimal method in both design and analysis of trials with a continuous primary outcome.

Unbalanced baseline in school-based interventions to prevent obesity: adjustment can lead to bias - a systematic review

BACKGROUND/AIMS

Cluster designs favor unbalanced baseline measures. The aim of the present study was to determine the frequency of unbalanced baseline BMI on school-based randomized controlled trials (RCT) aimed at obesity reduction and to evaluate the analysis strategies. We hypothesized that the adjustment of unbalanced baseline measures may explain the great discrepancy among studies.

METHODS

The source of data was the Medline database content from January 1995 until May 2012. Our search strategy combined key words related to school-based interventions with such related to weight and was not limited by language. The participants' ages were restricted to 6-18 years.

RESULTS

We identified 146 school-based studies on obesity prevention (or overweight or excessive weight change). Of the 146 studies, 36 were retained for the analysis after excluding reviews, feasibility studies, other outcomes, and repeated publications. 13 (35%) of the reviewed studies had statistically significant (p < 0.05) unbalanced measures of BMI at baseline. 11 studies with BMI balanced at baseline adjusted for the baseline BMI, whereas no baseline adjustment was applied to the 5 unbalanced studies.

CONCLUSION

Adjustment for the baseline BMI is frequently done in cluster randomized studies, and there is no standardization for this procedure. Thus, procedures that disentangle the effects of group, time and changes in time, such as mixed effects models, should be used as standard methods in school-based studies on the prevention of weight gain.

Assessing the Relationship between the Baseline Value of a Continuous Variable and Subsequent Change Over Time

Analyzing the relationship between the baseline value and subsequent change of a continuous variable is a frequent matter of inquiry in cohort studies. These analyses are surprisingly complex, particularly if only two waves of data are available. It is unclear for non-biostatisticians where the complexity of this analysis lies and which statistical method is adequate. With the help of simulated longitudinal data of body mass index in children, we review statistical methods for the analysis of the association between the baseline value and subsequent change, assuming linear growth with time. Key issues in such analyses are mathematical coupling, measurement error, variability of change between individuals, and regression to the mean. Ideally, it is better to rely on multiple repeated measurements at different times and a linear random effects model is a standard approach if more than two waves of data are available. If only two waves of data are available, our simulations show that Blomqvist’s method – which consists in adjusting for measurement error variance the estimated regression coefficient of observed change on baseline value – provides accurate estimates. The adequacy of the methods to assess the relationship between the baseline value and subsequent change depends on the number of data waves, the availability of information on measurement error, and the variability of change between individuals.

Effectiveness of a randomized school-based intervention involving families and teachers to prevent excessive weight gain among adolescents in Brazil

Objective

To evaluate the effectiveness of a school-based intervention involving the families and teachers that aimed to promote healthy eating habits in adolescents; the ultimate aim of the intervention was to reduce the increase in body mass index (BMI) of the students.

Design

Paired cluster randomized school-based trial conducted with a sample of fifth graders.

Setting

Twenty classes were randomly assigned into either an intervention group or a control group.

Participants

From a total of 574 eligible students, 559 students participated in the study (intervention: 10 classes with 277 participants; control: 10 classes with 282 participants). The mean age of students was 11 years.

Intervention

Students attended 9 nutritional education sessions during the 2010 academic year. Parents/guardians and teachers received information on the same subjects.

Main Outcome Measurement

Changes in BMI and percentage of body fat.

Results

Intention-to-treat analysis showed that changes in BMI were not significantly different between the 2 groups (β = 0.003; p = 0.75). There was a major reduction in the consumption of sugar-sweetened beverages and cookies in the intervention group; students in this group also consumed more fruits.

Conclusion

Encouraging the adoption of healthy eating habits promoted important changes in the adolescent diet, but this did not lead to a reduction in BMI gain. Strategies based exclusively on the quality of diet may not reduce weight gain among adolescents.

Trial Registration

Clinicaltrials.gov NCT01046474.

A randomized control trial of integrated care for families managing infant colic

This article presents a randomized clinical trial examining the effectiveness of a unique model of integrated care for the treatment of infant colic. Families seeking help for infant colic were randomized to either the family-centered treatment (TX; n = 31) or standard pediatric care (SC; n = 31). All parents completed 3 days of Infant Behavior Diaries (Barr et al., 1998) and the Colic Symptom Checklist (Lester, 1997), Beck Depression Inventory (Beck & Steer, 1984), and Parenting Stress Index 3rd ed.-SF (Abidin, 1995). TX families were seen three times by a pediatrician and a mental health clinician within 1, 2, and 6 weeks of baseline data. TX families received individualized treatment plans addressing problem areas of sleep, feeding, routine, and family mental health. SC families were seen only by their own healthcare provider. All families were visited at home by a research assistant to retrieve data at 2, 6, and 10 weeks after baseline. Family-based treatment accelerated the rate of reduction of infant crying faster than did standard pediatric care. Infants in the TX group had more hours of sleep at 2 weeks posttreatment and spent less time feeding at 2, 6, and 10 weeks posttreatment than did SC infants. Results indicate that individualized family-based treatment reduces infant colic more rapidly than does standard pediatric care.

When should one adjust for measurement error in baseline variables in observational studies?

Previously, we showed that in randomised experiments, correction for measurement error in a baseline variable induces bias in the estimated treatment effect, and conversely that ignoring measurement error avoids bias. In observational studies, non-zero baseline covariate differences between treatment groups may be anticipated. Using a graphical approach, we argue intuitively that if baseline differences are large, failing to correct for measurement error leads to a biased estimate of the treatment effect. In contrast, correction eliminates bias if the true and observed baseline differences are equal. If this equality is not satisfied, the corrected estimator is also biased, but typically less so than the uncorrected estimator. Contrasting these findings, we conclude that there must be a threshold for the true baseline difference, above which correction is worthwhile. We derive expressions for the bias of the corrected and uncorrected estimators, as functions of the correlation of the baseline variable with the study outcome, its reliability, the true baseline difference, and the sample sizes. Comparison of these expressions defines a theoretical decision threshold about whether to correct for measurement error. The results show that correction is usually preferred in large studies, and also in small studies with moderate baseline differences. If the group sample sizes are very disparate, correction is less advantageous. If the equivalent balanced sample size is less than about 25 per group, one should correct for measurement error if the true baseline difference is expected to exceed 0.2-0.3 standard deviation units. These results are illustrated with data from a cohort study of atherosclerosis.

A Bayesian measurement error approach to QT interval correction and prolongation

Assessment of QT interval prolongation is an integral part of clinical studies in drug development because a prolonged QT interval can cause sudden cardiac death. Traditionally a linear or non-linear regression method is applied to estimate subject- or group-specific heart rate corrected QT intervals (QTc) on which comparisons are based among treatment groups. These regression models rely on a fundamental assumption that the predictor variable (RR interval) is measured without error. However, the fact is that both QT and RR intervals measured in electrocardiogram (ECG) are subject to not only measurement error, but also fluctuation that is caused by physiological and biological factors. Hence the assumption in the regression models is most likely violated. In this paper we propose a Bayesian hierarchical measurement error model to evaluate QTc interval and prolongation. The proposed approach is illustrated using a real data set. Simulation studies show that our proposed Bayesian measurement error approach outperforms the current most commonly used frequentist methods.

Should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials?

In randomized clinical trials, a pre-treatment measurement is often taken at baseline, and post-treatment effects are measured at several time points post-baseline, say t=1, ..., T. At the end of the trial, it is of interest to assess the treatment effect based on the mean change from baseline at the last time point T. We consider statistical methods for (i) a point estimate and 95 per cent confidence interval for the mean change from baseline at time T for each treatment group, and (ii) a p-value and 95 per cent confidence interval for the between-group difference in the mean change from baseline. The manner in which the baseline responses are used in the analysis influences both the accuracy and the efficiency of items (i) and (ii). In this paper, we will consider the ANCOVA approach with change from baseline as a dependent variable and compare that with a constrained longitudinal data analysis (cLDA) model proposed by Liang and Zeger (Sankhya: Indian J. Stat. (Ser B) 2000; 62:134-148), in which the baseline is modeled as a dependent variable in conjunction with the constraint of a common baseline mean across the treatment groups. Some drawbacks of the ANCOVA model and potential advantages of the cLDA approach are discussed and illustrated using numerical simulations.