Random sample community-based health surveys: does the effort to reach participants matter?

Interpreting Randomized Controlled Trials

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial's sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial's convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

Process Evaluation of a Farm-to-WIC Intervention

BACKGROUND

Despite the promise of farm-to-institution interventions for addressing limited vegetable access as a barrier to intake, programs designed for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) are lacking. As such, little is known about the implementation of, and mechanisms of action through which, farm-to-WIC interventions affect vegetable intake and participant satisfaction with such programs.

OBJECTIVE

To examine whether a farm-to-WIC intervention to promote vegetable intake was implemented as intended, differences between participants who received the intervention relative to those in a usual-care control group in intermediate outcomes of vegetable-related knowledge, attitudes, and behaviors, and secondary outcomes of physical activity and weight status; and participant satisfaction with the intervention.

DESIGN

A process evaluation encompassing descriptive and comparative analyses of implementation fidelity logs and survey data collected as part of a pilot study was conducted.

PARTICIPANTS/SETTING

The setting was a large, New Jersey-based, urban WIC agency. Recruited between June 3 and August 1, 2019 through 3 of the agency's 17 sites (1 intervention and 2 control sites), participants were 297 primarily Hispanic adults (160 enrolled at the intervention site and 137 at control sites).

INTERVENTION

The intervention combined behaviorally focused instruction and handouts with the introduction of a WIC-based farmers' market, field trips to an area farmers' market, telephone coaching and support, and recipe demonstrations and tastings.

MAIN OUTCOME MEASURES

Primary outcomes were vegetable intake (measured via self-report and objectively using dermal carotenoids as a biomarker of intake) and the redemption of vouchers provided by WIC for fruit and vegetable purchases at farmers' markets (measured objectively using data provided by WIC). For the process evaluation, logs were used to document program activities. Vegetable-related knowledge, attitudes, and behaviors, physical activity, and satisfaction with the intervention were assessed with participant questionnaires. Weight status was assessed with direct measures of height and weight. Data were collected at baseline and at mid- and post-intervention (3 and 6 months post-baseline, respectively).

STATISTICAL ANALYSES PERFORMED

Descriptive statistics were used to characterize implementation fidelity. Associations between intermediate and secondary outcomes and vegetable intake were examined at baseline with Pearson correlations. Post-baseline between-group differences in the outcomes were examined with linear mixed-effects models adjusted for baseline values and covariates. Satisfaction with the intervention was assessed with inferential and thematic analyses.

RESULTS

Post-intervention, measures of vegetable intake were higher in the intervention relative to the control study group. Receipt of the intervention was also associated with a greater likelihood of voucher redemption. Nearly all participants (≥94%) received the intervention as intended at the WIC-based farmers' market; smaller percentages completed 1 or more planned trips to the area farmers' market (28%) and telephone coaching and support calls (88%). Although most intermediate and secondary outcomes were associated with measures of vegetable intake at baseline, the variables did not differ between study groups post-intervention. Mean satisfaction ratings were ≥6.8 on a 7-point scale. Recipe demonstrations, learning about vegetables, field trips, and the rapport with staff were liked most about the program. Although adding days and times for field trips was suggested, limited market days and hours of operation limited the ability to do so.

CONCLUSIONS

Preliminary data highlight the promise of this well-received intervention. Intermediate outcome findings suggest that other potential intervention mechanisms of action should be considered in future large-scale trials of this program. Broad-scale initiatives are needed to improve access to farmers' markets in underserved communities.

Assessment of Non-Response Bias in Estimates of Alcohol Consumption: Applying the Continuum of Resistance Model in a General Population Survey in England

Background

Previous studies have shown heavier drinkers are less likely to respond to surveys and require extended efforts to recruit. This study applies the continuum of resistance model to explore how survey estimates of alcohol consumption may be affected by non-response bias in three consecutive years of a general population survey in England.

Methods

Using the Health Survey for England (HSE) survey years 2011–13, number of contact attempts (1–6 and 7+) were explored by socio-demographic and drinking characteristics. The odds of drinking more than various thresholds were modelled using logistic regression. Assuming that non-participants were similar to those who were difficult to contact (the continuum of resistance model), the effect of non-response on measures of drinking was investigated.

Results

In the fully-adjusted regression model, women who required 7+ calls were significantly more likely to drink more than the UK Government’s recommended daily limit (OR 1.19, 95% CI 1.06–1.33, P = 0.003) and to engage in heavy episodic drinking (OR 1.23, 95% CI 1.07–1.42, P = 0.004), however this was not significant in men in the fully-adjusted model. When the continuum of resistance model was applied, there was an increase in average weekly alcohol consumption of 1.8 units among men (a 12.6% relative increase), and an increase of 1.5 units among women (a 20.5% relative increase). There was also an increase in the prevalence of heavy episodic drinking of 2.5% among men (an 12.0% relative increase) and of 2.0% among women (a 15.8% relative increase), although other measures of drinking were less affected.

Conclusion

Overall alcohol consumption and the prevalence of heavy episodic drinking were higher among HSE participants who required more extended efforts to contact. The continuum of resistance model suggests non-response bias does affect survey estimates of alcohol consumption.