Selected statistical issues in group randomized trials

Group randomized trials (GRTs) in public health research typically use a small number of randomized groups with a relatively large number of participants per group. Two fundamental features characterize GRTs: a positive correlation of outcomes within a group, and the small number of groups. Appropriate consideration of these fundamental features is essential for design and analysis. This paper presents the fundamental features of GRTs and the importance of considering these features in design and analysis. It also reviews and contrasts the main analytic methods proposed for GRTs, emphasizing the assumptions required to make these methods valid and efficient. Also discussed are various design issues, along with guidelines for choosing among them. A real data example illustrates these issues and methods.

Seattle 5 a Day worksite program to increase fruit and vegetable consumption

BACKGROUND

5 a Day for Better Health is a simple message encouraging people to eat more fruits and vegetables. The Seattle 5 a Day worksite investigators designed and evaluated an intervention, organized on stages of behavioral change, to increase worksitewide fruit and vegetable consumption.

METHODS

We recruited 28 worksites with cafeterias and randomized 14 to intervention and 14 to control. The intervention addressed both changes in the work environment and individual level behavior change. In each worksite, an employee advisory board, with study interventionist assistance, implemented the program. By surveying cross-sectional samples of 125 employees per worksite, we compared worksite mean fruit and vegetable consumption at 2-year follow-up with that at baseline. Unobtrusive site-level indicators including plate observation and cafeteria checklist were also used.

RESULTS

The difference at 2 years was 0.5 for the intervention worksites and 0.2 for the control worksites, with an intervention effect of 0.3 daily serving (P < 0.05). Other measures of fruit and vegetable consumption, including unobtrusive indicators, supported the effectiveness of the intervention.

CONCLUSIONS

This simple 5 a Day intervention is feasible and acceptable for use in worksites with cafeterias. There was a significant differential increase in fruit and vegetable consumption in the intervention worksites. This kind of worksite intervention can achieve important health benefits on a population basis, because of its potential to reach large numbers of people.

Precision and bias of food frequency-based measures of fruit and vegetable intakes

Accurate assessment of fruit and vegetable intakes is critical for cancer control research and public health surveillance. This report examines the bias and precision of two commonly used food frequency questionnaire methods to assess fruit and vegetable intakes: (a) the 5 A Day method, based on seven items; and (b) the summation method, based on adding total servings of all fruit and vegetable items on a comprehensive (100+ item) food frequency questionnaire. Data are from three studies in which 24-h dietary recalls, food records, or serum carotenoid concentrations could be used as criterion measures (n = 260, 1031, and 342). Studies differed markedly in distributions of participants' age, race/ethnicity, sex, and socioeconomic status. Mean intakes of total fruit and vegetables based on the 5 A Day method were consistently lower than those from either the summation method (3.11 versus 4.06), 24-h recalls (3.32 versus 4.07), or food records (3.11 versus 3.46; all P < 0.01), and this was due primarily to underestimates of vegetable intake. Correlations of the 5 A Day and summation measures with all criterion measures were similar and were consistently higher for fruit (range, 0.33-0.57) than for vegetables (range, 0.24-0.32). These results, which were consistent across diverse participant samples, suggest that the 5 A Day method yields both biased and imprecise measures of vegetable intake and that research to improve this measure is needed.

The relationship between organizational characteristics and the adoption of workplace smoking policies

Organizational-level variables that are hypothesized to influence the level of smoking policy restrictions and the prevalence of smoking control activities were tested in a sample of 1 14 worksites that participated in the Working Well Trial, a national trial of worksite health promotion. Predictors related to more restrictive policies included smaller size, larger percentage of white-collar workers, larger number of complaints about environmental tobacco smoke, less complexity, more formalization, and having a CEO who valued health and employees' well-being. The number of smoking control activities offered in a worksite was predicted by having a larger blue-collar workforce, a higher percentage of female employees, higher levels of workforce stability, and a CEO who valued health and employees' well-being. Efforts to identify predictors of companies' adoption and implementation of workplace-based policies and interventions are an important part of tobacco control efforts and will enhance future intervention and research efforts.

Seattle 5-a-Day Work-Site Project: process evaluation

The Seattle 5-a-Day Work-Site Project developed a community-based intervention to increase fruit and vegetable intake, using both environmental (including cafeteria and work-site-wide events) and individual strategies. The Employee Advisory Board developed its own protocol from a common skeleton and a minimum set of activities. Small work sites and work sites with fewer female employees delivered more displays, posters, and table tents per employee (p < .01 and p < .05, respectively). Dose was neither related to use of the intervention nor to change in fruit and vegetable intake. Use of informational materials increased fruit and vegetable intake in the cohort of employees with both baseline and follow-up data (p = .05). The intervention was associated both with increased employee use of the intervention (activities and materials) and with increased intake of fruit and vegetables. Work sites with medium average baseline intake were the most responsive. These findings can guide the development of more efficient community-based dietary interventions.

Validity of short food frequency questionnaires used in cancer chemoprevention trials: results from the Prostate Cancer Prevention Trial

Here, we describe the measurement characteristics of a 13-item dietary screener used in the Prostate Cancer Prevention Trial. We used data from 10,913 men who completed the 13-item dietary screener, a food frequency questionnaire (FFQ), and questionnaires on demographic and health-related characteristics and from 146 men who also completed multiple 24-h dietary recalls in a substudy. The analyses in this report focused on percentage energy from fat and saturated fat and used the mean estimates from the dietary recalls as the criterion measures. Absolute nutrient estimates from the screener were about one-third of the estimates from the recalls and the FFQ. Validity was defined as the Pearson correlation of the criterion measures of fat with the corresponding measures from the FFQ and the screener. The FFQ was a statistically significantly more precise measure of percentage energy from fat (r = 0.71) and saturated fat (r = 0.72) than was the screener (r = 0.50 and 0.53, respectively). There were also statistically significant differences in how well these instruments could detect variation in dietary fat across various participant characteristics, suggesting that the screener may not perform as well as the FFQ across demographic strata such as education (P < 0.001). The results from this study suggest that the use of short dietary screeners as the sole assessment instrument may result in a serious loss of information regarding important exposures (e.g., fat intake) and lost opportunities to enhance our knowledge regarding dietary factors and cancer risk.

Impact of the Working Well Trial on the worksite smoking and nutrition environment

This article reports the effect of a worksite cancer control intervention on aspects of the physical and social environment related to dietary and smoking behaviors of employees. Data are from 111 intervention and control worksites that participated in the Working Well Trial. Employee surveys and interviews with key organizational informants assessed environmental and normative changes relevant to nutrition and tobacco use. Results indicated significant effects of the intervention on all nutrition outcomes: access to healthy food, nutritional information at work, and social norms regarding dietary choice. Significant benefits were not found for smoking norms or smoking policies. However, changes occurred in both the control and intervention sites on these variables. This first large analysis of environmental and normative effects of a worksite intervention is consistent with the employee behavior change findings for the trial and serves as a model for future analyses of multilevel worksite health promotion programs.

Validity of methods used to assess vitamin and mineral supplement use

Assessing vitamin and mineral supplement use is important because supplement use per se is an exposure of interest for the risk of several chronic diseases and because supplements contribute a large proportion of total (diet plus supplement) micronutrient intake, another important exposure in epidemiologic research. Unfortunately, little is known about methods for obtaining valid information about supplement use. The authors conducted a validation study in 1996 comparing supplement data collected in a telephone interview and from a self-administered questionnaire with data derived from a detailed, in-person interview and transcription of the labels of supplement bottles (i.e., a gold standard) among adult supplement users in Washington State (n = 104). Spearman correlation coefficients comparing average daily supplemental vitamin and mineral intake from the interview or questionnaire with the gold standard ranged from 0.76 (95% confidence interval 0.66-0.83) for vitamin C to 0.08 (95% confidence interval -0.14 to 0.29) for iron, with a mean of about 0.5. The principal sources of error were inaccurate assumptions about the micronutrient composition of multiple vitamins and respondent confusion regarding the distinction between multiple vitamins and single supplements. These results suggest that commonly used epidemiologic methods of assessing supplement use may incorporate significant amounts of error in estimates of some nutrients.

A comparison of statistical methods for clustered data analysis with Gaussian error

We investigate by simulation the properties of four different estimation procedures under a linear model for correlated data with Gaussian error: maximum likelihood based on the normal mixed linear model; generalized estimating equations; a four-stage method, and a bootstrap method that resamples clusters rather than individuals. We pay special attention to the group randomized trials where the number of independent clusters is small, cluster sizes are big, and the correlation within the cluster is weak. We show that for balanced and near balanced data when the number of independent clusters is small (< or = 10), the bootstrap is superior if analysts do not want to impose strong distribution and covariance structure assumptions. Otherwise, ML and four-stage methods are slightly better. All four methods perform well when the number of independent clusters reaches 50.

Community organization to promote breast cancer screening ordering by primary care physicians

Community organization has been viewed as a promising approach to changing preventive behaviors. We evaluated the impact of community organization strategies to promote breast cancer screening ordering by primary care physicians in Washington State. Physicians practicing in two intervention and two control communities were surveyed by mail pre-intervention (1989) and post-intervention (1993). Intervention activities targeting the health care sector included the formation of local physician planning groups, a series of informational mailings, medical office staff training sessions, and reminder system support. There were no significant post-intervention differences in the self-reported mammography ordering of physicians practicing in the intervention and control areas. Over the four-year study period, the proportions of physicians who ordered regular mammography increased by 36%. By 1993, over 80% of the respondents routinely used mammographic screening. Concerns about the high price of mammograms and inadequate insurance coverage were significantly reduced over time in both community pairs. Also, use of patient reminder systems increased significantly between 1989 and 1993. Secular trends resulting from diffusion of strategies to promote mammography were responsible for increases in physician ordering of the procedure. Year 2000 goals for breast cancer screening use by physicians may already have been met in some communities.

Using administrative data to describe casemix: a comparison with the medical record

We compared the coding of comorbid conditions in an administrative database to that found in medical records for 485 men who had undergone a prostatectomy. Only a few specific conditions showed good agreement between charts and claims. Most showed poor agreement and appeared more frequently in the chart. A comorbidity index calculated from each of these sources was used to explore the differences in mortality for patients who had undergone transurethral vs open prostatectomy. The claims-based comorbidity index most often underestimated the index from the chart. Proportional hazards analysis showed that models including either comorbidity index were better than those without an index and models with information from both indices were best. No analysis eliminated the effect of type of prostatectomy on long-term mortality. Claims-based measures of comorbidity tend to underrepresent some conditions but may be an acceptable first step in controlling for differences across patient populations.

Internal validation of Medicare claims data

The Medicare database is commonly used for epidemiology and health services research, but validation of its data by chart review or questionnaire may be difficult and expensive. Since hospital and physician bills are independent in Medicare, however, these two data sources can be used to supplement and corroborate each other. This "internal validation" is illustrated here for hip fracture and prostatectomy. Agreement of the hospital and physician data streams regarding site of hip fracture (neck vs other), treatment of hip fracture (internal fixation vs arthroplasty), and type of prostatectomy (transurethral resection of prostate, open, or radical) was excellent, with percentage of agreement generally between 89% and 99%, and kappa statistics typically between 0.74 and 0.95. When validation with outside data sources is not readily available, such internal validation of Medicare data may be valuable.

An assessment of radical prostatectomy. Time trends, geographic variation, and outcomes. The Prostate Patient Outcomes Research Team

OBJECTIVES

To examine temporal trends and geographic variation in radical prostatectomy rates and short-term outcomes.

DESIGN

Population-based study of radical prostatectomy for the years 1984 through 1990. Poisson regression was used to estimate temporal and regional effects.

SETTING

The 50 states and the District of Columbia.

PARTICIPANTS

A 20% national sample of male Medicare beneficiaries aged 65 years or older.

MAIN OUTCOME MEASURES

Rate of radical prostatectomy; 30-day mortality; and major cardiopulmonary complications, vascular complications, or surgical repairs within 30 days of radical prostatectomy.

RESULTS

A total of 10,598 radical prostatectomies were identified. The adjusted rate of radical prostatectomy in 1990 was 5.75 times that in 1984. The relative increase was similar in all age groups. Substantial geographic variation existed in rates from 1988 through 1990: all states in the New England and Mid-Atlantic regions had rates equal to or below 60 per 100,000 male Medicare beneficiaries, while all states in the Pacific and Mountain regions had rates equal to or above 130 per 100,000. The mortality and morbidity after radical prostatectomy are not trivial for older men (aged 75 years and older)--almost 2% died and nearly 8% suffered major cardiopulmonary complications within 30 days of the operation.

CONCLUSION

The sharp increase and wide geographic variation in radical prostatectomy rates make the evaluation of this surgical procedure a pressing issue. The rising rate of radical prostatectomy among men aged 75 years and older merits special attention.

Further study of the increased mortality following transurethral prostatectomy: a chart-based analysis

Previous studies using large administrative databases found an elevated relative risk of reoperation and death after transurethral resection of the prostate compared to open prostatectomy. To investigate whether differences in case-mix unmeasured by administrative data explained this finding, we reviewed the charts of 485 patients who had undergone prostatectomy (236 open and 249 transurethral) at the Health Science Centre, Winnipeg, Manitoba, Canada between 1974 and 1980. Data from patient histories, physical examinations and laboratory evaluations were abstracted and used to control for case-mix in models comparing the rates of reoperation and mortality after transurethral versus open prostatectomy. Several models were specified. In all models the relative risk of dying after transurethral prostatectomy remained elevated (1.36 to 1.89), as did the risk for reoperation (3.62). A prospective trial is needed to establish the relative safety and effectiveness of transurethral and open prostatectomy.