Multiple imputation in a large-scale complex survey: a practical guide

The Cancer Care Outcomes Research and Surveillance (CanCORS) Consortium is a multisite, multimode, multiwave study of the quality and patterns of care delivered to population-based cohorts of newly diagnosed patients with lung and colorectal cancer. As is typical in observational studies, missing data are a serious concern for CanCORS, following complicated patterns that impose severe challenges to the consortium investigators. Despite the popularity of multiple imputation of missing data, its acceptance and application still lag in large-scale studies with complicated data sets such as CanCORS. We use sequential regression multiple imputation, implemented in public-available software, to deal with non-response in the CanCORS surveys and construct a centralised completed database that can be easily used by investigators from multiple sites. Our work illustrates the feasibility of multiple imputation in a large-scale multiobjective survey, showing its capacity to handle complex missing data. We present the implementation process in detail as an example for practitioners and discuss some of the challenging issues which need further research.

The treating physician as active gatekeeper in the recruitment of research subjects

BACKGROUND

Institutional Review Boards vary in regard to the conditions imposed on investigators concerning contacting potential subjects to participate in health-services research studies.

OBJECTIVE

The impact of more active involvement of the treating physician was examined in the approval process for recruiting study subjects.

DESIGN

In recruiting subjects for a Massachusetts-based, multihospital (n = 17), health-services research study of treatment patterns for early stage breast cancer that required patient interviews, four hospitals stipulated that the treating surgeon provide written permission to the investigators to allow any contact with a potential study subject for the purpose of recruitment (active physician involvement group); the remaining 13 hospitals stipulated that the treating surgeon need only respond to the investigators if contact with a potential subject was forbidden (passive physician involvement group).

SUBJECTS

Of the 1401 potential subjects treated for early stage breast cancer, 697 were in the active physician involvement group and 704 were in the passive physician involvement group.

MEASURES

The percentages of patients for whom contact was allowed for recruitment purposes and who enrolled in the study were determined for the active physician involvement group and the passive physician involvement group, respectively. Logistic regression models were used to assess the independent effect of physician involvement on study enrollment.

RESULTS

Of the 697 patients in the active physician involvement group, contact was approved by the treating surgeon for 72% (n = 505), compared with 91% (n = 638) of the passive physician involvement group (P <0.001). After adjustment for a variety of patient, physician, and hospital-level variables, patients in the passive physician involvement group were found to be significantly more likely to be enrolled in the study (adjusted OR 2.61; 95% CI, 1.53-4.45). However, among those patients approved for investigator contact, there were no significant differences between patients who were enrolled and patients who were not enrolled in the study with regard to physician involvement in the recruitment process (adjusted OR 1.13; 95% CI, 0.70-1.81).

CONCLUSION

Our findings demonstrate that more stringent IRB requirements on health services researchers to verify permission from the treating physician to access patients for recruitment purposes adversely impact on the enrollment of patients even in noninterventional research studies. Current procedures for involving the treating physician as a gatekeeper in the recruitment of research subjects may limit access to patient participation in research studies from the perspectives of both researchers and potential subjects.

A multiple imputation strategy for incomplete longitudinal data

Longitudinal studies are commonly used to study processes of change. Because data are collected over time, missing data are pervasive in longitudinal studies, and complete ascertainment of all variables is rare. In this paper a new imputation strategy for completing longitudinal data sets is proposed. The proposed methodology makes use of shrinkage estimators for pooling information across geographic entities, and of model averaging for pooling predictions across different statistical models. Bayes factors are used to compute weights (probabilities) for a set of models considered to be reasonable for at least some of the units for which imputations must be produced, imputations are produced by draws from the predictive distributions of the missing data, and multiple imputations are used to better reflect selected sources of uncertainty in the imputation process. The imputation strategy is developed within the context of an application to completing incomplete longitudinal variables in the so-called Area Resource File. The proposed procedure is compared with several other imputation procedures in terms of inferences derived with the imputations, and the proposed methodology is demonstrated to provide valid estimates of model parameters when the completed data are analysed. Extensions to other missing data problems in longitudinal studies are straightforward so long as the missing data mechanism can be assumed to be ignorable.

Discussion of treatment options for early-stage breast cancer: effect of provider specialty on type of surgery and satisfaction

OBJECTIVES

To describe the physicians with whom breast cancer patients discuss treatment options and assess whether discussing surgical options with a medical oncologist is associated with type of surgery and satisfaction.

RESEARCH DESIGN

Medical record abstraction and survey.

SUBJECTS

Women with early-stage breast cancer numbering 2,426 in two states-Massachusetts, where the rate of breast-conserving surgery is high, and Minnesota, where it is lower.

MEASURES

Receipt of breast-conserving surgery and satisfaction.

RESULTS

Women in Massachusetts discussed breast cancer treatments with more physicians than women in Minnesota (mean 3.5 vs. 2.8; P <0.001) and more often discussed surgical options with a medical oncologist (52% vs. 28%; P <0.001). Using propensity score analyses, in Massachusetts, discussing surgical options with a medical oncologist was not related to type of surgery (adjusted difference in rate of breast-conserving surgery: 3.9%, 95% CI -3.6% to 11.5%) but was associated with greater satisfaction (adjusted difference: 8.1, 95% CI 2.0% to 14.2%). In Minnesota, discussing surgical options with a medical oncologist was associated with breast-conserving surgery (adjusted difference: 12.6%, 95% CI 5.6% to 19.7%) with no difference in satisfaction (adjusted difference: -1.5%, 95% CI -6.8% to 3.8%).

CONCLUSIONS

Outcomes associated with discussing surgical treatments with a medical oncologist vary with local care patterns. Where breast-conserving surgery is standard care, seeing a medical oncologist is not related to type of surgery, but is associated with greater satisfaction. Where it is not the standard, seeing a medical oncologist is associated with more breast-conserving surgery and equivalent satisfaction. These findings suggest that collaborative care may benefit women with respect to treatment selection or satisfaction.

Quality of ambulatory care after myocardial infarction among Medicare patients by type of insurance and region

PURPOSE

To evaluate use of effective cardiac medications and rehabilitation after myocardial infarction in the ambulatory setting in health maintenance organizations (HMOs) and fee-for-service care, and by region.

SUBJECTS AND METHODS

We surveyed elderly Medicare patients during 1996 and 1997 in California (n = 516), Florida (n = 304), and the Northeast (n = 220; Massachusetts, New York, and Pennsylvania) approximately 18 months after myocardial infarction. We assessed use of cardiac medications and rehabilitation for HMO (n = 520) and fee-for-service (n = 520) patients matched by age, sex, month of infarct, and region.

RESULTS

Across all regions, similar proportions of HMO and fee-for-service patients were using aspirin (72%, n = 374 vs. 74%, n = 387), beta-blockers (38%, n = 195 vs. 32%, n = 168), angiotensin-converting enzyme inhibitors (31%, n = 159 vs. 29%, n = 148), cholesterol-lowering agents (28%, n = 146 vs. 30%, n = 157), and calcium channel blockers (31%, n = 162 vs. 31%, n = 159; all P >0.07), except in California where more HMO patients received beta-blockers (36%, n = 93 vs. 26%, n = 66, P = 0.01). In adjusted analyses, use of these drugs did not differ significantly between HMO and fee-for-service patients. Substantial regional differences were evident in the use of beta-blockers (Northeast 46%, n = 102; Florida 34%, n = 102; California 31%, n = 159) and cholesterol-lowering agents (California 35%, n = 182; Florida 24%, n = 73; Northeast 22%, n = 48; each P <0.001). Fee-for-service patients were more likely than HMO patients to receive cardiac rehabilitation in unadjusted (32%, n = 167, vs. 22%, n = 141, P = 0.001) and adjusted analyses.

CONCLUSIONS

Both HMO and fee-for-service patients would likely benefit from greater use of beta-blockers and cholesterol-lowering agents. Professional fees for cardiac rehabilitation may promote increased use among fee-for-service patients. Future studies should assess the quality of ambulatory cardiac care in different types of HMOs and the reasons for geographic variations in cardiac drug use.

Impact of underuse, overuse, and discretionary use on geographic variation in the use of coronary angiography after acute myocardial infarction

BACKGROUND

Geographic variation in the use of medical procedures has been well documented. However, it is not known whether this variation is due to differences in use when procedures are indicated, discretionary, or contraindicated.

OBJECTIVES

To examine whether use of coronary angiography after acute myocardial infarction (AMI) according to appropriateness criteria varied across geographic regions and whether underuse, overuse, or discretionary use accounted for variation in overall use.

DESIGN

Retrospective cohort study using data from the Cooperative Cardiovascular Project.

SETTING

Ninety-five hospital referral regions.

PATIENTS

There were 44,294 Medicare patients hospitalized with AMI during 1994 or 1995, classified according to appropriateness for angiography.

MAIN OUTCOME MEASURE

Variation in use of angiography, as measured by the difference between high and low rates of use across regions.

RESULTS

Across regions, variation in the use of angiography was similar for indications judged necessary; appropriate, but not necessary; or uncertain. Variation was lowest for indications judged unsuitable (difference between high rate and low rate across regions = 16.3%; 95% CI = 12.6%; 20.6%). The primary cause of variation in the overall rate of angiography was due to use for indications judged appropriate, but not necessary or uncertain. When variation associated with these indications was accounted for, the difference between the resulting high and low overall rates was 10.8% (9.4%, 12.4%). In contrast, variation in the overall rate remained high when underuse in necessary situations or overuse in unsuitable situations was accounted for.

CONCLUSIONS

Across regions, practice was more similar for patients categorized unsuitable for angiography than for patients with other indications. Variation in overall use of angiography appeared to be driven by utilization for discretionary indications rather than by underuse or overuse. If equivalent rates across geographic areas are judged desirable, then greater effort must be directed toward defining care for patients with discretionary indications.

Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores

We determined whether adherence to recommendations for coronary angiography more than 12 h after symptom onset but prior to hospital discharge after acute myocardial infarction (AMI) resulted in better survival. Using propensity scores, we created a matched retrospective sample of 19,568 Medicare patients hospitalized with AMI during 1994-1995 in the United States. Twenty-nine percent, 36%, and 34% of patients were judged necessary, appropriate, or uncertain, respectively, for angiography while 60% of those judged necessary received the procedure during the hospitalization. The 3-year survival benefit was largest for patients rated necessary [mean survival difference (95% CI): 17.6% (15.1, 20.1)] and smallest for those rated uncertain [8.8% (6.8, 10.7)]. Angiography recommendations appear to select patients who are likely to benefit from the procedure and the consequent interventions. Because of the magnitude of the benefit and of the number of patients involved, steps should be taken to replicate these findings.

Appropriateness of coronary angiography after myocardial infarction among Medicare beneficiaries. Managed care versus fee for service

BACKGROUND

Previous studies have documented that cardiac procedures are performed less frequently in patients enrolled in managed-care plans than in those with fee-for-service coverage. However, it is not known whether this difference is due to less frequent use of cardiac procedures when they are indicated or to less frequent use when they are not indicated.

METHODS

We compared the use of coronary angiography after acute myocardial infarction among Medicare beneficiaries who had traditional fee-for-service coverage with the use among Medicare beneficiaries enrolled in managed-care plans. The analysis was adjusted for differences in demographic and clinical characteristics of the patients and for characteristics of the hospitals to which they were admitted. We studied more than 50,000 beneficiaries in seven states and evaluated their care according to guidelines proposed by the American College of Cardiology and the American Heart Association (ACC-AHA).

RESULTS

Among the 44 percent of patients in both groups who had ACC-AHA class I indications (those for which angiography is useful and effective), more fee-for-service beneficiaries than managed-care enrollees underwent angiography (46 percent vs. 37 percent, P<0.001). The rate of angiography was very low among patients with class I indications who were admitted to hospitals without angiography facilities (31 percent in the fee-for-service group and 15 percent in the managed-care group, P<0.001). Among patients with class III indications (those for which angiography is not effective), the rate of use was low in both groups (approximately 13 percent).

CONCLUSIONS

In situations in which angiography is thought to be useful, it is used less often among Medicare beneficiaries enrolled in managed-care plans than among those with fee-for-service coverage. Moreover, rates of use among patients with class I indications are fairly low in both groups, suggesting that there is room for improving the care of elderly patients with myocardial infarction, especially those admitted to hospitals without angiography facilities.

Understanding variability in physician ratings of the appropriateness of coronary angiography after acute myocardial infarction

We examined variability in ratings of the appropriateness of coronary angiography for 890 clinical scenarios (indications) after an acute myocardial infarction (AMI) from a nine-member multispecialty panel as a function of panel characteristics and the attributes of the clinical indications. We documented a substantial degree of reliability in the ratings. However, key differences among the experts in terms of both their overall propensity to score high and their beliefs regarding the impact of clinical factors on appropriateness were identified. Age, cardiac complications, post-AMI angina, and noninvasive test results were the clinical factors most strongly related to appropriateness ratings for coronary angiography. Further research on the effectiveness of coronary angiography in older patients and in patients with shock, pulmonary edema, and silent ischemia is needed to improve our knowledge about the appropriateness of this procedure in these patients.

Applying Bayesian ideas to the development of medical guidelines

Measurements of the quality of health care, in particular the underuse and overuse of medical therapies and diagnostic tests, often involve employment of medical practice guidelines to assess the appropriateness of treatments. This paper presents a case study of a Bayesian analysis for the development of medical guidelines based on expert opinion, using ordinal categorical rater data. We develop guidelines for the use of coronary angiography following an acute myocardial infarction (AMI) for 890 clinical indications using statistical models fit to appropriateness ratings obtained from a nine-member expert panel. The main foci of our analyses were on the estimation of an appropriateness score for each of the clinical indications, an associated measure of precision, and functions of the underlying score. We considered two classes of models that assume the ratings are either in the form of grouped normal data or are ungrouped variables arising from a normal distribution, while permitting rater effects and indication heterogeneity in both. We estimated models using Markov chain Monte Carlo methods and constructed indices quantifying appropriateness based on posterior probabilities of selected model parameters. We compared our model-based approach to the standard approach currently employed in medical guideline development and found that the standard approach correctly identified 99 per cent of the appropriate indications while overestimating appropriateness 18 per cent of the time compared to our model-based approach.

Rating the appropriateness of coronary angiography--do practicing physicians agree with an expert panel and with each other?

BACKGROUND

Evaluations of the appropriateness of medical care are important to monitor the quality of care and to contain costs and enhance safety by reducing inappropriate care. Experts' views are usually incorporated into evaluations of appropriateness. However, practicing physicians may not concur with these views, and physicians' clinical backgrounds may influence their beliefs.

METHODS

We asked 1058 internists, family practitioners, and cardiologists in California, Florida, New York, Pennsylvania, and Texas to rate the appropriateness of coronary angiography after acute myocardial infarction for 20 common indications. Nine clinical experts also rated these indications using an established consensus method.

RESULTS

For 17 of the 20 indications, median ratings of surveyed physicians and the expert panel agreed within 1 unit on a 9-unit scale. Patients' older age had a negative effect on ratings by the expert panel but not on ratings by surveyed physicians. In multivariable analyses of surveyed physicians, cardiologists rated angiography as significantly more appropriate than did primary care physicians for complicated indications, and for uncomplicated indications cardiologists who performed invasive procedures gave higher appropriateness ratings for angiography than did cardiologists who did not perform such procedures and primary care physicians. For uncomplicated indications, physicians from hospitals providing coronary angioplasty and bypass surgery rated angiography as more appropriate than physicians from other hospitals. Physicians from New York and those employed by health maintenance organizations rated angiography as less appropriate than other physicians.

CONCLUSIONS

Surveyed physicians agreed with clinical experts about the appropriateness of coronary angiography after myocardial infarction for most indications, indicating that well-designed expert panels can closely reflect the views of practicing physicians. Variations in beliefs among practicing physicians suggest that evaluations of medical practice should incorporate the views of a range of relevant types of physicians.