Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument

BACKGROUND

Many different surveys have been used to evaluate the performance of medical groups, but there is a growing recognition that standardization could be helpful to consumers, purchasers, and others.

OBJECTIVES

We sought to develop a version of the Consumer Assessment of Health Plans Study (CAHPS) survey for use with medical groups (G-CAHPS) and assess its reliability and validity.

RESEARCH DESIGN

The research team reviewed existing instruments and conducted patient focus groups in 4 sites to identify aspects of care that were especially important to patients when evaluating medical groups. We tested a draft instrument in 75 cognitive interviews with adults 18 years of age or older in Knoxville, St. Louis, and California and pretests in 4 groups of adults in Boston and Denver. We then surveyed random samples of patients from medical groups and practice sites in California, Knoxville, St. Louis, and Denver. We analyzed the psychometric characteristics of the instrument.

RESULTS

Data support the reliability and validity of 3 multi-item measures of access, office staff service, and patient clinician communication. Measures related to specialty care and preventive counseling did not differentiate among medical groups.

CONCLUSIONS

The G-CAHPS instrument provides an assessment of selected aspects of care that are important to consumers and could be a useful complement to the plan-level CAHPS instrument.

Adjusting Pediatric Consumer Assessment of Health Plans Study (CAHPS) Scores to Ensure Fair Comparison of Health Plan Performances

OBJECTIVES

When comparing the scores from Consumer Assessment of Health Plans Study (CAHPS) surveys across health plans, it is important to adjust for patient characteristics (casemix) that are not under the control of the plan but that affect plan ratings. Our goal was to develop casemix models for pediatric CAHPS surveys.

METHODS

We analyzed responses to the pediatric CAHPS 2.0 surveys from 50,583 Medicaid beneficiaries and 43,579 persons with private health insurance. We identified patient characteristics with the most substantial impact on scores and assessed casemix models that include different combinations of adjusters. In addition, we tested whether casemix coefficients differed between the Medicaid and commercial samples and across health plans.

RESULTS

Parent age and education and child health status and race were important casemix adjustment variables for pediatric CAHPS surveys. Child age and gender had smaller effects. The relationship between patient characteristics and CAHPS scores was different in the Medicaid and commercial samples for some variables. The effects of a patient's characteristics on ratings and report scores were not consistent across Medicaid plans but more consistent for commercial plans.

CONCLUSIONS

Our analyses indicate that plan scores on the pediatric CAHPS survey should be adjusted for plan differences in casemix. Users should consider estimating separate models for Medicaid and commercially insured respondents. Such models should adjust for child health status, parent age, and parent education.

Plan, geographical, and temporal variation of consumer assessments of ambulatory health care

OBJECTIVE

To quantify contributions of health plans and geography to variation in consumer assessments of health plan quality.

DATA SOURCES

Responses of beneficiaries of Medicare managed care plans to the Consumer Assessment of Health Plans Study (CAHPS(R)) survey. Our data included more than 700,000 survey responses assessing 381 Medicare managed care (MMC) contracts over a period of five years.

STUDY DESIGN

The survey was administered to a nationally representative sample of beneficiaries of Medicare managed care plans.

PRINCIPAL FINDINGS

Member assessments of their health plans, customer service functions, and prescription drug benefits varied most across health plans; these also varied the most over time. Assessments of direct interactions with doctors and their practices were more affected by geographical location, and these assessments were quite stable over time. A health plan's global rating often changed significantly between consecutive years, but only rarely were there such changes in ratings of care or doctor. Nationally, mean assessments tended to decrease over the study period.

CONCLUSIONS

Our findings suggest that ratings of plans and reports about customer service and prescription access are affected by plan policies, benefits design, and administrative structures that can be changed relatively quickly. Conversely, assessments of other aspects of care are largely determined by characteristics of provider networks that are relatively stable. A consumer survey is unlikely to detect meaningful changes in quality of care from year to year unless quality improvement measures are developed that have substantially larger effects, possibly through area-wide initiatives, than historical temporal variations in quality.

Differences in Health-Related Quality of Life and Treatment Preferences Among Black and White Patients with End-Stage Renal Disease

BACKGROUND

Relatively little is known about racial differences in health-related quality of life (HRQL) among patients receiving dialysis for end-stage renal disease (ESRD) or how such differences may relate to preferences for renal transplantation.

METHODS

We surveyed 1392 patients, ages 18-54 approximately 10 months after they initiated dialysis in 4 regions of the United States. The HRQL measures analyzed were overall health, emotional health, physical activity, energy level, social activity, and effect of ESRD on daily life. We also examined whether the measures of HRQL were associated with patients' preferences for renal transplantation by race.

RESULTS

After adjustment for socioeconomic and clinical characteristics, Black women and men reported better overall health than White women and men, respectively. Black women reported higher energy levels than White women, and Black men reported less negative effects of ESRD on daily life compared to White men. Black men with high levels of physical activity were less likely to be certain about preferring a transplant than White men with similar levels of physical activity.

CONCLUSIONS

Black patients receiving dialysis reported better HRQL than White patients, even after controlling for potential confounders. Racial differences in preferences for renal transplantation among men may be associated with their levels of physical activity.

Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study

BACKGROUND

Multi-institution collaborative quality improvement programs are a well-established and broadly applicable quality improvement strategy, but there is little systematic assessment their effectiveness.

OBJECTIVE

To evaluate the effectiveness of a quality improvement collaborative in improving the quality of care for HIV-infected patients.

DESIGN

Controlled pre- and postintervention study.

SETTING

Clinics receiving funding from the Ryan White Comprehensive AIDS Resources Emergency Act.

PARTICIPANTS

44 intervention clinics and 25 control clinics matched by location (urban or rural), region, size, and clinic type.

MEASUREMENTS

Changes in quality-of-care measures abstracted from medical records of pre- and postintervention samples of patients at each study clinic. Measures examined included use and effectiveness of antiretroviral therapy, screening and prophylaxis, and access to care.

INTERVENTION

A multi-institutional quality improvement collaborative (the "Breakthrough Series").

RESULTS

9986 patients were studied. Clinical and sociodemographic characteristics of the intervention and control patients were similar (P > 0.05). Differences in changes in the quality of care were not statistically significant. The proportion of patients with a suppressed viral load increased by 11 percentage points (from 40.1% to 51.1%) in the intervention group compared with 5.3 percentage points (from 43.6% to 48.8%) in the control group, but this difference was not statistically significant (P = 0.18). In addition, rates of appropriate screening tests and prophylaxis did not differ between intervention and control sites.

LIMITATIONS

It was not possible to perform a pure randomized trial of the intervention or to assess other measures of quality, such as adherence and satisfaction.

CONCLUSIONS

This prospective, matched study of almost 10 000 patients found that a quality improvement collaborative did not significantly affect the quality of care. Additional research is needed to improve methods of teaching and implementing quality improvement programs to achieve better results.

Physicians' beliefs about racial differences in referral for renal transplantation

BACKGROUND

Black patients with end-stage renal disease are much less likely than white patients to undergo renal transplantation, but previous research has shown that black patients are only slightly less likely to desire this procedure. A better understanding of physicians' views about racial differences in access to transplantation may help reduce disparities in care.

METHODS

We surveyed 278 nephrologists in 4 US regions about quality of life and survival for black and white patients undergoing renal transplantation and reasons for racial differences in access to transplantation. We also surveyed 606 of their patients about their care.

RESULTS

Physicians were less likely to believe transplantation improves survival for blacks than whites (69% versus 81%; P = 0.001), but similarly likely to believe it improves quality of life (84% versus 86%). Factors commonly cited by physicians as important reasons why blacks are less likely than whites to be evaluated for transplantation included patients' preferences (66%), availability of living donors (66%), failure to complete evaluations (53%), and comorbid illnesses (52%). Fewer physicians perceived patient-physician communication and trust (38%) or physician bias (12%) as important reasons. Black patients were less likely than white patients to report receiving some or a lot of information about transplantation (55% versus 74%; P = 0.006) when their physicians did not view patient-physician communication and trust as an important reason for racial differences in care.

CONCLUSION

Nephrologists' views about the benefits of renal transplantation and reasons for racial differences in access to this procedure may affect how they present this treatment option to black and white patients.

Comparison of performance of traditional Medicare vs Medicare managed care

CONTEXT

Since 2000, the Centers for Medicare & Medicaid Services (CMS) has been collecting information on beneficiaries' experiences with health care for Medicare managed care (MMC) and traditional fee-for-service (FFS) Medicare.

OBJECTIVES

To compare beneficiary experiences with managed care and FFS arrangements throughout the country and to assess the stability of those differences over time.

DESIGN, SETTING, AND PARTICIPANTS

CMS administered managed care and FFS versions of the Consumer Assessment of Health Plans Study (CAHPS) survey to samples of beneficiaries (aged > or =65 years) from Medicare + Choice MMC organizations and from geographic strata within the traditional FFS Medicare program. We analyzed responses collected in 2000 and 2001 from 497,869 respondents: 299,058 beneficiaries enrolled in MMC plans (response rate, 82%) and 198,811 enrolled in FFS Medicare (response rate, 68%). Differences between MMC and FFS within states were assessed after adjustment for case mix and nonresponse. For estimates at the regional and national level, state estimates were combined after weighting by the MMC enrollment in the state.

MAIN OUTCOME MEASURES

Four overall ratings (of the plan, personal physician, care received overall, and care received from specialists), 5 measures summarizing beneficiaries' experiences with care (getting care needed; getting care quickly; communication with clinicians; courtesy and respect of physician's office staff; and paperwork, information, and customer service), and reports of receipt of 3 preventive services (flu shots, pneumococcal vaccinations, and being advised to quit smoking) were assessed.

RESULTS

Respondents in MMC and FFS plans were similar to each other and to the Medicare population as a whole. Nationally, FFS Medicare beneficiaries rated experiences with care measured by the CAHPS survey higher than did MMC beneficiaries; for instance, in ratings of care received overall (scale of 1-10) (8.91 FFS vs 8.86 MMC, P<.001, in 2000; and 8.88 FFS vs 8.78 MMC, P<.001, in 2001). Differences between FFS and MMC varied across states, however. Managed care enrollees reported significantly fewer problems with paperwork, information, and customer service (2.62 FFS vs 2.55 MMC, P<.001, in 2000; and 2.59 FFS vs 2.51 MMC, P<.001, in 2001). Enrollees in MMC were also more likely to report having received immunizations for influenza and pneumococcus (from any source) (in 2000, 77% of MMC vs 63% of FFS respondents; P<.001), and smokers were more likely to report having received counseling to quit smoking.

CONCLUSIONS

Our data suggest that managed care was better at delivering preventive services, whereas traditional Medicare was better in other aspects of care related to access and beneficiary experiences. These relative strengths should be considered when policy decisions are made that affect the availability of choice or influence beneficiaries to choose one model of care over another.

Disparities in HIV treatment and physician attitudes about delaying protease inhibitors for nonadherent patients

BACKGROUND

Current HIV treatment guidelines recommend delaying antiretroviral therapy for nonadherent patients, which some fear may disproportionately affect certain populations and contribute to disparities in care.

OBJECTIVES

To examine the relationship of physician's attitude toward prescribing protease inhibitors (PIs) to nonadherent patients with disparities in PI use and with health outcomes.

DESIGN

Prospective cohort study.

PATIENTS AND SETTING

A national probability sample of HIV-infected adults in the United States and their health care providers was surveyed between January 1996 and January 1998. We analyzed data on 1717 patients eligible for PI treatment and the 367 providers who cared for them.

MEASUREMENTS

Providers' attitude toward prescribing PIs to nonadherent patients, time until patients' first receipt of PIs, mortality, and physical health status.

MAIN RESULTS

Eighty-nine percent of providers agreed that patient adherence is important in their decision to prescribe PIs (Selective) while 11% disagreed (Nonselective). Patients who had a Selective provider received PIs later than those with a Nonselective provider (P =.05). Adjusting for patient demographics and health characteristics and provider demographics, HIV knowledge, and experience, Latinos, women, and poor patients received PIs later if their provider had a Selective attitude but as soon as others if their provider had a Nonselective attitude. African-American patients received PIs later than whites, irrespective of their providers' prescribing attitude. Patients with Selective providers had similar odds of mortality than those with Nonselective providers (odds ratio, 1.1; 95% confidence interval, 0.6 to 2.0), but had slightly worse adjusted physical health status at follow-up (49.1 vs 50.4, respectively; P =.04), after controlling for baseline physical health status and other patient and provider covariates.

CONCLUSIONS

Most providers consider patient adherence an important factor in their decision to prescribe PIs. This attitude appears to account for the relatively later use of PI treatment among Latinos, women, and the poor. Given the rising HIV infection rates among minorities, women, and the poor, further investigation of this treatment strategy and its impact on HIV resistance and outcomes is warranted.

Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey

OBJECTIVE

To estimate the reliability and validity of survey measures used to evaluate health plans and providers from the consumer's perspective.

DATA SOURCES

Members (166,074) of 306 U.S. health plans obtained from the National CAHPS Benchmarking Database 2.0, a voluntary effort in which sponsors of CAHPS surveys contribute data to a common repository.

STUDY DESIGN

Members of privately insured health plans serving public and private employers across the United States were surveyed by mail and telephone. Interitem correlations and correlations of items with the composite scores were estimated. Plan-level and internal consistency reliability are estimated. Multivariate associations of composite measures with global ratings are also examined to assess construct validity. Confirmatory factor analysis is used to examine the factor structure of the measure.

FINDINGS

Plan-level reliability of all CAHPS 2.0 reporting composites is high with the given sample sizes. Fewer than 170 responses per plan would achieve plan-level reliability of .70 for the five composites. Two of the composites display high internal consistency (Cronbach's alpha > or = .75), while responses to items in the other three composites were not as internally consistent (Cronbach's alpha from .58 to .62). A five-factor model representing the CAHPS 2.0 composites fits the data better than alternative two- and three-factor models.

CONCLUSION

Two of the five CAHPS 2.0 reporting composites have high internal consistency and plan-level reliability. The other three summary measures were reliable at the plan level and approach acceptable levels of internal consistency. Some of the items that form the CAHPS 2.0 adult core survey, such as the measure of waiting times in the doctor's office, could be improved. The five-dimension model of consumer assessments best fits the data among the privately insured; therefore, consumer reports using CAHPS surveys should provide feedback using five composites.

Patterns of coping among persons with HIV infection: configurations, correlates, and change

This study examines coping in response to HIV infection, using longitudinal data from a nationally representative sample (n = 2,864) of HIV-infected persons. We investigated configurations of coping responses, the correlates of configuration membership, the stability of coping configurations, and the relationship of coping to emotional well-being. Four coping configurations emerged from cluster analyses: relatively frequent use of blame-withdrawal coping, frequent use of distancing, frequent active-approach coping, and infrequent use of all three coping strategies ("passive" copers). Passive copers had few symptoms, high levels of physical functioning, and high emotional well-being; blame-withdrawal copers had the opposite pattern. Of those completing a second interview 1 year after baseline, 46% had the same coping configuration. Increases in the number of HIV-related symptoms raised the probability of blame-withdrawal coping at follow-up, whereas decreases raised the probability of passive coping. Infrequent use of coping responses at baseline was related to greater emotional well-being 1 year later. This result, in conjunction with the high levels of emotional well-being in the passive cluster, suggests that high levels of distress can induce blame-withdrawal coping whereas coping efforts are minimal when social support and emotional well-being are high. Results highlight issues in ascertaining the causal direction between coping and psychological outcomes, as well as in specifying the nature of stressful situations with which people are coping.

What do patients value in their hospital care? An empirical perspective on autonomy centred bioethics

OBJECTIVE

Contemporary ethical accounts of the patient-provider relationship emphasise respect for patient autonomy and shared decision making. We sought to examine the relative influence of involvement in decisions, confidence and trust in providers, and treatment with respect and dignity on patients' evaluations of their hospital care.

DESIGN

Cross-sectional survey.

SETTING

Fifty one hospitals in Massachusetts.

PARTICIPANTS

Stratified random sample of adults (N=27 414) discharged from a medical, surgical, or maternity hospitalisation between January and March, 1998. Twelve thousand six hundred and eighty survey recipients responded.

MAIN OUTCOME MEASURE

Respondent would definitely be willing to recommend the hospital to family and friends.

RESULTS

In a logistic regression analysis, treatment with respect and dignity (odds ratio (OR) 3.4, 99% confidence interval (CI) 2.8 to 4.2) and confidence and trust in providers (OR 2.5, CI 2.1 to 3.0) were more strongly associated with willingness to recommend than having enough involvement in decisions (OR 1.4, CI 1.1 to 1.6). Courtesy and availability of staff (OR 2.5, CI 2.1 to 3.1), continuity and transition (OR 1.9, CI 1.5 to 2.2), attention to physical comfort (OR 1.8, CI 1.5 to 2.2), and coordination of care (OR 1.5, CI 1.3 to 1.8) were also significantly associated with willingness to recommend.

CONCLUSIONS

Confidence and trust in providers and treatment with respect and dignity are more closely associated with patients' overall evaluations of their hospitals than adequate involvement in decisions. These findings challenge a narrow emphasis on patient autonomy and shared decision making, while arguing for increased attention to trust and respect in ethical models of health care.

Physician specialization and antiretroviral therapy for HIV

BACKGROUND

Since the introduction of the first protease inhibitor in January 1996, there has been a dramatic change in the treatment of persons infected with HIV. The changing nature of HIV care has important implications for the types of physicians that can best care for patients with HIV infection.

OBJECTIVE

To assess the association of specialty training and experience in the care of HIV disease with the adoption and use of highly active antiretroviral (ARV) therapy (HAART).

DESIGN

Observational cohort study of patients under care for HIV infection and their physicians.

PATIENTS AND SETTING

This analysis used data collected from a national probability sample of noninstitutionalized persons with HIV infection participating in the HIV Costs and Service Utilization Study and their primary physicians. We analyzed 1,820 patients being cared for by 374 physicians.

MEASUREMENTS

Rates of HAART use at 12 months and 18 months after the approval of the first protease inhibitor.

RESULTS

Forty percent of the physicians were formally trained in infectious diseases (ID), 38% were general medicine physicians with self-reported expertise in the care of HIV, and 22% were general medicine physicians without self-reported expertise in the care of HIV. The majority of physicians (69%) reported a current HIV caseload of 50 patients or more. In multivariable models controlling for patient characteristics, there were no differences between generalist experts and ID physicians in rates of HAART use in December 1996. When compared to ID physicians, however, patients being treated by non-expert general medicine physicians were less likely to be on HAART (odds ratio [OR], 0.32; 95% confidence interval [95% CI], 0.17 to 0.61). Patients being treated by low-volume physicians were also much less likely to be on HAART therapy than those treated by high-volume physicians (OR, 0.26; 95% CI, 0.14 to 0.48). These findings were attenuated by June 1997, suggesting that over time, the broader physician community successfully adopted HAART therapy. This finding is consistent with prior research on the diffusion of innovations.

CONCLUSIONS

Similar proportions of patients treated by expert generalists and ID specialists were on appropriate HAART therapy by December 1996 and July 1997. Patients treated by non-expert generalists, most of whom were the lowest-volume physicians, were much less likely to be on appropriate ARV therapy in the earlier time period. Our findings demonstrate that expert generalists who develop specialized expertise are able to provide care of quality comparable to that of specialists.

Guideline recommendations for treatment of schizophrenia: the impact of managed care

BACKGROUND

Medicaid-managed care has been shown to reduce the number and length of psychiatric hospitalizations, but little is known about the clinical and social consequences of such managed care programs. The purpose of this study was to compare the treatment of schizophrenia for disabled Medicaid beneficiaries who were and were not enrolled in managed care.

METHODS

This was a prospective observational study of patients who sought care for a psychiatric crisis from June 7, 1997, to May 13, 1999. Patients were followed up for 6 months. Inpatient and outpatient mental health facilities in Massachusetts were studied. The participants included 420 adult Medicaid beneficiaries, aged 24 to 64 years, who were treated for schizophrenia; 784 eligible beneficiaries were originally contacted and invited to participate (53.6% response). A private managed behavioral health care organization administered the Medicaid mental health benefit for about half the patients in the study. The other half were enrolled in the dually insured fee-for-service Medicare/Medicaid plan. The main outcome measures were adherence to the Schizophrenia Patient Outcomes Research Team treatment recommendations from inpatient and outpatient medical records, self-reported quality of interpersonal interactions between patient and clinician, self-reported care experiences and outcomes, and clinician-reported outcomes.

RESULTS

There were no differences between the managed care plan and the unmanaged fee-for-service plan in adherence to the schizophrenia treatment guidelines. However, much outpatient care in both programs was inconsistent with treatment guidelines. Inpatient treatment was far more likely to conform to guidelines than outpatient treatment. Patient ratings of their care were positive and not different between plans. Clinical outcome and health-related quality of life were not different between plans.

CONCLUSIONS

A major change in Massachusetts in the way mental health care is organized and financed had neither a negative nor a positive effect on care quality. However, adherence to nationally accepted guidelines for care was only modest, suggesting a need to improve the delivery of treatment to the most disabled highest-risk adults with schizophrenia.

The World Health Organization Health and Work Performance Questionnaire (HPQ)

This report describes the World Health Organization Health and Work Performance Questionnaire (HPQ), a self-report instrument designed to estimate the workplace costs of health problems in terms of reduced job performance, sickness absence, and work-related accidents-injuries. Calibration data are presented on the relationship between individual-level HPQ reports and archival measures of work performance and absenteeism obtained from employer archives in four groups: airline reservation agents (n = 441), customer service representatives (n = 505), automobile company executives (n = 554), and railroad engineers (n = 850). Good concordance is found between the HPQ and the archival measures in all four occupations. The paper closes with a brief discussion of the calibration methodology used to monetize HPQ reports and of future directions in substantive research based on the HPQ.

What do collaborative improvement projects do? Experience from seven countries

BACKGROUND

Health care organizations are increasingly adopting multiorganizational collaborative approaches to quality improvement. Collaboratives have been conducted in many countries. There are large variations in the way collaboratives are structured and run, but there is no widely accepted framework for describing the components of collaboratives. Thus, it is difficult to study which approaches are most effective.

METHOD

The authors conducted semistructured interviews with 15 leaders of collaboratives to ascertain the common components of collaboratives and identify variations in the ways these components are implemented.

RESULTS

The study identified seven features of collaboratives that the leaders interviewed thought were critical determinants of how effective the collaboratives were: sponsorship, topic, ideas for improvements, participants, senior leadership support, preliminary work and learning, and strategies for learning about and making improvements. For example, every interviewee mentioned that having participants collect data, perform audit work, or analyze the system they were in before the collaboration started was important to understanding their organization and the nature of the problems they had and to developing baseline data for later comparison. The authors describe variations in how these features have been implemented and possible functions of these features.

CONCLUSION

Systematically studying the impact of variations in the seven key features of collaboratives could yield important information about their role and impact.

A hospitalization from hell: a patient's perspective on quality

Patients usually cannot assess the technical quality of their care; however, examining a hospitalization through the patients' eyes can reveal important information about the quality of care. Patients are the best source of information about a hospital system's communication, education, and pain-management processes, and they are the only source of information about whether they were treated with dignity and respect. Their experiences often reveal how well a hospital system is operating and can stimulate important insights into the kinds of changes that are needed to close the chasm between the care provided and the care that should be provided. This article examines the case of a patient admitted for ankle arthrodesis due to severe hemophilia-related arthritis. The surgery was successful, but the hospital stay was marked by inefficiency and inconveniences, as well as events that reveal fundamental problems with the hospital's organization and teamwork. These problems could seriously compromise the quality of clinical care. Unfortunately, most of these events occur regularly in U.S. hospitals. Relatively easy and inexpensive ways to avoid many of these problems are discussed, such as reducing variability in non-urgent procedures and routinely asking patients about their experiences and suggestions for improvement.

Beneficiary reported experience and voluntary disenrollment in Medicare managed care

Disenrollment rates have often been used as indicators of health plan quality, because they are readily available and easily understood by purchasers, health plans, and consumers. Over the past few years, however, indicators that more directly measure technical quality and consumer experiences with care have become available. In this observational study, we examined the relationship between voluntary disenrollment rates from Medicare managed care (MMC) plans and other measures of health plan quality. The results demonstrate that voluntary disenrollment rates are strongly related to direct measures of patient experiences with care and are an important complement to other measures of health plan performance.

HIV patients' experiences with inpatient and outpatient care: results of a national survey

CONTEXT

Little is known about HIV patients' care experiences.

OBJECTIVE

To assess HIV patients' experiences with inpatient and outpatient care, and to assess the relationship and relative influence of patient characteristics and site of care on care experiences.

DESIGN

Cohort study.

SETTING

Patients with HIV receiving care outside of emergency rooms, prisons, or the military throughout the continental United States. One thousand seventy-four patients provided ratings of an inpatient stay and 2204 rated an outpatient visit; 818 patients provided evaluations of both inpatient and outpatient care.

PATIENTS

A national probability sample of persons in care for HIV from the HIV Cost and Services Utilization Study.

MEASUREMENTS

Outcome variables were rates of problems with, and global ratings of, inpatient and outpatient care.

RESULTS

Mean problem rates were 20.9% and 8.4% (lower score means fewer problems) for inpatient and outpatient care, respectively. On 9 of 10 of the individual inpatient report items, 15% or more of respondents reported problems. Global ratings of inpatient and outpatient care were 65.3 and 75.0 (0-100 scale, higher scores indicate better ratings), respectively. In multivariable models that controlled for site effects, the only patient characteristic that was consistently associated with problem rates and global ratings of care was mental health (P <0.0001 for both inpatient and outpatient care). Models including site effects explained two to four times as much variance as models excluding site effects.

CONCLUSIONS

Inpatients with HIV reported higher problem rates with inpatient than outpatient care. Better provider-patient communication during inpatient stays is needed. For both inpatient and outpatient care, quality improvement efforts may be most productively focused on providers and processes of care at sites rather than on specific patient subgroups.

The influence of patient characteristics on ratings of managed behavioral health care

Despite current emphasis on consumer-based performance measures, little is known about factors that influence consumer ratings of behavioral health care. This study examines the influence of patient characteristics, health care use, and insurance coverage on patients' ratings of their managed behavioral health care in both commercial and public plans. Older and healthier patients rated their behavioral health care and health plan more highly than did other patients. Patients with less education and those whose insurance paid all costs of care gave consistently higher plan ratings. Women and frequent users enrolled in commercial plans gave more positive care ratings. After adjusting for enrollee characteristics and coverage, there were no differences between ratings of patients in commercial and public plans. These results are consistent with other research that illustrates the importance of adjusting health care ratings for patient characteristics when comparing plans.

Evolving dissatisfaction among primary care physicians

OBJECTIVE

To examine trends in career satisfaction among physicians working with managed care plans.

STUDY DESIGN

Cross-sectional surveys conducted in 1996 and 1999.

PARTICIPANTS AND METHODS

We surveyed primary care physicians (PCPs) affiliated with 5 large health plans in Massachusetts and assessed physicians' ratings of overall satisfaction with their current practice situation and with managed care.

RESULTS

A total of 1336 PCPs (56%) responded to the 1996 survey and 966 (42%) responded to the 1999 survey. In 1996, 19.8% of physicians were somewhat or very dissatisfied with their current practice situation vs 33.4% in 1999, an increase of more than 50% (P < .001). Overall dissatisfaction with managed care increased from 28.3% to 62.2% (P < .001). In multivariable models, external influences on physicians' practices were important predictors of overall dissatisfaction. Physicians whose choice of hospitals was restricted (odds ratio, 2.23; 95% confidence interval, 1.30-3.78) and those who reported that managed care plans influenced their practice "a lot" (odds ratio, 1.85; 95% confidence interval, 1.10-3.11) were more likely to be dissatisfied. Adequacy of reimbursement was an important predictor of overall satisfaction and satisfaction with managed care, and ways physicians experienced financial incentives were associated with managed care dissatisfaction.

CONCLUSIONS

Primary care physicians in Massachusetts are increasingly dissatisfied with their practice and with managed care. Continued erosion of physician satisfaction could have significant implications for quality of care and the quality of the workforce attracted to medicine as a career.

Dimensions of plan performance for sick and healthy members on the Consumer Assessments of Health Plans Study 2.0 survey

BACKGROUND

The quality of health plan care may differ for members in good and poor health.

OBJECTIVE

To determine whether reports from sick and healthy members reflect distinct aspects of plan performance.

RESEARCH DESIGN

Mean health plan scores were analyzed on the 1998 and 1999 Medicare Managed Care (MMC) Consumer Assessments of Health Plans (CAHPS) surveys, treating responses from sick and healthy members as separate plan measures. Alternative definitions of health were compared and the one that defined groups with the most distinct experiences was selected. Using factor analysis, composites of report items defined for these groups were identified. Mean ratings were regressed on these composites.

SUBJECTS

Two hundred ninety thousand seven hundred thirty-nine Medicare managed care beneficiaries from 381 health plan-reporting units.

MEASURES

MMC-CAHPS survey responses, including four overall ratings and 30 specific report items.

RESULTS

A question about general health status best defined subgroups with distinct experiences. Report items grouped into eight factors: care for healthy members, care for sick members, finding and communicating with a doctor for sick members, plan customer service, plan-provided medical services and equipment, vaccinations, prescriptions, and smoking cessation advice. Ratings by each subgroup were generally most strongly predicted by reports on care for the same subgroup and by customer service and plan-provided services (for ratings of plan) and access to doctors.

CONCLUSIONS

Reports from sick and healthy members measure distinct dimensions of health plan quality, especially in the domain of patient care. Distinguishing these dimensions might help in informing consumers and targeting quality improvement efforts.

Factors affecting response rates to the Consumer Assessment of Health Plans Study survey

OBJECTIVES

Assess the determinants of nonresponse to a consumer health care survey.

METHODS

The first (1997; CAHPS 1.0) and third (1999; CAHPS 2.0) Medicare managed care (MMC) CAHPS surveys collected data on 215 and 365 health plan reporting units, respectively. Data indicated which beneficiaries responded by mail, responded by phone, could not be located, and did not respond. InterStudy data described plan characteristics. chi2 tests and logistic regression models, adjusted for clustering by plan, were used to test associations of individual and plan characteristics with availability of good contact information and response given good contact information.

RESULTS

Response rates in the 1997 and 1999 surveys were 75% and 80%, respectively. Older and disabled beneficiaries, women, nonwhite beneficiaries, and persons living in areas with more residents who were nonwhite, on public assistance, and less educated had lower response rates. These associations were partly explained by the distribution of bad contact information, but even among beneficiaries who could be located plan response rates varied greatly. For-profit plans are significantly more likely to have high rates of bad contact information and lower response rates. Telephone follow-up improved the sociodemographic representativeness of the sample, for both high and low response rate plans.

CONCLUSION

CAHPS-MMC survey procedures, in particular telephone follow-up, have resulted in high response rates, and current case-mix strategies compensate for some of the remaining effects of differing response rates on comparisons among plans. Further efforts to explore the determinants of response rates are warranted.

Similarities and differences in choosing health plans

BACKGROUND

Increasingly, consumers have multiple health insurance options. New information is being developed to help consumers with these choices.

OBJECTIVES

To study similarities and differences in how the publicly and privately insured choose health plans. To explore the effect of traditional enrollment materials and reports developed by the Consumer Assessment of Health Plans Study (CAHPS) on consumers' perceptions and decision-making.

RESEARCH DESIGN

Using data from eight CAHPS demonstrations, we tested for significant differences across consumers with employer-sponsored insurance, Medicaid, and Medicare.

SUBJECTS

Approximately 10,000 consumers with employer-sponsored, Medicaid, and Medicare health plans.

MEASURES

Perceptions of the health plan selection process, use of information sources, and reactions to and use of traditional enrollment materials and CAHPS reports.

RESULTS

Most consumers with all types of insurance thought that choosing a health plan was important and obtained information from multiple sources. Choosing a plan was more difficult for Medicare and Medicaid recipients than for the privately insured. When choosing a plan, Medicaid recipients cared most about convenience and access, whereas the privately insured emphasized providers and costs. The percentage of consumers who looked at and remembered the CAHPS report varied widely from 24% to 77%. In all but one of the demonstration sites, most consumers spent less than 30 minutes looking at the CAHPS report.

CONCLUSIONS

Group sponsors and the developers of information interventions such as CAHPS may need to invest in developing and testing different reporting approaches for Medicare, Medicaid, and privately insured consumers.

Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members

OBJECTIVE

To assess the nonresponse bias associated with mail-survey returns and the potential for telephone interviews with nonrespondents to reduce that bias.

METHODS

A mail survey about health care experiences was conducted with samples of 800 members in each of four health plans. Subsequent attempts were made to interview nonrespondents by telephone.

RESULTS

Response rates for the mail surveys averaged 46%; the telephone effort raised the average to 66%. On 17 of 19 measures of health status or need and use of health services, mail respondents were in poorer health and needed more services than interviewed nonrespondents. Thirteen of 36 reports and ratings of health care also differed significantly between the two groups. Based on administrative data, telephone interviews of mail nonrespondents improved the demographic representativeness of the responding samples. Adjusting mail returns to sample population characteristics could not replicate the dual-mode results.

CONCLUSIONS

Returns to mail surveys are likely to be related to survey content and hence are potentially biased. Nonresponse to phone surveys is less directly related to survey content. Telephone interviews with mail nonrespondents not only increase response rates but also can produce less biased samples than mail-only protocols.