Factors affecting response rates to the Consumer Assessment of Health Plans Study survey

Mixed Mode Substantially Increases Hospital Consumer Assessment of Healthcare Providers and Systems Response Rates Relative to Single-Mode Protocols

BACKGROUND

Low response rates (RRs) can affect hospitals' data collection costs for patient experience surveys and value-based purchasing eligibility. Most hospitals use single-mode approaches, even though sequential mixed mode (MM) yields higher RRs and perhaps better patient representativeness. Some hospitals may be reluctant to incur MM's potential additional cost and complexity without knowing how much RRs would increase.

OBJECTIVE

The aim of this study was to estimate the differences in RR and patient representation between MM and single-mode approaches and to identify hospital characteristics associated with the largest RR differences from MM of single-mode protocols (mail-only, phone-only).

RESEARCH DESIGN

Patients were randomized within hospitals to one of 3 modes (mail-only, phone-only, MM).

SUBJECTS

A total of 17,415 patients from the 51 nationally representative US hospitals participating in a randomized HCAHPS mode experiment.

RESULTS

Mail-only RRs were lowest for ages 18-24 (7%) and highest for ages 65+ (31%-35%). Phone-only RRs were 24% for ages 18-24, increasing to 37%-40% by ages 55+. MM RRs were 28% for ages 18-24, increasing to 50%-60% by ages 65-84. Lower hospital-level mail-only RRs strongly predicted greater gains from MM. For example, a hospital with a 15% mail-only RR has a predicted MM RR >40% (with >25% occurring in telephone follow-up).

CONCLUSION

MM increased representation of hard-to-reach (especially young adult) patients and hospital RRs in all mode experiment hospitals, especially in hospitals with low mail-only RRs.

Survey Protocols, Response Rates, and Representation of Underserved Patients: A Randomized Clinical Trial

Importance

Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts.

Objective

To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups.

Design, Setting, and Participants

This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023.

Exposures

Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone).

Main Outcomes and Measures

RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity.

Results

A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex.

Conclusions and Relevance

In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.

The Effect of Social Deprivation on Patient Satisfaction in Otolaryngology Clinics

OBJECTIVES

Multiple factors have been associated with lower satisfaction scores. We hypothesize that patients with a more deprived socioeconomic status will have different patient satisfaction scores than patients of higher socioeconomic status.

METHODS

We reviewed Press Ganey satisfaction scores for new, outpatient visits between January 1, 2014, and December 31, 2018. Due to the high ceiling effects of the survey, "satisfaction" was defined as achieving a perfect score of 100. We determined social deprivation using the 2015 Area Deprivation Index (ADI) which serves as a proxy for socioeconomic status derived from zip codes. Patient satisfaction was defined as a binary variable. Univariate and multivariate binary logistic regression analyses were used to identify factors correlated with patient satisfaction.

RESULTS

There were 3239 unique new patients with completed surveys. Univariate analysis demonstrated decreased odds of achieving satisfaction for both the Total Score and Provider Sub-Score for each decile increase in ADI (OR 0.94; CI = 0.908-0.981; P = .003 and OR 0.94; CI = 0.91-0.98; P = .002 respectively). Multivariate analysis revealed the odds for reporting satisfaction for each decile increase in ADI were 0.96 for Total Score (CI = 0.921-0.998; P = .038) and 0.96 for Provider Sub-Score (CI = 0.92-0.993; P = .019). Patients in the most deprived quartile, compared to the least deprived, were significantly less likely to be satisfied with their care for both Total Score (OR 0.70; CI = 0.564-0.865; P = .001) and Provider Sub-Score (OR 0.69; CI = 0.558-0.852; P = .001).

CONCLUSIONS

Increased social deprivation was an independent predictor of lower patient satisfaction in otolaryngology outpatients using the Press Ganey survey. This non-modifiable outcome should be taken into consideration when evaluating patient satisfaction scores and offers further support to the need of addressing and improving healthcare discrepancies in the field of otolaryngology.

LEVEL OF EVIDENCE

Level 3.

Patient Experience for Hispanic Older Adults Varies by Language Preference

BACKGROUND

Hispanic people with Medicare report worse patient experiences than non-Hispanic White counterparts. However, little research examines how these disparities may vary by language preference (English/Spanish).

OBJECTIVES

Using Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data, assess whether 2014-2018 disparities in patient experiences for Hispanic people with Medicare vary by language preference.

RESEARCH DESIGN

We fit a series of linear, case-mix adjusted models predicting Medicare CAHPS measures by race/ethnicity/language preference (Hispanic Spanish-respondents; Hispanic Spanish-preferring English-respondents; Hispanic English-preferring respondents; and non-Hispanic White English-respondents).

SUBJECTS

A total of 1,006,543 Hispanic and non-Hispanic White respondents to the Medicare 2014-2018 CAHPS surveys.

RESULTS

There were disparities for all Hispanic groups relative to non-Hispanic White English-respondents. Hispanic Spanish-preferring English-respondents reported worse experience than Hispanic Spanish-respondents for getting care quickly (-8 points), getting needed care (-5 points), doctor communication (-2 points), and customer service (-1 point), but better experiences for flu immunization (+2 points). Similarly, Hispanic Spanish-preferring English-respondents reported worse experience than Hispanic English-preferring respondents for getting care quickly (-4 points) and getting needed care (-2 points). Hispanic English-preferring respondents reported worse experience than Hispanic Spanish-respondents for getting care quickly (-4 points), getting needed care (-3 points), doctor communication and customer service (-2 points each), but better experience for flu immunization (+2 points).

CONCLUSIONS

Regardless of language preference, Hispanic people with Medicare experience disparities in patient care relative to non-Hispanic White English-preferring counterparts. Hispanic Spanish-preferring English-respondents report the worse experiences, followed by Hispanic English-preferring respondents. Hispanic Spanish-respondents experienced the least disparities of the three Hispanic language subgroups.

A Systematic Review of Strategies to Enhance Response Rates and Representativeness of Patient Experience Surveys

BACKGROUND

Data from surveys of patient care experiences are a cornerstone of public reporting and pay-for-performance initiatives. Recently, increasing concerns have been raised about survey response rates and how to promote equity by ensuring that responses represent the perspectives of all patients.

OBJECTIVE

Review evidence on survey administration strategies to improve response rates and representativeness of patient surveys.

RESEARCH DESIGN

Systematic review adhering to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.

STUDY SELECTION

Forty peer-reviewed randomized experiments of administration protocols for patient experience surveys.

RESULTS

Mail administration with telephone follow-up provides a median response rate benefit of 13% compared with mail-only or telephone-only. While surveys administered only by web typically result in lower response rates than those administered by mail or telephone (median difference in response rate: -21%, range: -44%, 0%), the limited evidence for a sequential web-mail-telephone mode suggests a potential response rate benefit over sequential mail-telephone (median: 4%, range: 2%, 5%). Telephone-only and sequential mixed modes including telephone may yield better representation across patient subgroups by age, insurance type, and race/ethnicity. Monetary incentives are associated with large increases in response rates (median increase: 12%, range: 7%, 20%).

CONCLUSIONS

Sequential mixed-mode administration yields higher patient survey response rates than a single mode. Including telephone in sequential mixed-mode administration improves response among those with historically lower response rates; including web in mixed-mode administration may increase response at lower cost. Other promising strategies to improve response rates include in-person survey administration during hospital discharge, incentives, minimizing survey language complexity, and prenotification before survey administration.

Development, methodology, and adaptation of the Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS®) patient experience survey, 2007–2019

The Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS®) surveys collect standardized information about patient experiences of care from nationally representative samples of people with Medicare to support consumers’ enrollment choices and enable the Centers for Medicare & Medicaid Services to monitor care quality and incentivize high quality patient-centered care. Since 2007, protocols for data collection, analysis, and reporting have evolved to address expanded Medicare coverage options and a shift from a single survey vendor to a model in which health plans hire approved vendors to administer the survey. During that time, response rates for all types of surveys have declined; increasing effort has gone toward increasing survey participation, especially among people whose preferred language is not English. In this paper, we describe the history, goals, and current use of the Medicare CAHPS surveys. We also summarize key methodological issues, such as sample design, field implementation and data cleaning, adjustment, scoring, and report production. Additionally, we discuss issues that may arise more generally in managing a large, annual national survey that has direct impact on policy, and consider how a long-running survey of this nature may need to evolve to reflect changes in health care delivery and promote standardization in survey administration while maintaining survey content.

Adjusting for Patient Characteristics to Compare Quality of Care Provided by Serious Illness Programs

Background: To compare serious illness programs (SIPs) using recently developed patient experience measures, adjustment must be made for patient characteristics not under control of the programs. Objectives: To develop a case-mix adjustment model to enable fair comparison of patient experience between SIPs by investigating the roles of patient characteristics, proxy response, and mode of survey administration (mail-only vs. mail with telephone follow-up) in survey responses. Methods: Using survey data from 2263 patients from 32 home-based SIPs across the United States, we fit regression models to assess the association between patient-level variables and scores for seven quality measures (Communication, Care Coordination, Help for Symptoms, Planning for Care, Support for Family and Friends, and two global assessments of care). Characteristics that are not consequences of the care the program delivered were considered as adjustors. Results: Final recommended case-mix adjustors are age, education, primary diagnosis, self-reported functional status, self-rated physical health, self-rated mental health, proxy respondent use, and response percentile (a measure of how soon a person responded compared with others in the same program and mode). Age, primary diagnosis, self-rated mental health, and proxy respondent use had the most impact on program-level scores. We also recommend adjusting for mode of survey administration. We find that up to 12 percent of pairs of programs would have their rankings reversed by adjustment. Conclusions: To ensure fair comparison of programs, scores should be case-mix adjusted for variables that influence patients' reports about care quality, but are not under the control of the program administering care.

Development and validation of a patient experience of care survey for emergency departments

OBJECTIVES

To (1) develop a survey to assess the patient experience of care in hospital-based emergency departments (ED) and (2) evaluate the reliability and validity of composite measures of patient experience using data collected through the experimental implementation of the newly developed Emergency Department Patient Experience of Care (EDPEC) Discharged to Community (DTC) Survey.

DATA SOURCE

4893 adult patients were treated in the ED of 16 hospitals across the United States in 2018.

STUDY DESIGN

The study utilized a cross-sectional survey.

DATA COLLECTION

Survey development activities included a literature review, focus groups, and cognitive interviews with recently discharged ED patients, technical expert panels, and multiple field experiments. Survey development resulted in a 34-item instrument; the analysis reported here focuses on 18 items on patient experience of care. Using data from the EDPEC DTC Survey in the 2018 Feasibility Test, we performed confirmatory factor analysis to group 15 evaluative survey items into composite measures. We examined internal consistency reliability, interunit reliability, and associations between each composite measure and patients' overall rating and willingness to recommend the ED.

PRINCIPAL FINDINGS

Analyses of 15 evaluative items identified four composite measures: Getting Timely Care, How Well Doctors and Nurses Communicate, Communication about Medications, and Communication about Follow-up. Patient-level internal consistency reliability exceeded 0.75 for two of four composites; ED-level internal consistency reliability exceeded 0.83 for all four composites. Interunit reliability estimates indicated that 450 survey completes per ED results in at least 0.70 reliability for all composites. Higher scores on each composite were associated with higher overall ratings and willingness to recommend the ED.

CONCLUSIONS

The composite measures derived from the EDPEC DTC Survey are statistically reliable and valid. These results provide guidance for EDPEC DTC Survey adopters on how to construct meaningful and psychometrically-sound composite measures for monitoring the quality of care they provide.

Evaluating Factors That Influence Patient Satisfaction in Otolaryngology Clinics

OBJECTIVES

To identify factors that influence patient satisfaction during outpatient visits in various settings of otolaryngology clinics in an academic medical center.

STUDY DESIGN

Retrospective review.

SETTING

Academic medical center.

METHODS

We reviewed Press Ganey patient satisfaction survey responses for new, outpatient visits between January 1, 2014 and December 31, 2018. Self-reported race was identified using electronic medical records. Multivariate binary logistic regression analyses were used to identify continuous and categorical variables associated with patient satisfaction.

RESULTS

There were 3998 unique new patient visits with completed surveys. Multivariate analysis revealed that responses for patients <18 years old are less likely to be satisfied with their care compared to patients ≥18 years old (OR 0.66; P < .001). For each 10-minute increase in wait time, patients were 43.4% less likely report satisfaction (P < .001). African American patients were also less likely to report satisfaction (OR 0.22; P = .043) while Native Hawaiian and Pacific Islanders were over 3 times more likely to be satisfied (OR 3.6; P = .013). Additionally, Medicare patients and those who were seen at community satellite clinics compared to the main University Hospital had increased odds of achieving satisfactory care (OR 1.3; P = .005 and OR 1.3; P = .002, respectively).

CONCLUSIONS

Wait time, clinic location, patient race, insurance provider, and age were all shown to significantly influence patient-reported satisfaction. Understanding how these variables influence patient satisfaction will hopefully lead to processes that improve patient satisfaction.

LEVEL OF EVIDENCE

Level 3.

The Emergency Department Patient Experience: In Their Own Words

In this paper, the authors review the verbatim comments from patients’ post-emergency department discharge survey results and highlight the impact that doctors and nurses have on the patient's experience. This paper also explores the benefits of delivering a positive experience on patients, clinicians, and healthcare systems. Many interventions that have worked are simple to implement and do not tend to require significant capital investment while having the potential of improving emergency department encounters for all those involved. There are always limitations, selection biases, and issues with generalizability in reviewing retrospective and subjective survey response data.

Racial/Ethnic Differences in Emergency Department Utilization and Experience

BACKGROUND

Previous work has demonstrated racial/ethnic differences in emergency department (ED) utilization, but less is known about racial/ethnic differences in the experience of care received during an ED visit.

OBJECTIVE

To examine differences in self-reported healthcare utilization and experiences with ED care by patients' race/ethnicity.

DESIGN

Adult ED patients discharged to community (DTC) were surveyed (response rate: 20.25%) using the Emergency Department Patient Experience of Care (EDPEC) DTC Survey. Linear regression was used to estimate case-mix-adjusted differences in patient experience between racial/ethnic groups.

PARTICIPANTS

3122 survey respondents who were discharged from the EDs of 50 hospitals nationwide January-March 2016.

MAIN MEASURES

Six measures: getting timely care, doctor and nurse communication, communication about medications, receipt of sufficient information about test results, whether hospital staff discussed the patient's ability to receive follow-up care, and willingness to recommend the ED.

KEY RESULTS

Black and Hispanic patients were significantly more likely than White patients to report visiting the ED for an ongoing health condition (40% Black, 30% Hispanic, 28% White, p<0.001), report having visited an ED 3+ times in the last 6 months (26% Black, 25% Hispanic, 19% White, p<0.001), and report not having a usual source of care (19% Black, 19% Hispanic, 8% White, p<0.001). Compared with White patients, Hispanic patients more often reported that hospital staff talked with them about their ability to receive needed follow-up care (+7.2 percentile points, p=0.038) and recommended the ED (+7.2 points, p=0.037); Hispanic and Black patients reported better doctor and nurse communication (+6.4 points, p=0.008; +4 points, p=0.036, respectively).

CONCLUSIONS

Hispanic and Black ED patients reported higher ED utilization, lacked a usual source of care, and reported better experience with ED care than White patients. Results may reflect differences in care delivery by staff and/or different expectations of ED care among Hispanic and Black patients.

Insights for Conducting Large-Scale Surveys with Veterans Who Have Experienced Homelessness

Surveys of underserved patient populations are needed to guide quality improvement efforts but are challenging to implement. The goal of this study was to describe recruitment and response to a national survey of Veterans with homeless experience (VHE). We randomly selected 14,340 potential participants from 26 U.S. Department of Veterans Affairs (VA) facilities. A survey contract organization verified/updated addresses from VA administrative data with a commercial address database, then attempted to recruit VHE through 4 mailings, telephone follow-up, and a $10 incentive. We used mixed-effects logistic regressions to test for differences in survey response by patient characteristics. The response rate was 40.2% (n=5,766). Addresses from VA data elicited a higher response rate than addresses from commercial sources (46.9% vs 31.2%, p<.001). Residential addresses elicited a higher response rate than business addresses (43.8% vs 26.2%, p<.001). Compared to non-respondents, respondents were older, less likely to have mental health, drug, or alcohol conditions, and had fewer VA housing and emergency service visits. Collectively, our results indicated a national mailed survey approach is feasible and successful for reaching VA patients who have recently experienced homelessness. These findings offer insight into how health systems can obtain perspectives of socially disadvantaged groups.

Press Ganey Surveys in Patients Undergoing Upper-Extremity Surgical Procedures: Response Rate and Evidence of Nonresponse Bias

BACKGROUND

Patient satisfaction surveys are important measures of the patient experience that provide data for quality improvement. The purpose of this study was to establish the response rate and the factors associated with the completion of the Press Ganey (PG) Ambulatory Surgery Survey (PGAS) in patients who underwent ambulatory upper-extremity surgical procedures.

METHODS

A prospective orthopaedic registry at a single academic ambulatory surgical center was retrospectively reviewed for patients who underwent an upper-extremity surgical procedure from 2015 to 2019. The institutional PG database was queried to determine the patients who completed the PGAS postoperatively. The response rate was calculated, and baseline characteristics and patient-reported outcome measures were compared between responders and nonresponders.

RESULTS

Of the 1,489 patients included, 201 (13.5%) were responders and 1,288 (86.5%) were nonresponders. Differences existed in baseline characteristics between groups, with responders being significantly older (p = 0.004) and having significantly higher proportions of White race (p < 0.001), college education (p = 0.011), employment (p = 0.005), marriage (p = 0.006), and higher income earners (p < 0.001). Responders had significantly better baseline Patient-Reported Outcomes Measurement Information System scores across multiple domains (p < 0.05), but these differences were not clinically meaningful.

CONCLUSIONS

PGAS response rates were low (13.5%), and differences between responders and nonresponders may be utilized by hospitals to target feedback from underrepresented patient populations. Surgeons, policymakers, and health-care administrators should use caution with the interpretation of PGAS results because responders may not be representative of all patients.

Racial/Ethnic Disparities in Patient Experiences With Health Care in Association With Earlier Stage at Colorectal Cancer Diagnosis: Findings From the SEER-CAHPS Data

BACKGROUND

Racial/ethnic minorities are more likely to be diagnosed at a later stage of colorectal cancer (CRC). Therefore, our objective was to identify whether racial/ethnic differences in patient experiences (PE) with health care are associated with stage at CRC diagnosis.

METHODS

The authors used the National Cancer Institute Surveillance, Epidemiology and End Results registry data linked with the Consumer Assessment of Healthcare Providers and Systems to conduct a retrospective cohort analysis. They examined composite measures from surveys to assess 3 domains: patient-centeredness, timeliness, and realized access. Multivariable logistic regression was used to determine the association between PE with care and earlier stage at diagnosis.

RESULTS

Of the 9211 patients, 31.1% non-Hispanic White, 27.2% non-Hispanic Black (NHB), 32.3% Hispanic, and 36.4% Asian were diagnosed with early stage cancer. Compared with non-Hispanic White patients, for the timeliness domain, Hispanic [β=-2.82; 95% confidence interval (CI), -5.42 to -0.39] and Asian (β=-6.65; 95% CI, -9.44 to -3.87) patients had significant lower adjusted mean score for getting care quickly. For the realized access domain, Asian (β=-5.78; 95% CI, -8.51 to -3.05) and NHB patients (β=-3.18; 95% CI, -5.50 to -0.87) had significantly lower adjusted mean score for getting needed prescription drugs compared with non-Hispanic White patients. Among NHB patients, a 5-Unit increase in getting needed care quickly was associated with higher odds of earlier CRC stage at diagnosis (odds ratio, 1.06; 95% CI, 1.01-1.10).

CONCLUSION

There are racial/ethnic disparities in PE with timeliness and realized access to care preceding a CRC diagnosis. Among NHB patients, poor experiences with timeliness and realized access of care may be associated with later stage at diagnosis.

The Relationship Between Primary Care Physician Burnout and Patient-Reported Care Experiences: a Cross-sectional Study

BACKGROUND

Primary care physician (PCP) burnout is prevalent and on the rise. Physician burnout may negatively affect patient experience of care.

OBJECTIVE

To identify the direct impact of PCP burnout on patient experience in various domains of care.

DESIGN

A cross-sectional observational study using physician well-being (PWB) surveys collected in 2016-2017, linked to responses from patient experience of care surveys. Patient demographics and practice characteristics were derived from the electronic health record. Linked data were analyzed at the physician level.

SETTING

A large non-profit multi-specialty ambulatory healthcare organization in northern California.

PARTICIPANTS

A total of 244 physicians practicing internal medicine or family medicine who responded to the PWB survey (response rate 72%), and 30,701 completed experience surveys from patients seeing these physicians.

MEASUREMENTS

Burnout was measured with a validated single-item question with a 5-point scale ranging from (1) enjoy work to (5) completely burned out and seeking help. Patient experience of patient-provider communication, access, and overall rating of provider was measured with Clinician & Group Consumer Assessment of Healthcare Providers & Systems (CG-CAHPS) survey. Patient experience scores (0-100 scale) were adjusted for age, gender, race/ethnicity, and English proficiency.

RESULTS

Physician burnout had a negative impact on patient-reported experience of patient-provider communication but not on access or overall rating of providers. A one-level increase in burnout was associated with 0.43 decrease in adjusted patient-provider communication experience score (P < 0.01).

LIMITATIONS

Data came from a single large healthcare organization. Patterns may differ for small- and mid-sized practices.

CONCLUSION

Physician burnout adversely affects patient-provider communication in primary care visits. Efforts to improve physician work environments could have a meaningful positive impact on patient experience as well as physician well-being.

Long-term Clinical Outcomes Among Responders and Nonresponders to the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) Survey

Rationale & Objective

The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) survey, introduced into the End-Stage Renal Disease Quality Incentive Program, is the only patient-reported outcome currently used for value-based reimbursement in dialysis. Current response rates are ∼30% and differences in long-term clinical outcomes between survey responders and nonresponders are unknown.

Study Design

Retrospective cohort study.

Setting & Participants

Patients from all Dialysis Clinic Incorporated facilities from across the United States who met survey eligibility (aged ≥18 years and had been treated at their facility for at least 3 months).

Exposures

Patient-level demographic, clinical, and treatment-related characteristics.

Outcomes

Mortality, all-cause hospitalization, and kidney transplantation.

Analytical Approach

Time-to-event analyses using competing-risks models. Sensitivity analyses performed after multiple imputation for missing covariate data.

Results

Among 10,395 eligible patients, 3,794 (36%) responded to the survey. During a median follow-up of 33 months, 4,588 patients died, 7,638 patients were hospitalized at least once, and 789 patients received a transplant. In multivariable models, survey response was associated with lower mortality (subdistribution hazard ratio [sHR], 0.80; 95% CI, 0.75-0.86) and hospitalization (sHR, 0.94; 95% CI, 0.89-0.99) and higher likelihood for a kidney transplant (sHR, 1.27; 95% CI, 1.10-1.46). Results were consistent across sensitivity analyses after multiple imputation for missing covariates.

Limitations

Small amount of missing covariate data, baseline covariate data assigned at the first month of the 3-month survey administration period, reasons for nonresponse unknown.

Conclusions

Response to the ICH CAHPS survey is associated with lower risk for mortality and hospitalization and higher likelihood for kidney transplantation. These findings suggest that survey responders are healthier than nonresponders, emphasizing the need for caution when interpreting facility-level survey results to inform quality improvement and public policy efforts and the critical need to better capture patient-reported outcomes from more vulnerable patients.

Job type and other socio-demographic factors associated with participation in a national, cross-sectional study of Danish employees

OBJECTIVES

Participation in population-based surveys and epidemiological studies has been declining over the years in many countries. The aim of this study was to examine the association between job type and participation in the work environment and health in Denmark survey with/without taking into account other socio-demographic factors.

DESIGN

Cross-sectional survey using questionnaire data on working environment and registry data on job type, industry and socio-economic variables.

SETTING

The work environment and health study.

PARTICIPANTS

A total of 50 806 employees (15 767 in a stratified workplace sample; 35 039 in a random sample) working at least 35 hours/month and earning at least 3000 Danish Krones.

OUTCOME MEASURES

The outcome was participation (yes/no) and logistic regression was used to estimate the OR for participation with 95% CI.

RESULTS

In the random sample, women were more likely to participate than men, and married/non-married couples were more likely to participate than persons living alone or more families living together. Participation increased with higher age, higher annual personal income, higher education and Danish origin, and there were marked differences in participation between job types and geographical regions. For armed forces, craft and related trade workers, and skilled agricultural, forestry and fishery workers, the association between job type and participation was strongly attenuated after adjustment for sex and age. Additional adjustment for annual income, education, cohabitation, country of origin and geographical region generally attenuated the association between job type and participation. Similar results were found in the stratified workplace sample.

CONCLUSION

In this population of Danish employees, participation varied across types of jobs. Some but not all the variation between job types was explained by other socio-demographic factors. Future studies using questionnaires may consider targeting efforts to (sub-)populations, defined by job type and other factors, where response probability is particularly important.

Adapting Patient Experience Data Collection Processes for Lower Literacy Patient Populations Using Tablets at the Point of Care

BACKGROUND

Patient experience surveys are widely used to capture the patient-reported quality of care and are increasingly being used for formal reporting purposes. There is evidence that certain patient subgroups are less likely to respond to traditional CAHPS surveys. As patient-facing technologies become more common, it is important to examine whether tablet-based patient experience surveys have the potential to promote responses from more diverse populations.

OBJECTIVES

To develop, gain perspectives about, and pilot an English and Spanish low-literacy adaptation of the Consumer Assessment of Healthcare Providers & Systems Clinician & Group Survey (CG-CAHPS) administered on a tablet device at the point of care.

RESEARCH DESIGN

Cognitive testing and evaluation of a quality improvement pilot comparing a tablet-based adaptation and traditional paper-based versions of the CG-CAHPS survey.

SUBJECTS

English-speaking and Spanish-speaking patients receiving primary care in an urban community clinic.

MEASURES

To compare the acceptability of low-literacy tablet-based and traditional paper-based patient experience surveys, we examined the concordance of responses between survey modes and preferences for modality, as well as perspectives on usability and reporting care experiences. We examined demographic differences in responses to tablet-based versus mailed surveys from a quality improvement pilot.

RESULTS

The majority of cognitive interview participants preferred a low-literacy, tablet-based survey over a paper-based survey with traditional wording. In a quality improvement pilot comparing tablet-based administration at the point of care versus mailed surveys, respondents to the tablet-based survey were more likely to be younger and Latino.

CONCLUSIONS

If designed with patient input, tablet-based surveys have the potential to improve the collection of patient experience data among diverse populations.

Are Caregivers Who Respond to the Child HCAHPS Survey Reflective of All Hospitalized Pediatric Patients?

OBJECTIVES

The Child Hospital Consumer Assessment of Healthcare Providers and Systems (C-HCAHPS) survey was developed to measure satisfaction levels of pediatric inpatients' caregivers. Studies in adults have revealed that certain demographic groups (people of color or who are multiracial and people with public insurance) respond to surveys at decreased rates, contributing to nonresponse bias. Our primary goal was to determine if results from the C-HCAHPS survey accurately reflect the intended population or reveal evidence of nonresponse bias. Our secondary goal was to examine whether demographic or clinical factors were associated with increased satisfaction levels.

METHODS

This was a retrospective cohort study of responses (n = 421) to the C-HCAHPS survey of patients admitted to a tertiary-care pediatric hospital between March 2016 and March 2017. Respondent demographic information was compared with that of all hospital admissions over the same time frame. Satisfaction was defined as "top-box" scores for questions on overall rating and willingness to recommend the hospital.

RESULTS

Caregivers returning surveys were more likely to be white, non-Hispanic, and privately insured (P < .001). Caregivers with the shortest emergency department wait times were more likely to assign top-box scores for global rating (P = .025). We found no differences in satisfaction between race and/or ethnicity, length of stay, insurance payer, or total cost.

CONCLUSIONS

Caregivers who identified with underrepresented minority groups and those without private insurance were less likely to return surveys. Among the surveys received, short emergency department wait time and older age were the only factors measured that were associated with higher satisfaction. Efforts to increase patient satisfaction on the basis of satisfaction scores may exacerbate existing disparities in health care.

The Association Between Residency Learning Climate and Inpatient Care Experience in Clinical Teaching Departments in the Netherlands

PURPOSE

To examine the association between residency learning climate and inpatient care experience.

METHOD

The authors analyzed 1,201 evaluations of the residency learning climate (using the Dutch Residency Educational Climate Test questionnaire) and 6,689 evaluations of inpatient care experience (using the Consumer Quality Index Inpatient Hospital Care questionnaire) from 86 departments across 15 specialties in 18 hospitals in the Netherlands between 2013 and 2014. The authors used linear hierarchical panel analyses to study the associations between departments' overall and subscale learning climate scores and inpatient care experience global ratings and subscale scores, controlling for respondent- and department-level characteristics and correcting for multiple testing.

RESULTS

Overall learning climate was not associated with global department ratings (b = 0.03; 95% confidence interval -0.17 to 0.23) but was positively associated with specific inpatient care experience domains, including communication with doctors (b = 0.11; 0.02 to 0.20) and feeling of safety (b = 0.09; 0.01 to 0.17). Coaching and assessment was positively associated with communication with doctors (b = 0.22; 0.08 to 0.37) and explanation of treatment (b = 0.22; 0.08 to 0.36). Formal education was negatively associated with pain management (b = -0.16; -0.26 to -0.05), while peer collaboration was positively associated with pain management (b = 0.14; 0.03 to 0.24).

CONCLUSIONS

Optimizing the clinical learning environment is an important step toward ensuring high-quality residency training and patient care. These findings could help clinical teaching departments address those aspects of the learning environment that directly affect patient care.

Using predicted Spanish preference to target bilingual mailings in a mail survey with telephone follow-up

OBJECTIVE

Spanish-preferring Medicare beneficiaries are underrepresented in national patient experience surveys. We test a method for improving their representation via higher response rates.

DATA SOURCES/STUDY SETTING

2009-2010 Medicare CAHPS surveys; Medicare population.

STUDY DESIGN

We used surname and address to predict Spanish-language preference for a national sample of 177 139 beneficiaries. We randomized half of the 10 000 non-Puerto Rico beneficiaries with the highest predicted probabilities of Spanish preference (>10 percent) to bilingual mailings (intervention) and half to standard English-only mailings (control).

DATA COLLECTION

Medicare CAHPS Survey data were collected through mail surveys with telephone follow-up of nonrespondents.

PRINCIPAL FINDINGS

Mail response rate was higher for intervention (28.7 percent) than control (23.9 percent) (P < 0.0001); phone response rates among mail nonrespondents were similar in intervention and control arms (15.8 percent vs 15.7 percent, P = 0.90). Targeted bilingual mailings induced 6.5 percent of those who would not have responded to respond by mail and 54.0 percent of those who would have responded in English to respond in Spanish. Beneficiaries with greater Spanish probabilities showed greater increases in response rates, a higher proportion of responses in Spanish, and lower control response rates among.

CONCLUSIONS

Targeted bilingual mailing of mixed-mode surveys using commonly available surname and address information can efficiently increase representation of this underrepresented group.

Evaluation of non-response to the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) survey

Background

The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) Survey is the first patient reported outcome measure included in the U.S. Medicare End Stage Renal Disease Quality Incentive Program. Administered twice yearly, it assesses in-center dialysis experience and survey responses are tied to dialysis facility payments. Low response rates, currently approximately 35%, raise concern for possible underrepresentation of patient groups.

Methods

Cross-sectional analysis of survey administration in 2012 to all in-center hemodialysis patients in Dialysis Clinic, Inc. (DCI) facilities nationally over 18 years old who received hemodialysis at their facility for at least 3 months. Patient-level covariates included demographic, clinical, laboratory, and functional characteristics. Random effects multivariable logistic regression was used to assess survey non-response.

Results

Among 11,055 eligible patients 6541 (59%) were non-responders. Of the remaining 4514 responders, 549 (14%) surveys were not usable due to presence of proxy help or incomplete responses. Non-responders were more likely to be men, non-white, younger, single, dual Medicare/Medicaid eligible, less educated, non-English speaking, and not active on the transplant list; non-responders had longer ESRD vintage, lower body mass index, lower serum albumin, worse functional status, and more hospitalizations, missed treatments, and shortened treatments. Similar associations were found using more parsimonious multivariable analyses and after imputing missing data.

Conclusions

Non-responders to the ICH CAHPS significantly differed from responders, broadly spanning individuals with fewer socioeconomic advantages and greater illness burden, raising limitations in interpreting facility survey results. Future research should assess reasons for non-response to improve ICH CAHPS generalizability and utility.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3618-4) contains supplementary material, which is available to authorized users.

Effects of Caregiver and Decedent Characteristics on CAHPS Hospice Survey Scores

CONTEXT

The Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Hospice Survey assesses the care experiences of hospice patients and their families. Public reporting of hospice performance on these survey measures began in February 2018.

OBJECTIVES

Develop an appropriate case-mix adjustment (CMA) model to allow for fair comparisons between hospices.

METHODS

We analyzed CAHPS Hospice Survey data reflecting experiences of 915,442 patients who received care from 2513 hospice programs between April 2015 and March 2016. Decedent and caregiver characteristics were identified for inclusion in CMA based on their variation across hospices (as measured by intraclass correlation coefficients [ICCs]) and how predictive they were of responses to survey questions (as assessed by linear regression).

RESULTS

The final CMA model included decedent age, payer for hospice care, primary diagnosis, length of final episode of hospice care, caregiver age, caregiver education, relationship to decedent, survey language/language spoken at home, and response percentile. The characteristics that varied most across hospices were language (ICC = 0.48 for Spanish survey or home language) and payer for hospice care (ICC = 0.42 for Medicare only; ICC = 0.35 for Medicare and private insurance). The characteristics that were most predictive of caregivers' survey responses were payer for hospice care, caregiver education, and survey language/language spoken at home. Lack of appropriate adjustment would incorrectly rank hospices by 1.2-5.4 percentile points.

CONCLUSION

To ensure fair comparisons across hospices, CAHPS Hospice Survey measure scores should be adjusted for several caregiver and decedent characteristics.

Are There Racial Disparities in Family-Reported Experiences of Care in Inpatient Pediatrics?

Despite increased emphasis on patient satisfaction as a quality measure in health care, little is known about the influence of race in parent-reported experience of care in pediatrics. This study evaluates the association of race with patient satisfaction scores in an inpatient pediatric tertiary care hospital in one year. Risk-adjusted multivariable logistic regression was performed to evaluate the association of minority race with the likelihood to provide a top-box (=5) satisfaction score for 38 individual questions across 8 domains. Of the 904 participants, 269 (29.8%) identified as belonging to a minority race. Parents of minority children reported 30% to 50% lower satisfaction across questions related to well-established themes of interpersonal communication and cultural competency. Overall, minorities also reported lower satisfaction for the domain of nursing care (odds ratio 0.7, P = .016). These findings suggest a need for training and interventions to improve communication and mitigate disparities in how minority patients and their families perceive pediatric care.

Differences between Proxy and Patient Assessments of Cancer Care Experiences and Quality Ratings

OBJECTIVE

To assess the impact of proxy survey responses on cancer care experience reports and quality ratings.

DATA SOURCES/STUDY SETTING

Secondary analysis of data from Cancer Care Outcomes Research and Surveillance (CanCORS). Recruitment occurred from 2003 to 2005.

STUDY DESIGN

The study was a cross-sectional observational study. The respondents were patients with incident colorectal or lung cancer or their proxies.

DATA COLLECTION/EXTRACTION METHODS

Analyses used linear regression models with an independent variable for proxy versus patient responses as well as study site and clinical covariates. The outcomes were experiences with medical care, nursing care, care coordination, and care quality rating. Multiple imputation was used for missing data.

PRINCIPAL FINDINGS

Among 6,471 respondents, 1,011 (16 percent) were proxies. The proportion of proxy respondents varied from 6 percent to 28 percent across study sites. Adjusted proxy scores were modestly higher for medical care experiences (+1.28 points [95 percent CI:+ 0.05 to +2.51]), but lower for nursing care (-2.81 [95 percent CI: -4.11 to -1.50]) and care coordination experiences (-2.98 [95 percent CI: -4.15 to -1.81]). There were no significant differences between adjusted patient and proxy ratings of quality.

CONCLUSIONS

Proxy responses have small but statistically significant differences from patient responses. However, if ratings of care are used for financial incentives, such differences could be exaggerated across practices or areas if proxy use varies.

Development and validation of a new patient experience tool in patients with serious illness

Background

Patients with serious chronic illnesses face increasingly complex care and are at risk of poor experience due to a fragmented health system. Most current patient experience tools are not designed to address the unique care aspects of this population and the few that exist are delivered too late in the disease trajectory and are not administered longitudinally which makes them less useful across settings.

Methods

We developed a new tool designed to address these gaps. The 25 item scale was tested and refined using randomly cross-validated exploratory and confirmatory factor analyses. Participants were not yet hospice eligible but sick enough to receive benefits of a supportive care approach in the last 2 to 3 years of life. Full information maximum likelihood models were run to confirm the factor structure developed in exploratory analyses. Goodness-of-fit was assessed with the Comparative Fit Index, the Tucker-Lewis Index, and the Root Mean Square Error of Approximation. Test-retest reliability was assessed with the intraclass correlation coefficient and internal consistency of the final scale was examined using Cronbach’s alpha.

Results

Exploratory factor analysis revealed three domains — Care Team, Communication, and Care Goals — after removing weak loading and cross loading items. The initial three domain measurement model suggested in the development cohort was tested in the validation cohort and exhibited poor fit X² (206) = 565.37, p < 0.001; CFI = 0.879; TLI = 0.864; RMSEA = 0.076. After model respecification, including removing one additional item and allowing paths between theoretically plausible error terms, the final 21 item tool exhibited good fit X² (173) = 295.63, p < 0.001; CFI = 0.958; TLI = 0.949; RMSEA = 0.048. Cronbach’s alpha revealed high reliability of each domain (Care Team = 0.92, Communication = 0.83, Care Goals = 0.77) and the entire scale (α = 0.91). ICC showed adequate test-retest validity (ICC = 0.58; 95% CI: 0.52–0.65) of the full scale.

Conclusions

When administered earlier in the chronic illness trajectory, a new patient experience scale focused on care teams across settings, communication, and care goals, displayed strong reliability and performed well psychometrically.

Trial registrations

This trial (NCT01746446) was registered at ClinicalTrials.gov on November 27, 2012 (retrospectively registered).

Electronic supplementary material

The online version of this article (doi:10.1186/s12904-016-0172-x) contains supplementary material, which is available to authorized users.

Evidence of non-response bias in the Press-Ganey patient satisfaction survey

Background

Measures of patient satisfaction have gained prominence in recent years as changes to the American health care system have led to the incorporation of such metrics into reimbursement models. The response rate for widely-used outpatient satisfaction metrics and variables influencing the probability of survey nonresponse remain largely unknown.

Methods

We reviewed all unique adult patients (16,779) who completed an outpatient encounter in the Department of Orthopaedic surgery at our academic institution from 1/1/13 to 10/24/13. Survey data was linked to each clinic visit, and patient factors including age, sex, insurance type, zip code, and orthopaedic subspecialty visited were recorded.

The overall survey response rate was calculated. Logistic regression was performed, and unadjusted and adjusted odds ratios of patients’ probability of responding to the Press-Ganey survey were calculated.

Results

Two thousand seven hundred sixty two (16.5 %) of individuals completed a Press-Ganey patient satisfaction survey and 14017 patients did not respond. For those patients considered responders, 906 patients (32.8 %) did not complete all the survey items. Among these 906 patients, the mean number of missing items was 2.24 (Standard Deviation SD: 2.19).

Age, sex, insurance type, and orthopaedic subspecialty were all found to be associated with the odds of responding to our patient satisfaction survey. Advancing age increased the odds of responding to the survey (Adjusted Odds Ratio (OR) = 3.396 for ≥65 vs. 18–29, p < 0.001). Several variables were associated with a decreased odds of survey response, and included male sex (Adjusted OR = 0.782 for Males vs. Females, p < 0.001), insurance type (Adjusted OR = 0.311 for Medicaid/Self-Pay vs. Private), and subspecialty type (Adjusted OR = 0.623 for Trauma vs. Adult Reconstruction).

Conclusions

The response rate to the Press-Ganey Medical Practice Survey of outpatient satisfaction is low in an orthopaedic outpatient population, and furthermore, is impacted by patient characteristics such as age, sex, insurance type, and type of orthopaedic subspecialist encountered. The findings of the present study should inform future non-response weighting procedures in this area. More research is needed to assess non-response bias—including follow-up studies of non-respondents—in order to more accurately measure of patient satisfaction.

Patient satisfaction after acute admission for psychosis

Background Measuring patient satisfaction in mental health care potentially provides valuable information, but studies in acutely admitted psychosis patients are scarce. Aims The aims were to assess satisfaction among patients acutely admitted with psychosis, to compare satisfaction in voluntarily versus involuntarily admitted patients, and to assess the influence of symptom load and insight. Methods The UKU Consumer Satisfaction Rating Scale (UKU-ConSat) was used. A total of 104 patients completed the UKU-ConSat at discharge/follow-up (between 6-11 weeks after admittance if not discharged earlier) (mean duration of stay 4 weeks), thus corresponding to the end of the acute treatment phase. Results A total of 88.4% had total scores above zero (satisfied). Only three of the eight single items were statistically significantly different among patients admitted voluntarily versus involuntarily, and only the information item score remained significantly different in adjusted analyses. Insight level at admittance, and an increasing level of insight during the acute phase were positively associated with patient satisfaction, whereas levels and changes in positive and negative psychosis symptoms were indirectly related to satisfaction via this process of insight. Conclusions The vast majority of the acutely admitted patients were satisfied with treatment. There were few differences between the involuntarily and voluntarily admitted patient groups, except that the involuntary care group was clearly less satisfied with the information provided. Poor insight had a major negative impact on treatment satisfaction in psychosis. The provision of sufficient and adequate information is an important target for mental health care service improvement.

Comparison of Data Quality for Reports and Ratings of Ambulatory Care by African American and White Medicare Managed Care Enrollees

Objective: Compare missing data and reliability of health care evaluations between African Americans and Whites in Medicare managed care health plans. Method: Consumer Assessment of Healthcare Providers and Systems (CAHPS) 3.0 health plan survey data collected from 109,980 Medicare managed care enrollees (101,189 Whites, 8,791 African Americans) in 321 plans. Participants self-administered the survey and four single-item global ratings of care. Results: Missing data rates were significantly higher for African Americans than Whites on all CAHPS items ( p < .0001). Internal consistency reliability estimates for the CAHPS scales did not differ significantly between African Americans and Whites, but plan-level reliability estimates for the scales and global rating items were significantly lower for African Americans than Whites. Discussion: Higher missing data rates and lower plan-level reliability estimates for African American Medicare managed care enrollees suggest caution in making race/ethnicity comparisons. Future efforts are needed to enhance the quality of data collected from older African Americans.

Understanding the determinants of patient satisfaction with surgical care using the Consumer Assessment of Healthcare Providers and Systems surgical care survey (S-CAHPS)

BACKGROUND

Patient satisfaction has been emphasized increasingly in all aspects of medicine, including the imposition of financial penalties for underperformance. Current measures of patient satisfaction, however, do not address aspects specific to the care of operative patients. Therefore, our aim was to examine the recently validated Consumer Assessment of Healthcare Providers and Systems (S-CAHPS) to determine which aspects of perioperative care are predictive of satisfaction with the surgeon.

METHODS

All patients undergoing a general surgery operation at our institution during a 5-month period were sent a modified S-CAHPS within 3 days of discharge. Patients were then divided into 2 groups: those who rated their surgeon as the best possible and those giving a lower rating. Univariate and multivariate analyses were used to determine predictors of satisfaction with operative care. S-CAHPS results were then compared with other satisfaction measures in a subset of patients.

RESULTS

The response rate was 45.3% (456/1,007). The average age was 59 ± 16 years, duration of stay was 4.1 ± 6.6 days, and 23% underwent unscheduled operations. A total of 72% of patients rated their surgeon as the best surgeon possible. On multivariate analysis, preoperative communication and attentiveness on the day of operation were the most important determinants of overall surgeon rating. S-CAHPS scores correlated with other standard measures of satisfaction (HCAHPS scores).

CONCLUSION

S-CAHPS is a novel operative satisfaction tool and is feasible to administer to patients undergoing general operative procedures. Surgeon characteristics most predictive of high patient satisfaction are effective preoperative communication and attentiveness on the day of operation.

Does Race/Ethnicity or Socioeconomic Status Influence Patient Satisfaction in Pediatric Surgical Care?

OBJECTIVE

To evaluate patient satisfaction in outpatient pediatric surgical care and assess differences in scores by race/ethnicity and socioeconomic status (SES).

STUDY DESIGN

Observational, cross-sectional analysis.

SETTING

Outpatient pediatric surgical specialty clinics at a tertiary academic center.

SUBJECT AND METHODS

Families of patients received a patient satisfaction survey following their initial care visit in 2012. Mean scores were calculated and compared by child race/ethnicity and insurance type, where insurance with medical assistance (MA) served as a proxy for low SES. Kruskal-Wallis tests were used to compare scores between groups. Surveys were dichotomized to low and high scorers, and multivariate logistic regression was used to calculate the likelihood of high satisfaction.

RESULTS

Of 527 surveys completed, 132 (25%) were for children with MA and 143 (27%) were for racial/ethnic minority children. The overall satisfaction score for all specialties was 84.8, which did not significantly differ by SES (P = .98) or minority status (P = .52). The survey item with the highest score in both SES groups was "degree to which provider talked with you using words you could understand" (overall mean 91.94, P = .23). Multivariate analysis showed that patient age, sex, race/ethnicity, insurance type, neighborhood SES, neighborhood diversity, or surgical department did not significantly influence satisfaction.

CONCLUSION

This is the first study to evaluate the relationship between SES and race/ethnicity with patient satisfaction in outpatient pediatric surgical specialty care. In this analysis, no disparities were identified in the patient experience by individual- or community-level factors. Although the survey methodologies may be limited, these findings suggest that provision of care in pediatric surgical specialties can be simultaneously equitable, culturally competent, and family centered.

Can patients reliably identify safe, high quality care?

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is a publicly reported tool that measures patient satisfaction. As both patients and Centers for Medicare & Medicaid Services (CMS) reimbursement rely on survey results as a metric of quality of care, we reviewed the current literature to determine if patient satisfaction correlates with quality, safety, or patient outcomes. We found varying associations between safety culture, process of care measure compliance, and patient outcomes with patient satisfaction on the HCAHPS survey. Some studies found inverse relationships between quality and safety metrics and patient satisfaction. The measure that most reliably correlated with high patient satisfaction was low readmission rate. Future studies using patient specific data are needed to better identify which factors most influence patient satisfaction and to determine if patient satisfaction is a marker of safer and better quality care. Furthermore, the HCAHPS survey should continue to undergo evaluations to assure it generates predictable results.

People with limiting long-term conditions report poorer experiences and more problems with hospital care

Background

Long-term conditions have a significant impact on individuals, their families, and the health service. As people with these conditions represent a high proportion of hospital admissions, investigating their experiences of inpatient care has become an important area of investigation. We conducted a secondary analysis of the NHS adult inpatient survey for England to compare the hospital experiences of three groups of patients: those without long-term conditions, those with a single long-term condition, and those with multiple long-term conditions. We were particularly interested in the extent to which these patients received self-management support from hospital staff, so we developed a brief summary tool drawn from salient questions in the survey to aid the comparison.

Methods

Analysis of data from the 2011 national adult inpatient survey (n = 65,134) to compare the experiences of three groups of patients: those with no limiting long-term conditions (No-LLTC), those with one limiting long-term condition (S-LLTC), and those with two or more limiting long-term conditions (M-LLTC). The main outcome measure was patients’ self-reports of their experience of inpatient care, including staff-patient interactions, information provision, involvement in decisions and support for self-care and overall ratings of care. A short form scale, the Oxford Patient Involvement and Experience scale (OxPIE) was developed from the adult inpatient survey and used to compare the groups using logistic regression.

Results

There were significant differences between the No-LLTC group in comparison to both the S-LLTC and M-LLTC groups. Patients with limiting long-term conditions reported significantly worse hospital experiences than those without, as measured by OxPIE: S-LLTC odds ratio = 1.23, 95% CI 1.03-1.48; M-LLTC odds ratio = 1.64, 95% CI 1.19 – 2.26. Responses to a single global rating question were more positive but not strongly correlated with OxPIE.

Conclusions

Patients with LLTCs were more critical of their inpatient care than those with no LLTCs. Those with more than one long-term condition reported worse experiences than those with a single limiting condition. Simple rating questions may not be sufficiently sensitive to reflect important aspects of patients’ experience.

Whose experience is measured? A pilot study of patient satisfaction demographics in pediatric otolaryngology

OBJECTIVES/HYPOTHESIS

Despite a national emphasis on patient-centered care and cultural competency, minority and low-income children continue to experience disparities in health care quality. Patient satisfaction scores are a core quality indicator. The objective of this study was to evaluate race and insurance-related disparities in parent participation with pediatric otolaryngology satisfaction surveys.

STUDY DESIGN

Observational analysis of patient satisfaction survey respondents from a tertiary pediatric otolaryngology division.

METHODS

Demographics of survey respondents (Press Ganey Medical Practice Survey©) between January and July 2012 were compared to a clinic comparison group using t test and chi-square analyses. Multivariate logistic regression analyses were performed to assess likelihood to complete a survey based on race or insurance status.

RESULTS

A total of 130 survey respondents were compared to 1,251 patients in the comparison group. The mean patient age for which the parent survey was completed was 5.7 years (6.1 years for the comparison group, P = 0.18); 59.2% of children were ≤ 5 years old. Relative to the comparison group, survey respondents were more often white (77.7% vs. 58.1%; P <0.001) and privately insured (84.6% vs. 60.8%; P <0.001). Similarly, after controlling for confounding variables, parents of children who were white (OR 1.8, 95% CI 1.13-2.78, P = 0.013) or privately insured (OR 2.9, 95% CI 1.74-4.85, P <0.001) were most likely to complete a survey.

CONCLUSION

Methods to evaluate satisfaction did not capture the racial or socioeconomic patient distribution within this pediatric division. These findings challenge the validity of applying patient satisfaction scores, as currently measured, to indicate health care quality. Future efforts to measure and improve patient experience should be inclusive of a culturally diverse population.

The impact of demographic characteristics on nonresponse in an ambulatory patient satisfaction survey

BACKGROUND

Despite the increasing use of patient satisfaction data for compensation and comparison of performance, low response rate and potential bias of the respondents pose a significant threat to validity of the data. The demographic profiles of respondents and nonrespondents to a patient satisfaction survey in the ambulatory care setting were compared to explore the impact ofnonresponse bias.

METHODS

Patient satisfaction survey data were collected from October through December 2010 for outpatient facilities at a large academic medical center. The association between respondent characteristics and satisfaction ratings on three dimensions of the clinical care process--(1) interpersonal communication (clarity of language), (2) service delivery (overall care during visit), and (3) likelihood of recommending practice to others-were assessed with bivariate and multivariate linear regression. Weighted analyses were performed to examine the impact of nonresponse.

RESULTS

Surveys were mailed to 15,549 patients, of which 4,952 (32%) were returned. Respondents had greater proportions of elderly, female, and English speakers. Bivariate analyses showed significant difference in satisfaction ratings by age, language, and insurance type. Multivariate regression analysis showed significant confounding across variables. On the basis of the calculated weighted means, mean satisfaction ratings were discrepant for language and age; however, the overall satisfaction ratings for each dimension were minimally affected.

CONCLUSION

Nonresponse rates and satisfaction ratings differed by age, language, and insurance type. However, if it is assumed that nonrespondents within these demographic groups have similar satisfaction ratings as respondents, then nonresponse levels appear to have minimal effects on overall satisfaction ratings.

Association between alcohol screening scores and mortality in black, Hispanic, and white male veterans

BACKGROUND

Scores on the Alcohol Use Disorders Identification Test Consumption (AUDIT-C) questionnaire are associated with mortality, but whether or how associations vary across race/ethnicity is unknown.

METHODS

Self-reported black (n = 13,068), Hispanic (n = 9,466), and white (n = 182,688) male Veterans Affairs (VA) outpatients completed the AUDIT-C via mailed survey. Logistic regression models evaluated whether race/ethnicity modified the association between AUDIT-C scores (0, 1 to 4, 5 to 8, and 9 to 12) and mortality after 24 months, adjusting for demographics, smoking, and comorbidity.

RESULTS

Adjusted mortality rates were 0.036, 0.033, and 0.054, for black, Hispanic, and white patients with AUDIT-C scores of 1 to 4, respectively. Race/ethnicity modified the association between AUDIT-C scores and mortality (p = 0.0022). Hispanic and white patients with scores of 0, 5 to 8, and 9 to 12 had significantly increased risk of death compared to those with scores of 1 to 4; Hispanic ORs: 1.93, 95% CI 1.50 to 2.49; 1.57, 1.07 to 2.30; 1.82, 1.04 to 3.17, respectively; white ORs: 1.34, 95% CI 1.29 to 1.40; 1.12, 1.03 to 1.21; 1.81, 1.59 to 2.07, respectively. Black patients with scores of 0 and 5 to 8 had increased risk relative to scores of 1 to 4 (ORs 1.28, 1.06 to 1.56 and 1.50, 1.13 to 1.99), but there was no significant increased risk for scores of 9 to 12 (ORs 1.27, 0.77 to 2.09). Post hoc exploratory analyses suggested an interaction between smoking and AUDIT-C scores might account for some of the observed differences across race/ethnicity.

CONCLUSIONS

Among male VA outpatients, associations between alcohol screening scores and mortality varied significantly depending on race/ethnicity. Findings could be integrated into systems with automated risk calculators to provide demographically tailored feedback regarding medical consequences of drinking.

Reliability adjustment for reporting hospital outcomes with surgery

BACKGROUND

Reliability adjustment, a novel technique for quantifying and removing statistical "noise" from quality rankings, is becoming more widely used outside surgery. We sought to evaluate its impact on hospital outcomes assessed with the American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP).

METHODS

We used prospective, clinical data from the ACS-NSQIP to identify all patients undergoing colon resection in 2007 (n = 181 hospitals, n = 18,455 patients). We first used standard NSQIP techniques to generate risk-adjusted mortality and morbidity rates for each hospital. Using hierarchical logistic regression models, we then adjusted these for reliability using empirical Bayes techniques. To evaluate the impact of reliability adjustment, we first estimated the extent to which hospital-level variation was reduced. We then compared hospital mortality and morbidity rankings and outlier status before and after reliability adjustment.

RESULTS

Reliability adjustment greatly diminished apparent variation in hospital outcomes. For risk-adjusted mortality, there was a 6-fold difference before (1.4%-7.8%) and less than a 2-fold difference (3.2% to 5.7%) after reliability adjustment. For risk-adjusted morbidity, there was a 2-fold difference (18.0%-38.2%) before and a 1.5-fold difference (20.8%-34.8%) after reliability adjustment. Reliability adjustment had a large impact on hospital mortality and morbidity rankings. For example, with rankings based on mortality, 44% (16 hospitals) of the "best" hospitals (top 20%) were reclassified after reliability adjustment. Similarly, 22% (8 hospitals) of the "worst" hospitals (bottom 20%) were reclassified after reliability adjustment.

CONCLUSIONS

Reliability adjustment reduces variation due to statistical noise and results in more accurate estimates of risk-adjusted hospital outcomes. Given the risk of misclassifying hospitals and surgeons using standard approaches, this technique should be considered when reporting surgical outcomes.

Cost-related nonadherence to prescribed medication therapy among Medicare Part D beneficiaries with end-stage renal disease

PURPOSE

Medication nonadherence due to cost issues among community-dwelling patients with end-stage renal disease (ESRD) enrolled in Medicare prescription drug plans was evaluated.

METHODS

Demographic and health status data were collected on 1329 patients with ESRD enrolled in Medicare Part D prescription drug plans who responded to a Centers for Medicare and Medicaid Services consumer survey in early 2007. The survey data were assessed for self-reported cost-related nonadherence (CRN), defined as delaying or not filling a prescription due to cost concerns. Multivariate logistic regression analysis was performed to evaluate CRN risk relative to patient demographic characteristics, socioeconomic status, other chronic conditions, health behaviors, and access to health care services.

RESULTS

Overall, survey respondents with ESRD were significantly more likely than those without ESRD to report CRN in the prior six months (unadjusted odds ratio [OR], 2.34; 95% confidence interval [CI], 2.00-2.75). After controlling for potential confounding factors such as other chronic conditions, the data analysis continued to show a significant association between ESRD and an increased risk of CRN (adjusted OR, 1.23; 95% CI, 1.07-1.41). Black race and receipt of Medicare Part D Low-Income Subsidy assistance were significant independent predictors of CRN for respondents with ESRD.

CONCLUSION

In early 2007, 31% of survey respondents with ESRD enrolled in Medicare Part D drug plans reported CRN in the preceding six months. After adjusting for potential confounders, respondents with ESRD remained 23% more likely than respondents without ESRD to report CRN in the preceding six months.

Can claims-based data be used to recruit black and Hispanic subjects into clinical trials?

OBJECTIVE

Evaluate the accuracy of an algorithm at identifying ethnic minorities from administrative claims for enrollment into a clinical trial.

DATA SOURCES/STUDY SETTING

Claims data from a health benefits company.

STUDY DESIGN

We compared results of a three-step algorithm to self-reported race/ethnicity.

DATA COLLECTION/EXTRACTION METHODS

Using the algorithm, we identified subjects with high probability of being minority and ascertained self-reported race/ethnicity.

PRINCIPAL FINDINGS

We identified 164 subjects as likely minority based on our algorithm. Of these, 94 completed the survey and 87 identified themselves as black or Hispanic. The positive predictive value of the algorithm was 93 percent (CI: 85-97).

CONCLUSIONS

Claims data can be used to efficiently identify minorities for participation in clinical trials.

A Comparison of Alternative Indicators for the Risk of Nonresponse Bias

The response rate has played a key role in measuring the risk of nonresponse bias. However, recent empirical evidence has called into question the utility of the response rate for predicting nonresponse bias. The search for alternatives to the response rate has begun. The present article offers a typology for these indicators, briefly describes the strengths and weaknesses of each type, and suggests directions for future research. New standards for reporting on the risk of nonresponse bias may be needed. Certainly, any analysis into the risk of nonresponse bias will need to be multifaceted and include sensitivity analyses designed to test the impact of key assumptions about the data that are missing due to nonresponse.

Examining multiple sources of differential item functioning on the Clinician & Group CAHPS® survey

OBJECTIVE

To evaluate psychometric properties of a widely used patient experience survey.

DATA SOURCES

English-language responses to the Clinician & Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS®) survey (n = 12,244) from a 2008 quality improvement initiative involving eight southern California medical groups.

METHODS

We used an iterative hybrid ordinal logistic regression/item response theory differential item functioning (DIF) algorithm to identify items with DIF related to patient sociodemographic characteristics, duration of the physician-patient relationship, number of physician visits, and self-rated physical and mental health. We accounted for all sources of DIF and determined its cumulative impact.

PRINCIPAL FINDINGS

The upper end of the CG-CAHPS® performance range is measured with low precision. With sensitive settings, some items were found to have DIF. However, overall DIF impact was negligible, as 0.14 percent of participants had salient DIF impact. Latinos who spoke predominantly English at home had the highest prevalence of salient DIF impact at 0.26 percent.

CONCLUSIONS

The CG-CAHPS® functions similarly across commercially insured respondents from diverse backgrounds. Consequently, previously documented racial and ethnic group differences likely reflect true differences rather than measurement bias. The impact of low precision at the upper end of the scale should be clarified.

How do the experiences of Medicare beneficiary subgroups differ between managed care and original Medicare?

OBJECTIVE

To examine whether disparities in health care experiences of Medicare beneficiaries differ between managed care (Medicare Advantage [MA]) and traditional fee-for-service (FFS) Medicare.

DATA SOURCES

132,937 MA and 201,444 FFS respondents to the 2007 Medicare Consumer Assessment of Health Care Providers and Systems (CAHPS) survey.

STUDY DESIGN

We defined seven subgroup characteristics: low-income subsidy eligible, no high school degree, poor or fair self-rated health, age 85 and older, female, Hispanic, and black. We estimated disparities in CAHPS experience of care scores between each of these groups and beneficiaries without those characteristics within MA and FFS for 11 CAHPS measures and assessed differences between MA and FFS disparities in linear models.

PRINCIPAL FINDINGS

The seven subgroup characteristics had significant (p<.05) negative interactions with MA (larger disparities in MA) in 27 of 77 instances, with only four significant positive interactions.

CONCLUSION

Managed care may provide less uniform care than FFS for patients; specifically there may be larger disparities in MA than FFS between beneficiaries who have low incomes, are less healthy, older, female, and who did not complete high school, compared with their counterparts. There may be potential for MA quality improvement targeted at the care provided to particular subgroups.

Understanding nonresponse to the 2007 Medicare CAHPS survey

PURPOSE

The Medicare Consumer Assessments of Healthcare Providers and Systems (MCAHPS) survey, a primarily English-language mail survey with English and Spanish telephone follow-up, is the primary means of assessing the health care experiences of American seniors. We examine unit (whole survey) and item nonresponse for this survey to explore issues regarding surveying seniors about their health care.

DESIGN AND METHODS

We describe overall rates and analyze predictors of unit and item nonresponse for the 695,197 Medicare beneficiaries selected for the 2007 MCAHPS survey (335,249 unit respondents, 49% overall response rate).

RESULTS

Asians, African Americans, and Hispanics responded at adjusted response rates 7-17 percentage points lower than non-Hispanic Whites (p < .001 for each). Among seniors, response rates dropped beyond age 75. Asians and older beneficiaries were especially likely to respond by mail, and African Americans and Hispanics by phone. Breakoff from telephone surveys was most common among African Americans and older respondents. Among respondents, older age was the strongest predictor of item missingness (e.g., those 85 years and older failed to answer items at twice the rate of those aged 65-74 years, p < .001). Non-Hispanic Whites had lower rates of item missingness than other racial/ethnic groups (p < .001 for each; one-third lower than African Americans).

IMPLICATIONS

Survey research on older adults, especially regarding racial/ethnic disparities in health care, could benefit from improved response rates. These results suggest that targeted prenotification materials and campaigns, tailored follow-up, targeted Spanish mailings, Chinese translations/calls, and adjustments to telephone protocols may improve representation and response.

Market variations in intensity of Medicare service use and beneficiary experiences with care

OBJECTIVE

Examine associations between patient experiences with care and service use across markets.

DATA SOURCES/STUDY SETTING

Medicare fee-for-service (FFS) and managed care (Medicare Advantage [MA]) beneficiaries in 306 markets from the 2003 Consumer Assessments of Healthcare Providers and Systems (CAHPS) surveys. Resource use intensity is measured by the 2003 end-of-life expenditure index.

STUDY DESIGN

We estimated correlations and linear regressions of eight measures of case-mix-adjusted beneficiary experiences with intensity of service use across markets.

DATA COLLECTION/EXTRACTION

We merged CAHPS data with service use data, excluding beneficiaries under 65 years of age or receiving Medicaid.

PRINCIPAL FINDINGS

Overall, higher intensity use was associated (p<.05) with worse (seven measures) or no better care experiences (two measures). In higher-intensity markets, Medicare FFS and MA beneficiaries reported more problems getting care quickly and less helpful office staff. However, Medicare FFS beneficiaries in higher-intensity markets reported higher overall ratings of their personal physician and main specialist. Medicare MA beneficiaries in higher-intensity markets also reported worse quality of communication with physicians, ability to get needed care, and overall ratings of care.

CONCLUSIONS

Medicare beneficiaries in markets characterized by high service use did not report better experiences with care. This trend was strongest for those in managed care.

Geographic area variations in the Medicare health plan era

BACKGROUND

Prior research identified variations in care experiences across Medicare health plans (Medicare Advantage [MA]), but the relative amount of variation in MA and traditional fee-for-service (FFS) Medicare is unknown.

OBJECTIVES

Compare variation and correlations of beneficiary reports of care experiences across geographic areas in MA and FFS.

METHODS

Using the 2001 to 2004 Medicare CAHPS surveys, we analyzed 14 measures of care experiences and preventive services reported by 433,092 MA beneficiaries (82% response rate) and 244,731 in FFS (69% response rate). We estimated hierarchical regression models with random effects for state, hospital referral region, and plan, adjusting for respondent characteristics. We examined the relative variation in FFS and MA scores across areas and among individual MA plans, the correlation between FFS and MA scores across areas, and variability relative to average MA-FFS differences in scores.

RESULTS

Although MA and FFS scores are highly correlated, variation is greater in MA than FFS across states and local areas for almost all measures. MA plan variation within areas accounts for 25% to 50% of explained MA variation. MA-FFS differences are smaller than the standard deviations of differences across areas for 10 of 14 measures.

CONCLUSIONS

Relative performance between MA and FFS may differ across areas and locally between individual plans and FFS. Quality improvement initiatives should address local system factors that affect both MA and FFS, and identify organizational factors that make some MA plans more successful in improving quality.