Use of CAHPS® patient experience survey data as part of a patient-centered medical home quality improvement initiative

Beyond satisfaction in person-centered pharmacy services

Patient self-reported satisfaction is commonly used as an assessment of service experience and quality for community pharmacy services. This commentary discusses alternative foundational approaches to evaluating service experience and quality in patient-centered care. It describes historical and recent literature pertaining to the development and use of satisfaction measures for service design and patient experience assessment. It then highlights potential limitations of patient satisfaction as an assessment tool for patient-centeredness and patient experience identified in the pharmacy literature, which include criticisms that use of patient satisfaction may compromise accuracy in measuring quality due to factors such as patients having poor knowledge of and low expectations for quality and having a predisposition toward rating satisfaction highly when experiencing no-cost and/or unfamiliar services. Moreover, satisfaction measurements may change based on service exposure, with patient preferences for service offerings changing with increased service exposure and variation in patient-specific and environmental factors. After discussing limitations and criticism of patient self-reported satisfaction, we introduce alternative assessments methods which may facilitate more accurate assessments of patient experience and patient-centered pharmacy services such as patient-reported outcomes measures (PROMs), patient-reported experience measures (PREMs), and human-centered design techniques such as journey mapping, prototyping, and user testing to design and assess patient-centered pharmacy services. These alternative assessments are rooted in, or related to preferred implementation science approaches to establishing, evaluating, and sustaining pharmacy services.

Patient Comments on the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) Survey Reflect Improvements in Provider Behaviors From Coaching

GOAL

Patient experience survey data are used to examine the patient-centeredness of care, identify areas for improvement, and monitor interventions aimed to enhance the patient experience. Most healthcare organizations measure patient experience using Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. Studies have documented the use of CAHPS closed-ended survey responses for completing public reports, monitoring internal feedback and performance, identifying areas of improvement, and evaluating interventions to improve care. However, limited evidence exists on the utility of patients' comments on CAHPS surveys for evaluating provider-level interventions. To explore this potential, we examined comments on the CAHPS Clinician and Group (CG-CAHPS) 2.0 visit survey before and after a provider intervention. The "shadow coaching" intervention had been shown to improve provider performance and patient experience scores on the CG-CAHPS overall provider rating and provider communication composite.

METHODS

We examined how patient comments on the CG-CAHPS survey differed before and after shadow coaching of 74 providers. We described the valence (tone), content, and actionability of 1,935 comments-1,051 collected before coaching and 884 collected after coaching-to see how these aspects changed before and after providers were coached.

PRINCIPAL FINDINGS

Patient comments reflected improved CG-CAHPS scores after shadow coaching. The proportion of positive comments increased, and comments about doctors were more positive. Comments about time spent in the examination room decreased, apparently reflecting the decreased proportion of negative comments after coaching. Comments regarding three of the four aspects of provider communication asked on the CG-CAHPS survey were more positive after coaching (provider listens carefully, shows respect, spends enough time); the valence of comments about the fourth aspect (provider explains things in a way that is easy to understand) did not change. Also, comments describing an overall positive evaluation of the practice increased. Comments were generally less actionable after coaching, perhaps reflecting the increased positivity of the comments.

PRACTICAL APPLICATIONS

Patient comments collected before the provider intervention reflected overall improvements in provider behavior, as indicated by medium-to-large statistically significant improvements in CG-CAHPS composite scores. These results suggest that patient comments from the CG-CAHPS survey can be used as input for quality improvement or an evaluation of provider-level interventions. Tracking the valence and content of comments about providers before and after an intervention to improve care is a practical method to learn how provider behavior changes.

The association between housing stability and perceived quality of life among emerging adults with a history of homelessness

Many cities across the United States are experiencing homelessness at crisis levels, including rises in the numbers of unhoused emerging adults (18-25). Emerging adults experiencing homelessness may be at higher risk of experiencing negative outcomes, given that being unhoused increases risk for a variety of behaviors. To better understand the current living circumstances of emerging adults with a history of homelessness, as well as their perceptions about associations between housing stability and quality of life (QOL), we conducted 30 semi-structured in-depth interviews with individuals recruited from drop-in centers for youth experiencing homelessness. At the time of recruitment n=19 were stably housed and n=11 were unstably housed. Two coders analyzed these data inductively and deductively, using pre-identified domains and open coding. Coding reliability was assessed. Three main themes emerged, each with subthemes: 1) Housing quality (neighborhood safety, convenience, housing unit characteristics); 2) QOL before stable housing (physical and mental wellbeing, social wellbeing, and other determinants of QOL, such as encounters with law enforcement); and 3) Changes in QOL after stable housing (same subthemes as for pre-housing stability QOL). Findings indicated a pattern of perceived relationships between housing stability, housing quality, built and social environments, and QOL in the context of emerging adults who experienced or continued to experience homelessness. However, results were mixed with regards to the perceived effects of housing stability on alcohol and other drug use. Taken together, results indicate several areas of challenge, but also highlight opportunities to facilitate improvements in QOL among vulnerable emerging adults who experience homelessness.

Follow-Up Shadow Coaching Improves Primary Care Provider-Patient Interactions and Maintains Improvements When Conducted Regularly: A Spline Model Analysis

INTRODUCTION

Shadow coaching improves provider-patient interactions, as measured by CG-CAHPS® overall provider rating (OPR) and provider communication (PC). However, these improvements erode over time.

AIM

Examine whether a second coaching session (re-coaching) improves and sustains patient experience.

SETTING

Large, urban Federally Qualified Health Center PROGRAM: Trained providers observed patient care by colleagues and provided suggestions for improvement. Providers with OPRs<90 (0-100-point scale) were eligible.

EVALUATION

We used stratified randomization based on provider type and OPR to assign half of the 40 eligible providers to re-coaching. For OPR and PC, we fit mixed-effects regression models with random-effects for provider (level of treatment assignment) and fixed-effects for time (linear spline with knots and possible "jump" at initial coaching and re-coaching), previous OPR, patient characteristics, and sites. We observed a statistically significant medium jump among re-coached providers after re-coaching on OPR (3.7 points) and PC (3.5 points); differences of 1, 3, and ≥5-points for CAHPS measures are considered small, medium, and large. Improvements from re-coaching persisted for 12 months for OPR and 8 months for PC.

DISCUSSION

Re-coaching improved patient experience more than initial coaching, suggesting the reactivation of knowledge from initial coaching. However, re-coaching gains also eroded. Coaching should occur every 6 to 12 months to maintain behaviors and scores.

A Patient-Centred Medical Home Care Model for Community-Dwelling Older Adults in Singapore: A Mixed-Method Study on Patient's Care Experience

Patient-Centred Medical Home (PCMH) is a strategy to enhance patient-centredness to improve care experience. We aimed to understand patient experience of an integrated PCMH model for complex community-dwelling older adults in Singapore. We used a mixed-method design with a prospective single-group pre-post quantitative component and a concurrent qualitative component. Participants were administered the validated Consumer Assessment of Health Providers and Systems Clinician & Group Survey (CG-CAHPS) at baseline (N = 184) and 6-month (N = 166) post-enrolment. We conducted focus group discussions (FGDs) on a purposive sample of 24 participants. Both methods suggest better care experience in PCMH relative to usual care. There were improvements in the CG-CAHPS measures on patient–provider communication, care coordination, office staff interactions, support for patients in caring for their own health, and provider rating in PCMH relative to usual care. In the FGDs, participants reported benefits of consolidated appointments and positive experience in sustained patient–provider relationship, shared-decision making, and family/caregiver engagement in PCMH. Participants may not fully comprehend the concept of integrated care, hindering both the effective communication of the intended care model and perceived benefits such as the provision of multidisciplinary team-based care.

Patient Experience with In-Person and Telehealth Visits Before and During the COVID-19 Pandemic at a Large Integrated Health System in the United States

BACKGROUND

Information about telehealth versus in-office visits and how patient experience before compared to during the COVID-19 pandemic is important for healthcare planning.

OBJECTIVE

To compare patient experience by visit type and before and during the pandemic.

DESIGN

Survey of patients assessing ambulatory care before and during the pandemic.

PARTICIPANTS

A total of 58,500 adult patients (13,928 primary care and 44,581 specialty physician visits) at a large integrated health system with 197 clinics on the west coast of the United States. The majority were female (59%), 55 or older (65%), and non-Hispanic White (55%), and had an in-office visit (87%) while 10% had a tele-video and 3% a phone visit.

MAIN MEASURES

Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Survey 3.0 doctor communication, care coordination, access, and office staff composites; an overall rating of the doctor; and whether the patient would recommend the doctor to family and friends.

KEY RESULTS

Patient experience with telehealth visits was as positive as or more positive than that with traditional office-based visits. Doctor communication on tele-video visits was viewed as slightly more positive than that of in-office or phone visits. Tele-video visits were also slightly more positive than in-office visits for care coordination, overall rating of the doctor, and willingness to recommend to family and friends. Office staff were viewed less positively on the phone than tele-video or in-office visits. Patient experience was similar before and during the COVID-19 pandemic (e.g., on a 0-100 possible range with a higher score being better, doctor communication was 94.4 before and 94.9 during).

CONCLUSIONS

The positive experiences with telehealth, especially tele-video, may be due to patient appreciation of efforts made to maintain access, the focused nature of telehealth visits, and help by staff for navigation technical issues. Lessons learned about delivering responsive telehealth care can be used to ensure high-quality care after the pandemic.

Development and validation of a patient experience of care survey for emergency departments

OBJECTIVES

To (1) develop a survey to assess the patient experience of care in hospital-based emergency departments (ED) and (2) evaluate the reliability and validity of composite measures of patient experience using data collected through the experimental implementation of the newly developed Emergency Department Patient Experience of Care (EDPEC) Discharged to Community (DTC) Survey.

DATA SOURCE

4893 adult patients were treated in the ED of 16 hospitals across the United States in 2018.

STUDY DESIGN

The study utilized a cross-sectional survey.

DATA COLLECTION

Survey development activities included a literature review, focus groups, and cognitive interviews with recently discharged ED patients, technical expert panels, and multiple field experiments. Survey development resulted in a 34-item instrument; the analysis reported here focuses on 18 items on patient experience of care. Using data from the EDPEC DTC Survey in the 2018 Feasibility Test, we performed confirmatory factor analysis to group 15 evaluative survey items into composite measures. We examined internal consistency reliability, interunit reliability, and associations between each composite measure and patients' overall rating and willingness to recommend the ED.

PRINCIPAL FINDINGS

Analyses of 15 evaluative items identified four composite measures: Getting Timely Care, How Well Doctors and Nurses Communicate, Communication about Medications, and Communication about Follow-up. Patient-level internal consistency reliability exceeded 0.75 for two of four composites; ED-level internal consistency reliability exceeded 0.83 for all four composites. Interunit reliability estimates indicated that 450 survey completes per ED results in at least 0.70 reliability for all composites. Higher scores on each composite were associated with higher overall ratings and willingness to recommend the ED.

CONCLUSIONS

The composite measures derived from the EDPEC DTC Survey are statistically reliable and valid. These results provide guidance for EDPEC DTC Survey adopters on how to construct meaningful and psychometrically-sound composite measures for monitoring the quality of care they provide.

Practices and changes associated with patient-centered medical home transformation

OBJECTIVES

Knowing which patient-centered medical home (PCMH) care delivery changes and quality improvement (QI) practices further PCMH implementation is essential.

STUDY DESIGN

We used the 2008-2017 National Committee of Quality Assurance (NCQA) PCMH directory of 15,188 primary care practices that received Level 1, 2, or 3 NCQA PCMH recognition to construct a stratified national sample of 105 practices engaged in PCMH transformation. We examined their QI practices and PCMH changes associated with PCMH transformation.

METHODS

We derived QI practice and PCMH change variables from semistructured interviews. Practice leaders completed the PCMH Assessment (PCMH-A) measuring the practice's degree of PCMH implementation, which is a proxy for patient-centeredness. Controlling for practice characteristics, we regressed PCMH-A scores on QI practice and PCMH change variables.

RESULTS

Practices undergoing PCMH transformation nationwide most commonly made care delivery changes in access and continuity of care. To improve quality, practices most commonly engaged in discussing and targeting areas of patient experience improvement, trending performance, and conducting targeted QI. However, practices lower in patient-centeredness as measured by the PCMH-A were more likely to engage in efforts to improve patient experiences, such as reviewing patient experience data or engaging in 1-on-1 provider counseling related to patient interactions. Mature PCMH practices focused on changes in continuity of care.

CONCLUSIONS

Practices undertake a wide variety of care delivery changes and QI practices simultaneously to meet PCMH requirements. The patient experience-specific QI practices and PCMH care delivery changes that practices make to improve patient-centeredness differ by years of PCMH recognition.

Improving Care Experiences for Patients and Caregivers at End of Life: A Systematic Review

BACKGROUND

End-of-life care is increasing as the US population ages. Approaches to providing high-quality end-of-life care vary across setting, diseases, and populations. Several data collection tools measure patient and/or caregiver care experiences at end of life and can be used for quality improvement. Previous reviews examined palliative care improvements or available measures but none explicitly on improving care experiences. We reviewed literature on improving patient and/or caregiver end-of-life care experiences.

DESIGN

We searched U.S. English-language peer-reviewed and grey literature after 2000 on adult end-of-life care experiences. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for quantitative studies, Enhancing Transparency in Reporting the Synthesis of Qualitative Research approach for qualitative studies, and Assessment of Multiple Systematic Reviews tool for the literature reviews.

SETTING

Palliative and hospice care.

POPULATION

Full-text abstraction of 84 articles, identifying 16 articles.

MEASURES

Patient and/or caregiver end-of-life care experiences (captured through administrative data or direct report).

RESULTS

Articles examined palliative care experiences across settings; none studied hospice care experiences. Patients and/or caregivers assessed overall care experiences, clinician-staff interactions, provider communication, respect and trust, timeliness of care, spiritual support, caregiver knowledge of care plans, or bereavement support. Efforts aimed at improving end-of-life care experiences are limited and show mixed results.

CONCLUSIONS

Literature on improving patient and/or caregiver end-of-life care experiences is emerging and focused on palliative care experiences. Evidence on improving hospice care experiences is lacking. Research on strategies for improving end-of-life care experiences should go beyond overall care experiences to include specific aspects of palliative and hospice care experiences.

Association Between Patients' Self-Reported Gaps in Care Coordination and Preventable Adverse Outcomes: a Cross-Sectional Survey

BACKGROUND

Whether patients' reports of gaps in care coordination reflect clinically significant problems is unclear.

OBJECTIVE

To determine any association between patient-reported gaps in care coordination and patient-reported preventable adverse outcomes.

DESIGN AND PARTICIPANTS

We administered a cross-sectional survey on experiences with healthcare to participants in the national Reasons for Geographic and Racial Differences in Stroke (REGARDS) study who were ≥ 65 years old. Of the 15,817 participants in REGARDS at the time of our survey (August 2017-November 2018), 11,138 completed the survey. We restricted the sample to participants who reported ≥ 2 ambulatory visits and ≥ 2 ambulatory providers in the past year (N = 7568).

MAIN MEASURES

We considered 7 gaps in ambulatory care coordination, elicited with previously validated questions. We considered 4 outcomes: (1) a test that was repeated because the doctor did not have the result of the first test, (2) a drug-drug interaction that occurred due to multiple prescribers, (3) an emergency department visit that could have been prevented by better communication among providers, and (4) a hospital admission that could have been prevented by better communication among providers. We used logistic regression to determine the association between ≥ 1 gap in care coordination and ≥ 1 preventable outcome, adjusting for potential confounders.

KEY RESULTS

The average age of the sample was 77.0 years; 55% were female, and 34% were African-American. More than one-third of participants (38.1%) reported ≥ 1 gap in care coordination and nearly one-tenth (9.8%) reported ≥ 1 preventable outcome. Having ≥ 1 gap in care coordination was associated with an increased odds of ≥ 1 preventable outcome (adjusted odds ratio 1.55; 95% confidence interval 1.33, 1.81).

CONCLUSIONS

Participants' reports of gaps in care coordination were associated with an increased odds of preventable adverse outcomes. Future interventions should leverage patients' observations to detect and resolve gaps in care coordination.

Nationwide Qualitative Study of Practice Leader Perspectives on What It Takes to Transform into a Patient-Centered Medical Home

BACKGROUND

Despite widespread adoption of patient-centered medical home (PCMH), little is known about why practices pursue PCMH and what is needed to undergo transformation.

OBJECTIVE

Examine reasons practices obtained and maintained PCMH recognition and what resources were needed.

DESIGN

Qualitative study of practice leader perspectives on PCMH transformation, based on a random sample of primary care practices engaged in PCMH transformation, stratified by US region, practice size, PCMH recognition history, and practice use of Consumer Assessment of Healthcare Providers and Systems (CAHPS®) PCMH survey.

PARTICIPANTS

105 practice leaders from 294 sampled practices (36% response rate).

APPROACH

Content analysis of interviews with practice leaders to identify themes.

RESULTS

Most practice leaders had local control of PCMH transformation decisions, even if practices adopted quality initiatives under the direction of an organization or network. Financial incentives, being in a statewide effort, and the intrinsic desire to improve care or experiences were the most common reasons practice leaders decided to obtain PCMH recognition and pursue associated care delivery changes. Leadership support and direction were highlighted as essential throughout PCMH transformation. Practice leaders reported needing specialized staff knowledge and significant resources to meet PCMH requirements, including staff knowledgeable about how to implement PCMH changes, track and monitor improvements, and navigate implementation of simultaneous changes, and staff with specific quality improvement (QI) expertise related to evaluating changes and scaling-up programs.

CONCLUSION

PCMH efforts necessitated support and assistance to frontline, on-site practice leaders leading care delivery changes. Such change efforts should include financial incentives (e.g., direct payment or additional reimbursement), leadership direction and support, and internal or external staff with experience with the PCMH application process, implementation changes, and QI expertise in monitoring process and outcome data. Policies that recognize and meet the needs of on-site practice leaders will better promote primary care practice transformation and move practices further toward their PCMH transformation goals.

Redesigning primary care in an academic medical center: lessons, challenges, and opportunities

PURPOSE

To evaluate patient access, provider productivity, and patient satisfaction during a 24-month redesign process of an academic medical center, which requires balance between clinical and educational missions.

METHODS

A series of activities were conducted to optimize primary care across 17 attending physicians, 6 Advanced Practice Providers (APPs), and 39 residents. Patient access was defined as the next available appointment for either existing/established patients or new patients. Productivity was measured using panel sizes for each provider. Patient satisfaction was based on the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CGCAHPS).

RESULTS

Despite decreasing clinical effort to allow faculty and APPs to participate in education and research, there was an overall increase in access for both new and established patients, and an increase the percent of each providers' panel that was full from 78.89% in 2017 to 115.29% in 2019. When comparing panel sizes for the 11 faculty present before and after strategic changes, we found significant increase in both overall panel size, and actual to expected ratios between 2017 and 2019. In addition, throughout the time period, patient satisfaction remained high with no significant changes.

CONCLUSIONS

While this project was limited to one site, the inclusion of a set of well-planned metrics, and tracking of processes over time can provide insight for ongoing primary care redesign efforts at similar sites seeking to balance the academic mission with clinical productivity and high patient satisfaction.

Differences in Consumer Assessment of Healthcare Providers and Systems Clinician and Group Survey Scores by Recency of the Last Visit: Implications for Comparability of Periodic and Continuous Sampling

BACKGROUND

Patient experience data can be collected by sampling patients periodically (eg, patients with any visits over a 1-year period) or sampling visits continuously (eg, sampling any visit in a monthly interval). Continuous sampling likely yields a sample with more frequent and more recent visits, possibly affecting the comparability of data collected under the 2 approaches.

OBJECTIVE

To explore differences in Consumer Assessment of Healthcare Providers and Systems Clinician and Group survey (CG-CAHPS) scores using periodic and continuous sampling.

RESEARCH DESIGN

We use observational data to estimate case-mix-adjusted differences in patient experience scores under 12-month periodic sampling and simulated continuous sampling.

SUBJECTS

A total of 29,254 adult patients responding to the CG-CAHPS survey regarding visits in the past 12 months to any of 480 physicians, 2007-2009.

MEASURES

Overall doctor rating and 4 CG-CAHPS composite measures of patient experience: doctor communication, access to care, care coordination, and office staff.

RESULTS

Compared with 12-month periodic sampling, simulated continuous sampling yielded patients with more recent visits (by definition), more frequent visits (92% of patients with 2+ visits, compared with 76%), and more positive case-mix-adjusted CAHPS scores (2-3 percentage points higher).

CONCLUSIONS

Patients with more frequent visits reported markedly higher CG-CAHPS scores, but this causes only small to moderate changes in adjusted physician-level scores between 12-month periodic and continuous sampling schemes. Caution should be exercised in trending or comparing scores collected through different schemes.

Implementation of Practice Transformation: Patient Experience According to Practice Leaders

OBJECTIVE

Examine practice leaders' perceptions and experiences of how patient-centered medical home (PCMH) transformation improves patient experience.

SUBJECTS

Thirty-six interviews with lead physicians (n = 13), site clinic administrators (n = 13), and nurse supervisors (n = 10).

METHODS

Semi-structured interviews at 14 primary care practices within a large urban Federally Qualified Health Center (FQHC) delivery system to identify critical patient experience domains and mechanisms of change. Identified patient experience domains were compared with Consumer Assessment of Healthcare Providers and Systems (CAHPS) items.

RESULTS

We identified 28 patient experience domains improved by PCMH transformation, of which 22 are measured by CAHPS, and identified 24 mechanisms of change commonly reported by practice leaders during PCMH transformation.

CONCLUSIONS

PCMH practice transformation can improve patient experience. Most patient experience domains reported as improved during PCMH efforts are measured by CAHPS items. Practices would benefit from collecting specific information on staff behaviors related to teamwork, team-based communication, scheduling, emergency and inpatient follow-up, and referrals. All 3 types of practice leaders reported 4 main mechanisms of PCMH change that improved patient experience. Our findings provide guidance for practice leaders on which strategies of PCMH practice transformation lead to specific improvements in patient experience measures. Further research is needed on the relationship between PCMH changes and changes in CAHPS patient experience scores.