Development and evaluation of CAHPS questions to assess the impact of health information technology on patient experiences with ambulatory care

Simplified rules-based tool to facilitate the application of up-to-date management recommendations in cardio-oncology

BACKGROUND

Millions of cancer survivors are at risk of cardiovascular diseases, a leading cause of morbidity and mortality. Tools to potentially facilitate implementation of cardiology guidelines, consensus recommendations, and scientific statements to prevent atherosclerotic cardiovascular disease (ASCVD) and other cardiovascular diseases are limited. Thus, inadequate utilization of cardiovascular medications and imaging is widespread, including significantly lower rates of statin use among cancer survivors for whom statin therapy is indicated.

METHODS

In this methodological study, we leveraged published guidelines documents to create a rules-based tool to include guidelines, expert consensus, and medical society scientific statements relevant to point of care cardiovascular disease prevention in the cardiovascular care of cancer survivors. Any overlap, redundancy, or ambiguous recommendations were identified and eliminated across all converted sources of knowledge. The integrity of the tool was assessed with use case examples and review of subsequent care suggestions.

RESULTS

An initial selection of 10 guidelines, expert consensus, and medical society scientific statements was made for this study. Then 7 were kept owing to overlap and revisions in society recommendations over recent years. Extensive formulae were employed to translate the recommendations of 7 selected guidelines into rules and proposed action measures. Patient suitability and care suggestions were assessed for several use case examples.

CONCLUSION

A simple rules-based application was designed to provide a potential format to deliver critical cardiovascular disease best-practice prevention recommendations at the point of care for cancer survivors. A version of this tool may potentially facilitate implementing these guidelines across clinics, payers, and health systems for preventing cardiovascular diseases in cancer survivors.

TRIAL REGISTRATION

ClinicalTrials.Gov Identifier: NCT05377320.

Capturing patient experience

Patient experience is a crucial measure of the quality of patient care in healthcare settings. It encompasses all the patient's interactions with staff and their exposure to equipment, procedures, environment and service structure during one care episode. Capturing patient experience is one way of ensuring that patients' voices are heard and can form the basis of an audit or service improvement project designed to evaluate or enhance the patient-centredness of care. Nurses are increasingly involved in audits and service improvement projects, so it is important that they understand what patient experience is, how it differs from patient satisfaction and what approaches can be used to measure it. This article defines patient experience, describes data collection methods and discusses factors to consider when planning to collect patient experience data, notably the validity, reliability and rigour of the data collection tool.

Social media hospital ratings and HCAHPS survey scores

PURPOSE

To empirically verify whether patient hospital satisfaction ratings on social media such as Yelp provide similar information as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) surveys.

DESIGN/METHODOLOGY/APPROACH

OLS and ordinal regressions performed on secondary data obtained from Yelp.com and 2016 Hospital Compare database disclosed by CMS.

FINDINGS

Results show that the patient hospital satisfaction ratings from Yelp can predict the patient experience of care domain scores obtained through the annual HCAHPS surveys and are also positively and significantly correlated to the overall hospital quality performance scores given by CMS.

RESEARCH LIMITATIONS/IMPLICATIONS

Study suggests that social media patient review information could be used to supplement the information obtained from HCAHPS surveys, thereby providing hospitals more accurate information about their patient experiences.

PRACTICAL IMPLICATIONS

Hospital leaders need not wait an entire year to receive their HCHAPS scores to know about the issues related to their patient experience that need improvement and can periodically refer to free Yelp patient review scores on Yelp.com to obtain similar information.

ORIGINALITY/VALUE

To the best of knowledge, this research is the first to empirically demonstrate that patient reviews freely obtained from social media sites like Yelp can provide similar information as obtained from HCAHPS surveys and can thus be used to supplement HCAHPS.

Some Aspects of Patient Experience Assessed by Practices Undergoing Patient-Centered Medical Home Transformation Are Measured by CAHPS, Others Are Not

BACKGROUND AND OBJECTIVES

Delivering care as a patient-centered medical home (PCMH) is being widely adopted across the United States by primary care practices to better meet patient needs. A key PCMH element is measuring patient experience for practice improvement. The National Committee for Quality Assurance (NCQA) PCMH recognition program requires practices to both measure patient experience and engage in continuous practice/quality improvement to attain PCMH recognition and then throughout full PCMH transformation. The NCQA recommends but does not require that practices administer the Consumer Assessment of Healthcare Providers and Systems (CAHPS) clinician and group patient experience survey (CG-CAHPS) plus 14 CAHPS PCMH items, known as the CAHPS PCMH survey. We examine aspects of patient experience measured by practices with a varying number of years on their journey of PCMH transformation.

METHODS

We randomly selected practices from the 2008-2017 NCQA directory of practices that had applied for PCMH recognition based on region, physician count, number of years and level of PCMH recognition, and use of the CG-CAHPS PCMH survey. We collected characteristics of the practices from practice leader(s) knowledgeable about the practice's PCMH history and patient experience data. We confirmed the patient experience surveys used during their PCMH history and requested copies of their non-CAHPS survey(s). For practices not administering the recommended CG-CAHPS survey (53/105 practices), we obtained and coded the content of their non-CAHPS surveys (68%; 36/53). We mapped the patient experience domains and specific measures to the CG-CAHPS survey (versions 2.0 and 3.0), CAHPS PCMH item set (versions 2.0 and 3.0), and the available CG-CAHPS supplemental items.

RESULTS

Whether or not practices administered the CG-CAHPS items, most of them addressed topics contained in the CG-CAHPS survey such as Access to care, Provider communication, Office staff helpfulness/courteousness, Care coordination, and Shared decision-making. The most common CAHPS measures included were Office staff helpfulness/courteousness and Provider communication. Common non-CAHPS measures included were Ease of scheduling, Being informed about delays, and Provider helpfulness/courteousness.

CONCLUSION

NCQA PCMH practices included CAHPS items on their patient experience surveys even if they did not administer the full CG-CAHPS survey or the recommended CAHPS PCMH survey. To enhance the usefulness of patient experience surveys for practices undergoing PCMH changes, additional CAHPS measures could be developed related to key areas of PCMH change, including expanded access to care (ie, after-hours and weekend visits, ease of scheduling, being informed about delays), use of shared decision-making, and improvements in provider communication (ie, the provider is courteous, communication by other clinical staff members with the patient). These additional measures would assist practice leaders in capturing the breadth and depth of their PCMH transformation and its influence on providing more patient-centered care. Developing such items would help standardize the measurement of changes related to patient experience during PCMH transformation. Research is needed to determine whether a CAHPS survey is the best source of this information.

Patient experience of an integrated care model in a family practice clinic & FQHC

In order to investigate the patient experience of integrated behavioral health care in primary care settings, we implemented a patient cohort model from a combined site sample (N = 727) consisting of a family practice clinic and a Federally Qualified Health Center. Patient experience was measured using 12 questions from a validated measure, the Agency for Healthcare Research and Quality's Consumer Assessment of Health Care Providers and Systems (CAHPS®), Home and Community Based Services version, and six additional questions about interactions with an integrated behavioral health care team. We assessed bivariate relationships between satisfaction with integration and the clinic practice and self-reported physical health or self-reported mental/emotional health. We also utilized multiple regression to evaluate this relationship. Our analyses showed a statistically significant and small to moderate direct correlation between patients' self-reported health (both physical and mental/emotional health) and their ratings of the practice as a whole (p = .0003), such that patients who rated their physical and/or mental/emotional health as better were more likely to rate their overall satisfaction with the practice higher. The results of this study suggest that primary care patients with only mild to moderate health conditions (physical and/or mental/emotional) may experience greater satisfaction with integrated behavioral health care than patients with multiple and/or severe health conditions. In contrast, patients with multiple and/or severe health conditions may experience lower satisfaction with integrated behavioral health care and may be better served through higher levels of care.

The CMS State Innovation Models Initiative and Improved Health Information Technology and Care Management Capabilities of Physician Practices

The Centers for Medicare and Medicaid Services' (CMS) State Innovation Models (SIMs) initiative funded 17 states to implement health care payment and delivery system reforms to improve health system performance. Whether SIM improved health information technology (HIT) and care management capabilities of physician practices, however, remains unclear. National surveys of physician practices (N = 2,722) from 2012 to 2013 and 2017 to 2018 were linked. Multivariable regression estimated differential adoption of 10 HIT functions and chronic care management processes (CMPs) based on SIM award status (SIM Round 1, SIM Round 2, or non-SIM). HIT and CMP capabilities improved equally for practices in SIM Round 1 (5.3 vs. 6.8 capabilities, p < .001), SIM Round 2 (4.7 vs. 7.0 capabilities, p < .001), and non-SIM (4.2 vs. 6.3 capabilities, p < .001) states. The CMS SIM Initiative did not accelerate the adoption of ten foundational physician practice capabilities beyond national trends.

A systematic review of the validity and reliability of patient-reported experience measures

OBJECTIVES

To identify patient-reported experience measures (PREMs), assess their validity and reliability, and assess any bias in the study design of PREM validity and reliability testing.

DATA SOURCES/STUDY SETTING

Articles reporting on PREM development and testing sourced from MEDLINE, CINAHL and Scopus databases up to March 13, 2018.

STUDY DESIGN

Systematic review.

DATA COLLECTION/EXTRACTION METHODS

Critical appraisal of PREM study design was undertaken using the Appraisal tool for Cross-Sectional Studies (AXIS). Critical appraisal of PREM validity and reliability was undertaken using a revised version of the COSMIN checklist.

PRINCIPAL FINDINGS

Eighty-eight PREMs were identified, spanning across four main health care contexts. PREM validity and reliability was supported by appropriate study designs. Internal consistency (n = 58, 65.2 percent), structural validity (n = 49, 55.1 percent), and content validity (n = 34, 38.2 percent) were the most frequently reported validity and reliability tests.

CONCLUSIONS

Careful consideration should be given when selecting PREMs, particularly as seven of the 10 validity and reliability criteria were not undertaken in ≥50 percent of the PREMs. Testing PREM responsiveness should be prioritized for the application of PREMs where the end user is measuring change over time. Assessing measurement error/agreement of PREMs is important to understand the clinical relevancy of PREM scores used in a health care evaluation capacity.

Survey Instruments to Assess Patient Experiences With Access and Coordination Across Health Care Settings: Available and Needed Measures

BACKGROUND

Improving access can increase the providers a patient sees, and cause coordination challenges. For initiatives that increase care across health care settings, measuring patient experiences with access and care coordination will be crucial.

OBJECTIVES

Map existing survey measures of patient experiences with access and care coordination expected to be relevant to patients accessing care across settings. Preliminarily examine whether aspects of access and care coordination important to patients are represented by existing measures.

RESEARCH DESIGN

Structured literature review of domains and existing survey measures related to access and care coordination across settings. Survey measures, and preliminary themes from semistructured interviews of 10 patients offered VA-purchased Community Care, were mapped to identified domains.

RESULTS

We identified 31 existing survey instruments with 279 items representing 6 access and 5 care coordination domains relevant to cross-system care. Domains frequently assessed by existing measures included follow-up coordination, primary care access, cross-setting coordination, and continuity. Preliminary issues identified in interviews, but not commonly assessed by existing measures included: (1) acceptability of distance to care site given patient's clinical situation; (2) burden on patients to access and coordinate care and billing; (3) provider familiarity with Veteran culture and VA processes.

CONCLUSIONS

Existing survey instruments assess many aspects of patient experiences with access and care coordination in cross-system care. Systems assessing cross-system care should consider whether patient surveys accurately reflect the level of patients' concerns with burden to access and coordinate care, and adequately reflect the impact of clinical severity and cultural familiarity on patient preferences.

eHealth Literacy: Patient Engagement in Identifying Strategies to Encourage Use of Patient Portals Among Older Adults

Innovations in chronic disease management are growing rapidly as advancements in technology broaden the scope of tools. Older adults are less likely to be willing or able to use patient portals or smartphone apps for health-related tasks. The authors conducted a cross-sectional survey of older adults (ages ≥50) with hypertension or diabetes to examine relationships between portal usage, interest in health-tracking tools, and eHealth literacy, and to solicit practical solutions to encourage technology adoption. Among 247 patients surveyed in a large integrated delivery health system between August 2015 and January 2016, eHealth literacy was positively associated with portal usage (OR [95% CI]: 1.3 [1.2-1.5]) and interest in health-tracking tools (1.2 [1.1-1.3]). Portal users compared to nonusers (N = 137 vs.110) had higher rates of interest in using websites/smartphone apps to track blood pressure (55% vs. 36%), weight (53% vs. 35%), exercise (53% vs. 32%), or medication (46% vs 33%, all P < 0.05). Portal users noted cumbersome processes for accessing portals and variations in provider availability for online scheduling and response times to messages. Portal nonusers expressed concerns about data security, lack of personalization, and limited perceived value of using portals. Both groups noted the importance of computer literacy and technical support. Patient stakeholders recommended marketing initiatives that capture patient stories demonstrating real-life applications of what patients can do with digital technology, how to use it, and why it may be useful. Health systems also must screen for eHealth literacy, provide training, promote proxy users, and institute quality assurance that ensures patients' experiences will not vary across the system.

Shared decision-making following disclosure of coronary heart disease genetic risk: results from a randomized clinical trial

Whether disclosure of genetic risk for coronary heart disease (CHD) influences shared decision-making (SDM) regarding use of statins to reduce CHD risk is unknown. We randomized 207 patients, age 45-65 years, at intermediate CHD risk, and not on statins, to receive the 10-year risk of CHD based on conventional risk factors alone (n=103) or in combination with a genetic risk score (n=104). A genetic counselor disclosed this information followed by a physician visit for SDM regarding statin therapy. A novel decision aid was used in both encounters to disclose the CHD risk estimates and facilitate SDM regarding statin use. Patients reported their decision quality and physician visit satisfaction using validated surveys. There were no statistically significant differences between the two groups in the SDM score, satisfaction with the clinical encounter, perception of the quality of the discussion or of participation in decision-making and physician visit satisfaction scores. Quantitative analyses of a random subset of 80 video-recorded encounters using the OPTION5 scale also showed no significant difference in SDM between the two groups. Disclosure of CHD genetic risk using an electronic health record-linked decision aid did not adversely affect SDM or patients' satisfaction with the clinical encounter.

TRIAL REGISTRATION NUMBER

NCT01936675; Results.

Comparing Web-based with Mail Survey Administration of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Survey

CONTEXT

The CAHPS^® survey instruments are widely used to assess patient experiences with care but there is limited information about web-based data collection with them.

OBJECTIVE

To compare web-based data collection with standard mail survey mode of collection of CAHPS^® Clinician and Group survey data.

DESIGN SETTING AND PATIENTS

We randomized mode of data collection (web versus mail) of the CAHPS^® Clinician and Group Survey to patients who had visited one of six clinics over a four-month period in Minnesota. A total of 410 patients responded to the web-based survey (14% response rate) and 982 patients responded to the mail survey (33% response rate).

MAIN OUTCOME MEASURES

Responses to CAHPS^® survey dimensions and individual question responses, response rates, and participant characteristics.

RESULTS

There were no significant differences in CAHPS^® survey composites and individual question responses by mode, except for those addressing access. Those responding via the web reported less positive experiences with access to an appointment for urgent care as soon as needed, getting an appointment for routine care as soon as needed, getting answers to medical questions as soon as needed, and follow-up on test results (t's=-3.64, -7.15, -2.58, -2.23; p's=0.0003, <0.0001, 0.01, 0.03, respectively). Web respondents had more positive experiences about office wait time for the most recent visit (t = 2.32, p=0.021). Those who participated in the study tended to be older than those that did not (ƛ²=247.51, df=8, p<0.0001 for mail; ƛ²= 4.56, df=8, p<0.0001 for the web). Females were significantly more likely than males to respond to the survey overall (24% vs. 18%, ƛ²=6.45, 1 df, p=0.011) and relatively more likely than males to respond to web (15% vs. 13%, ƛ²=1.32, 1 df, p=0.25) than mail (34% vs. 30%, ƛ²=5.42, 1 df, p=0.02). Mail respondents were more likely than web respondents to be male (28% versus 18%, ƛ²=16.27, 1 df, p<0.0001) and older (27% of the mail respondents and 19% of the web respondents were 65 or older, ƛ²=10.88, 1 df, p=0.001). Costs of web-based surveys were less than mailed surveys and were returned more quickly than mailed surveys. The correlations between reports and ratings of clinicians and clinics by mode were unreliable because of the relatively small number of web responses.

CONCLUSION

Web-based surveys yielded comparable results to mail (except for questions addressing access) more quickly at lower costs. The low response rates in this study are a concern although this was not intended as a test of increasing response rates. Strategies to increase response rates will be a key element of web-based data collection. The differences in costs will be an incentive for organizations to continue to pursue web-based surveying. Further studies are needed to evaluate the generalizability of the results of this one.

Development of and field test results for the CAHPS PCMH Survey

OBJECTIVE

To develop and evaluate survey questions that assess processes of care relevant to Patient-Centered Medical Homes (PCMHs).

RESEARCH DESIGN

We convened expert panels, reviewed evidence on effective care practices and existing surveys, elicited broad public input, and conducted cognitive interviews and a field test to develop items relevant to PCMHs that could be added to the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician & Group (CG-CAHPS) 1.0 Survey. Surveys were tested using a 2-contact mail protocol in 10 adults and 33 pediatric practices (both private and community health centers) in Massachusetts. A total of 4875 completed surveys were received (overall response rate of 25%).

ANALYSES

We calculated the rate of valid responses for each item. We conducted exploratory factor analyses and estimated item-to-total correlations, individual and site-level reliability, and correlations among proposed multi-item composites.

RESULTS

Ten items in 4 new domains (Comprehensiveness, Information, Self-Management Support, and Shared Decision-Making) and 4 items in 2 existing domains (Access and Coordination of Care) were selected to be supplemental items to be used in conjunction with the adult CG-CAHPS 1.0 Survey. For the child version, 4 items in each of 2 new domains (Information and Self-Management Support) and 5 items in existing domains (Access, Comprehensiveness-Prevention, Coordination of Care) were selected.

CONCLUSIONS

This study provides support for the reliability and validity of new items to supplement the CG-CAHPS 1.0 Survey to assess aspects of primary care that are important attributes of PCMHs.