Patient Experience Assessment is a Requisite for Quality Evaluation: A Discussion of the In-Center Hemodialysis Consumer Assessment of Health Care Providers and Systems (ICH CAHPS) Survey

Evaluation and Measurement Properties of a Patient-Reported Experience Measure for Home Dialysis

BACKGROUND

No previously validated patient-reported experience measures exist for use among patients undergoing home dialysis. We tested the Home Dialysis Care Experience survey, a newly developed 26-item experience measure, among patients from 30 dialysis facilities in the United States.

METHODS

Using mail and telephone survey modalities, we approached 1372 patients treated with peritoneal dialysis or home hemodialysis for participation. Using the results from completed surveys, we evaluated item calibration by assessing item floor and ceiling effects. We tested three sets of composite scores and used factor analysis to assess model fit for each. We evaluated associations of composite scores with global ratings and separately with patient and dialysis facility characteristics. Finally, we measured test-retest reliability in patients who completed the survey at two separate time points.

RESULTS

Overall, 495 eligible patients completed at least one survey (response rate 36%). Of these, 49 completed the survey in Spanish and 61 completed a second survey within 30 days. We did not detect significant floor or ceiling effects, except for one item that demonstrated >90% responses at the top response option. Analyses supported one 12-item composite scale with high internal consistency reliability: Quality of Home Dialysis Care and Operations (Cronbach alpha=0.85). This scale strongly correlated with overall staff rating ( r =0.73) and overall center rating ( r =0.70). Patient demographic and dialysis facility characteristics were not consistently associated with composite scale scores or overall staff or center ratings. Intraclass correlation coefficients in the test-retest population were 0.74 for the Quality scale, 0.88 for overall staff rating, and 0.90 for overall center rating.

CONCLUSIONS

The Home Dialysis Care Experience survey is a 26-item measure that includes one composite scale and two global rating scores and is an informative tool to evaluate patient experience of care for home dialysis.

Online Reviews of Hemodialysis Centers Correlate With Medicare and Medicaid Survey Measures of Patient Experience

BACKGROUND AND OBJECTIVES

Patient experience in outpatient hemodialysis has been shown to be significantly correlated with health outcomes. The current gold standard for assessing patient experience in outpatient hemodialysis is the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS). Online reviews of outpatient hemodialysis centers could potentially serve as an additional source of information regarding patient experience, but they have not been well validated. This study aims to determine whether overall scores and subscores from patient-authored online reviews of outpatient dialysis centers are correlated with current gold standard survey-based measures of patient experience in outpatient hemodialysis.

METHODS

All reviews of hemodialysis centers posted to the online review site CiteHealth.com between March 2008 and October 2019 were collected (1081 reviews of 762 centers). Publicly-available ICH-CAHPS survey summary data and End Stage Renal Disease Quality Incentive Program (ESRD QIP) summary data from May 2016 to October 2019 were obtained from the Dialysis Facility Compare website. Spearman correlation coefficients were calculated between facilities' mean online review overall scores and subscores within a given year and their ICH-CAHPS ratings from the same year. Statistical significance was assessed with a 2-tailed permutation test. A Bonferroni correction for multiple hypothesis testing was applied.

RESULTS

The mean "Overall" score from CiteHealth.com had a significant positive correlation with the "Center Care Quality," "Staff," and "Facility" scores from ICH-CAHPS surveys. No significant correlation could be found between the mean "Overall" CiteHealth.com score and any other ICH-CAHPS patient satisfaction metric. There was a significant positive correlation between the mean CiteHealth.com "Center" score and the ICH-CAHPS "Center Care Quality" score, the mean CiteHealth.com "Facility" score and the ICH-CAHPS "Facility" score, and the mean CiteHealth.com "Staff" score and the ICH-CAHPS "Staff" score. No significant correlation was found between the mean CiteHealth.com "Nephrologist" score and the ICH-CAHPS "Nephrologist" score. No significant correlation was found between online review scores and ESRD QIP health outcome measures.

CONCLUSION

Certain components of online reviews are significantly correlated with ICH-CAHPS measures of patient experience. Additionally, online reviews come with narrative comments that can offer specific insights into positive and negative aspects of patient care that cannot always be elucidated by numeric survey questions. Online reviews may have utility as an adjunctive source of information to patient experience surveys such as the ICH-CAHPS.

Patient-Reported Experiences with Dialysis Care and Provider Visit Frequency

BACKGROUND AND OBJECTIVES

New payment models resulting from the Advancing American Kidney Health initiative may create incentives for nephrologists to focus less on face-to-face in-center hemodialysis visits. This study aimed to understand whether more frequent nephrology practitioner dialysis visits improved patient experience and could help inform future policy.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

In a cross-sectional study of patients receiving dialysis from April 1, 2015 through January 31, 2016, we linked patient records from a national kidney failure registry to patient experience data from the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems survey. We used a multivariable mixed effects linear regression model to examine the association between nephrology practitioner visit frequency and patient-reported experiences with nephrologist care.

RESULTS

Among 5125 US dialysis facilities, 2981 (58%) had ≥30 In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems surveys completed between April 2015 and January 2016, and 243,324 patients receiving care within these facilities had Medicare Parts A/B coverage. Face-to-face practitioner visits per month were 71% with four or more visits, 17% with two to three visits, 4% with one visit, and 8% with no visits. Each 10% absolute greater proportion of patients seen by their nephrology practitioner(s) four or more times per month was associated with a modestly but statistically significant lower score of patient experience with nephrologist care by -0.3 points (95% confidence interval, -0.5 to -0.1) and no effect on experience with other domains of dialysis care.

CONCLUSIONS

In an analysis of patient experiences at the dialysis facility level, frequent nephrology practitioner visits to facilities where patients undergo outpatient hemodialysis were not associated with better patient experiences.

An Evidence-Based Theory About PRO Use in Kidney Care: A Realist Synthesis

BACKGROUND

There is international interest on the use of patient-reported outcomes (PROs) in nephrology.

OBJECTIVES

Our objectives were to develop a kidney-specific program theory about use of PROs in nephrology that may enhance person-centered care, both at individual and aggregated levels of care, and to test and refine this theory through a systematic review of the empirical literature. Together, these objectives articulate what works or does not work, for whom, and why.

METHODS

Realist synthesis methodology guided the electronic database and gray literature searches (in January 2017 and October 2018), screening, and extraction conducted independently by three reviewers. Sources included all nephrology patients and/or practitioners. Through a process of extraction and synthesis, each included source was examined to assess how contexts may trigger mechanisms to influence specific outcomes.

RESULTS

After screening 19,961 references, 84 theoretical and 34 empirical sources were used. PROs are proposed to be useful for providing nephrology care through three types of use. The first type is use of individual-level PRO data at point of care, receiving the majority of theoretical and empirical explorations. Clinician use to support person-centered care, and patient use to support patient engagement, are purported to improve satisfaction, health, and quality of life. Contextual factors specific to the kidney care setting that may influence the use of PRO data include the complexity of kidney disease symptom burden, symptoms that may be stigmatized, comorbidities, and time or administrative constraints in dialysis settings. Electronic collection of PROs may facilitate PRO use given these contexts. The second type is use of aggregated PRO data at point of care, including public reporting of PROs to inform decisions at point of care and improve quality of care, and use of PROs for treatment decisions. The third type is use of aggregated PRO data by organizations, including publicly available PRO data to compare centers. In single-payer systems, regular collection of PROs by dialysis centers can be achieved through economic incentives. Both the second and third types of PRO use include pressures that may trigger quality improvement processes.

CONCLUSION

The current state of the evidence is primarily theoretical. There is pressing need for empirical research to improve the evidence-base of PRO use at individual and aggregated levels of nephrology care.

Contributions of treatment centre and patient characteristics to patient-reported experience of haemodialysis: a national cross-sectional study

OBJECTIVES

To examine the relative importance of patient and centre level factors in determining self-reported experience of care in patients with advanced kidney disease treated by maintenance haemodialysis (HD).

DESIGN

Analysis of data from a cross sectional national survey; the UK Renal Registry (UKRR) national Kidney patient-reported experience measure (PREM) survey (2018). Centre-level data were obtained from the UKRR report (2018).

SETTING

National survey of patients with advanced kidney disease receiving treatment with maintenance HD in UK renal centres in 2018.

PARTICIPANTS

The Kidney PREM was distributed to all UK renal centres by the UKRR in May 2018. Each centre invited patients receiving outpatient treatment for kidney disease to complete the PREM. These included patients with chronic kidney disease, those receiving dialysis-both HD and peritoneal dialysis, and those with a functioning kidney transplant. There were no formal inclusion/exclusion criteria.

MAIN OUTCOME MEASURES

The Kidney PREM has 38 questions in 13 subscales. Responses were captured using a 7-point Likert scale (never 1, always 7). The primary outcome of interest was the mean PREM score calculated across all questions. Multilevel modelling was used to determine the proportion of variation of the mean PREM score across centres due to patient-related and centre-related factors.

RESULTS

There were records for 8253 HD patients (61% men, 77% white) from 69 renal centres (9-710 patients per centre). There was significant variation in mean PREM score across centres (5.35-6.53). In the multivariable analysis there was some variation in relation to both patient- and centre-level factors but these contributed little to explaining the overall variation. However, multilevel modelling showed that the overwhelming proportion of the explained variance (45%) was explained by variation between centres (40%), only a small proportion of which is identified by measured factors. Only 5% of the variation was related to patient-level factors.

CONCLUSIONS

Centre rather than patient characteristics determine the experience of care of patients receiving HD. Further work is required to define the characteristics of the treating centre which determine patient experience.

Incorporating patient-reported outcomes (PROs) into dialysis policy: Current initiatives, challenges, and opportunities

Governments at national and state levels regulate dialysis care in the United States to ensure safe practices, and continually elevate the quality of care. An objective of these regulatory policies is the independent evaluation of dialysis unit outcomes by patients, caregivers, and the community to facilitate choices as well as to advance equal access to high quality dialysis care. These polices recognized decades ago that it was fundamental to include the patient perspective in the assessment and evaluation of dialysis care quality by requiring both individual and aggregate patient reported outcomes (PROs). Although there is support for integrating the patient perspective, concerns persist about the implementation of these polices including selection of PRO measures, administration timing and reach, as well as interpretation of results including benchmarking to permit comparisons across organizations. The experience from the early adoption of PROs into dialysis policies in conjunction with advances in electronic health records, personal data capture and monitoring, and analytics is poised to address these concerns. The dialysis community has the opportunity to lead the way in innovation related to PRO implementation not only in kidney disease care, but also for other healthcare conditions or contexts such as oncology, surgical, and acute care.

A Model for Improving Health Care Quality for Transgender and Gender Nonconforming Patients

PROBLEM DEFINITION

Transgender and gender nonconforming (TGNC) populations are disproportionately affected by limited health care access and poor health outcomes and commonly report discrimination and mistreatment in health care settings. Despite these disparities, comprehensive approaches to improve the quality of health care of TGNC patient populations are currently lacking.

INITIAL APPROACH

The Vanderbilt Program for LGBTQ Health has developed a multifaceted, community-engaged approach to improve the quality of health care of TGNC patients, which includes the creation of a transgender patient advocacy program, a community advisory board, and a transgender health clinic. To support the continuous quality improvement of transgender health care, the program is currently piloting a novel multilevel monitoring and evaluation (M&E) system to collect information at the individual patient visit and health systems levels.

NEXT STEPS

The next steps for Vanderbilt's community-engaged M&E system are to identify the clinics and health services most used by TGNC patients and assess the level of patient satisfaction in each area. This process will support the identification of high- and low-performing clinics and health services and allow for targeted delivery of trainings to improve the quality of culturally competent health care TGNC patients receive systemwide.

CONCLUSION

In collaboration with TGNC patient populations and community stakeholders, Vanderbilt has created a model to improve the quality of both transition- and non-transition-related health care at the systems level that can be adopted by other health care systems nationally.

Kidney Disease Awareness and Knowledge among Survivors ofAcute Kidney Injury

BACKGROUND

Acute kidney injury (AKI) survivors are at risk for chronic kidney disease, recurrent AKI, and cardiovascular disease. The transition from hospital to ambulatory care is an opportunity to reduce these sequelae by launching self-care plans through effective patient education. How well AKI survivors are informationally prepared to apply kidney-specific self-care is unknown. The purpose of this study was to identify awareness and disease-specific knowledge among AKI survivors.

METHODS

We performed a cross-sectional survey of AKI-related awareness and knowledge in 137 patients with Kidney Disease Improving Global Outcomes Stage II or III AKI near the time of hospital discharge. Patients were asked (1) "Did you experience AKI while in the hospital?" and (2) "Do you have a problem with your kidney health?" Objective knowledge of AKI was evaluated with a 15-item adapted version of the validated Kidney Knowledge Survey that included topics such as common causes, risk factors, and how AKI is diagnosed.

RESULTS

Median age was 54 (interquartile range 43-63) and 81% were white. Eighty percent of patients were unaware that they had experienced AKI and 53% were both unaware they had experienced AKI or had a "problem with their kidneys." Multivariable logistic regression identified being male and lack of nephrology consult as predictors of unawareness with ORs of 3.92 (95% CI 1.48-10.33) and 5.10 (95% CI 1.98-13.13), respectively. Less than 50% recognized nonsteroidal anti-inflammatory drugs, contrast, or phosphate-based cathartics as risk factors for AKI. Two-thirds of patients did not agree that they knew a lot about AKI and more than 80% desired more information.

CONCLUSIONS

Most patients with moderate to severe AKI are unaware of their condition, lack understanding of risk factors for recurrent AKI, and desire more information. Patient-centered communication to optimize awareness, understanding, and care will require coordinated educational strategies throughout the continuum of AKI care.

Kidney Disease Quality of Life 36-Item Short Form Survey (KDQOL-36) Normative Values for the United States Dialysis Population and New Single Summary Score

BACKGROUND

The Kidney Disease Quality of Life 36-item short form survey (KDQOL-36) is a widely used, patient-reported outcome measure for patients on dialysis. Efforts to aid interpretation are needed.

METHODS

We used a sample of 58,851 dialysis patients participating in the Medical Education Institute (MEI) KDQOL Complete program, and 443,947 patients from the US Renal Data System (USRDS) to develop the KDQOL-36 Summary Score (KSS) for the kidney-targeted KDQOL-36 scales (Burdens of Kidney Disease [BKD], Symptoms and Problems of Kidney Disease [SPKD], and Effects of Kidney Disease [EKD]). We also used the MEI and USRDS data to calculate normative values for the Short Form-12 Health Survey's Physical Component Summary (PCS) and Mental Component Summary (MCS), and the KDQOL-36's BKD, SPKD, and EKD scales for the United States dialysis population. We used confirmatory factor analysis (CFA) models for KDQOL-36 kidney-targeted items, evaluated model fit with the comparative fit index (CFI; >0.95 indicates good fit) and root-mean-squared error of approximation (RMSEA; <0.06 indicates good fit), and estimated norms by matching the joint distribution of patient characteristics in the MEI sample to those of the USRDS sample.

RESULTS

A bifactor CFA model fit the data well (RMSEA=0.046, CFI=0.990), supporting the KSS (α=0.91). Mean dialysis normative scores were PCS=37.8 and MCS=50.9 (scored on a T-score metric); and KSS=73.0, BKD=52.8, SPKD=79.0, and EKD=74.1 (0-100 possible scores).

CONCLUSIONS

The KSS is a reliable summary of the KDQOL-36. The United States KDQOL-36 normative facilitate interpretation and incorporation of patient-related outcome measures into kidney disease care.

Measuring patient and family perceptions of team processes and outcomes in healthcare teams: questionnaire development and psychometric evaluation

BACKGROUND

There is a lack of validated instruments examining dimensions of team functioning from the perspective of patients and families consistent with a conceptual framework. The study aimed to develop and assess the psychometric properties of the Patient-Perceptions of Team Effectiveness (PTE) questionnaire.

METHODS

A cross-sectional survey was undertaken in three studies. Data were collected from May-October 2016 for Study 1, April 2018-ongoing for Study 2, and October 2016 to June 2017 for Study 3. Online and paper versions of the self-administered questionnaire were available in English and in French. The initial questionnaire included 41 items. Study 1 included 320 respondents. Reliability was assessed using Cronbach alpha. Face validity (n = 250) was assessed using a structured questionnaire. Content validity was examined using subject matter experts and Spearman's item-total correlations. Construct validity was examined using known group comparisons (i.e., clinical specialty, education, length of follow-up, reason of consultation). Content analysis was used for open-ended questions.

RESULTS

The questionnaire took 10 to 15 min to complete. Positive assessments were noted for instructions, formatting, font size and logical ordering of questions. In Study 1, reliability indices for the PTE-Overall, Team Processes and Outcomes subscales ranged from 0.72 to 0.84. Item-total correlations ranged from 0.551 to 0.794 (p <  0.001). Differences were noted between clinical specialties, education, length of follow-up, reason of consultation, low and high functioning teams. No differences were noted between English and French language respondents. Psychometric properties were re-assessed in Study 2 and 3 after unclear questions were reworked. Reliability indices for the subscales ranged from 0.76 to 0.94 and differences remained significant between low and high functioning teams.

CONCLUSION

The final 43-item instrument is easy to administer to patients and families. The studies provide evidence of validity to support the propositions in the conceptual framework. The patient-level measures can be aggregated to the team, organizational or system level. The information can be used to assess healthcare team functioning in acute and primary care and determine the role patients and families are playing in teams. Further testing is needed with patients and families who are hospitalized or receiving care from teams in rural areas.

Evaluation of non-response to the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) survey

Background

The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) Survey is the first patient reported outcome measure included in the U.S. Medicare End Stage Renal Disease Quality Incentive Program. Administered twice yearly, it assesses in-center dialysis experience and survey responses are tied to dialysis facility payments. Low response rates, currently approximately 35%, raise concern for possible underrepresentation of patient groups.

Methods

Cross-sectional analysis of survey administration in 2012 to all in-center hemodialysis patients in Dialysis Clinic, Inc. (DCI) facilities nationally over 18 years old who received hemodialysis at their facility for at least 3 months. Patient-level covariates included demographic, clinical, laboratory, and functional characteristics. Random effects multivariable logistic regression was used to assess survey non-response.

Results

Among 11,055 eligible patients 6541 (59%) were non-responders. Of the remaining 4514 responders, 549 (14%) surveys were not usable due to presence of proxy help or incomplete responses. Non-responders were more likely to be men, non-white, younger, single, dual Medicare/Medicaid eligible, less educated, non-English speaking, and not active on the transplant list; non-responders had longer ESRD vintage, lower body mass index, lower serum albumin, worse functional status, and more hospitalizations, missed treatments, and shortened treatments. Similar associations were found using more parsimonious multivariable analyses and after imputing missing data.

Conclusions

Non-responders to the ICH CAHPS significantly differed from responders, broadly spanning individuals with fewer socioeconomic advantages and greater illness burden, raising limitations in interpreting facility survey results. Future research should assess reasons for non-response to improve ICH CAHPS generalizability and utility.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3618-4) contains supplementary material, which is available to authorized users.

Patient-Reported Experiences of Dialysis Care Within a National Pay-for-Performance System

IMPORTANCE

Medicare's End-Stage Renal Disease Quality Incentive Program incorporates measures of perceived value into reimbursement calculations. In 2016, patient experience became a clinical measure in the Quality Incentive Program scoring system. Dialysis facility performance in patient experience measures has not been studied at the national level to date.

OBJECTIVE

To examine associations among dialysis facility performance with patient experience measures and patient, facility, and geographic characteristics.

DESIGN

In this cross-sectional analysis, patients from a national end-stage renal disease registry receiving in-center hemodialysis in the United States on December 31, 2014, were linked with dialysis facility scores on the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS) survey. Of 4977 US dialysis facilities, 2939 (59.1%) reported ICH-CAHPS scores from April 8, 2015, through January 11, 2016. Multivariable linear regression models with geographic random effects were used to examine associations of facility ICH-CAHPS scores with patient, dialysis facility, and geographic characteristics and to identify the amount of total between-facility variation in patient experience scores explained by these categories. Data were analyzed from September 15, 2017, through June 1, 2018.

EXPOSURES

Dialysis facility, geographic characteristic, and 10% change in patient characteristics.

MAIN OUTCOMES AND MEASURES

Dialysis facility ICH-CAHPS scores and the total between-facility variation explained by different categories of characteristics.

RESULTS

Of the 2939 facilities included in the analysis, adjusted mean ICH-CAHPS scores were 2.6 percentage points (95% CI, 1.5-3.7) lower in for-profit facilities, 1.6 percentage points (95% CI, 0.9-2.2) lower in facilities owned by large dialysis organizations, and 2.3 percentage points (95% CI, 0.5-4.2) lower in free-standing facilities compared with their counterparts. More nurses per patient was associated with 0.2 percentage points (95% CI, 0.03-0.3) higher scores; a privately insured patient population was associated with 1.2 percentage points (95% CI, 0.2-2.2) higher scores. Facilities with higher proportions of black patients had 0.95 percentage points (95% CI, 0.78-1.12) lower scores; more Native American patients, 1.00 percentage point (95% CI, 0.39-1.60) lower facility scores. Geographic location and dialysis facility characteristics explained larger proportions of the overall between-facility variation in ICH-CAHPS scores than did patient characteristics.

CONCLUSIONS AND RELEVANCE

This study suggests that for-profit operation, free-standing status, and large dialysis organization designation were associated with less favorable patient-reported experiences of care. Patient experience scores varied geographically, and black and Native American populations reported less favorable experiences. The study findings suggest that perceived quality of care delivered in these settings are of concern, and that there may be opportunities for improved implementation of patient experience surveys as is highlighted.

Methodological considerations in using patient reported measures in dialysis clinics

Patient reported measures (PRMs), including patient-reported outcomes, play a critical role in dialysis care. The usage of PRMs is extensive in dialysis clinics. While there are excellent PRMs to choose from, and their implementation as part of quality improvement and performance monitoring is extensive, there are still methodological challenges to be addressed. In this paper, we identify key methodological concerns around use of PRMs in dialysis centers in the United States and make recommendations for improving the use of PRMs in dialysis related to Selection of PRMs, Mode of Administration, and Support for PRM Use. These recommendations include: (1) Continue the use of Kidney Disease Quality of Life 36-item survey (KDQOL™-36) for dialysis centers' internal quality improvement activities and the In-Center Hemodialysis Consumer Assessment of Health Care Providers and Systems (ICH-CAHPS survey®) for public dialysis center performance monitoring, but promote efforts to modify these instruments by incorporating PROMIS general health items (KDQOL-36) and reducing the length of the ICH-CAHPS. (2) Adopt a PRM of whether dialysis patients have been informed about all dialysis and transplant options. (3) Evaluate equivalence between electronic and paper versions of PRMs prior to widespread use of electronic administration. (4) Explore reimbursement of costs of PRM administration by the Centers for Medicare and Medicaid Services and kidney organizations. (5) Continue development of provider trainings in PRM administration and interpretation. These recommendations will help dialysis care decision-makers, clinicians, and applied researchers take the next steps toward enhancing PRM use in dialysis.