Overall scores as an alternative to global ratings in patient experience surveys; a comparison of four methods

Association between nurse-child communication and family caregivers' global ratings to hospital: a retrospective study

PURPOSE

Effective nurse-child communication is a fundamental aspect of delivering pediatric nursing care. Family caregivers' global ratings to hospital are considered a proxy-reported measure for assessing a child's inpatient stay experience. We investigate the associations between nurse-child communication and family caregivers' global ratings to hospital.

DESIGN AND METHODS

A retrospective analysis of a national child patient experience survey data was conducted. Patient experience with nurse-child communication and the family caregivers' global ratings of hospital were measured using the Child Hospital Consumer Assessment of Healthcare Providers and Systems. Hierarchical linear models were constructed to examine the association between nurse-child communication measures and family caregivers' global ratings to hospital.

RESULTS

Data from 1010 patients at six National Regional Centers for Pediatric in China were collected. The overall rating of hospitals and the willingness to recommend the hospital showed increasing trends as the nurse-child communication score increased. How often nurses encourage children to ask questions was significantly associated with family caregivers' overall ratings of hospital and the family caregivers' willingness to recommend the hospital.

CONCLUSIONS

Effective communication by nurses with the child is associated with significantly higher global ratings to the hospital by family caregivers during inpatient care. Encouraging children to ask questions is a promising contributor to caregivers' global ratings to hospital.

PRACTICE IMPLICATIONS

Pediatric nurses should emphasis encouraging children to ask questions for effective communication in nursing practice. Future research is also needed to develop more targeted strategies to assist pediatric nurse to communicate with child better.

Evaluation and countermeasures of contracted services of Chinese family doctors from demanders' point of view - a case study of a city

BACKGROUND

The "gatekeepers" for residents' health are their family doctors. The implementation of contracted services provided by family doctors is conducive to promoting hierarchical diagnosis and treatment and achieving the objective of providing residents comprehensive and full-cycle health services. Since its implementation in 2016, the contract service system for Chinese family doctors has yielded a number of results while also highlighting a number of issues that require further investigation. Consequently, the purpose of this study is to assess the impact of family doctors' contracted services in a Chinese city from the perspective of demanders (i.e., contracted residents), identify the weak links, and then propose optimization strategies.

METHODS

In this study, a city in Shandong Province, China was selected as the sample city. In January 2020, 1098 contracted residents (including 40.5% men and 59.5% women) from 18 primary medical institutions (including township health centers and community health centers) were selected for on-site investigation. Take the PCAT-AS(Adult Short) scale revised in Chinese as the research tool to understand the medical experience of contracted residents in primary medical institutions, and interview some family doctors and residents to obtain more in-depth information.

RESULTS

Among the four core dimensions of PCAT-AS, the score of Continuous was the highest (3.44 ± 0.58); The score of Coordinated was the lowest (3.08 ± 0.66); Among the three derived dimensions, the score of Family-centeredness was the highest (3.33 ± 0.65); The score of Culturally-competent was the lowest (2.93 ± 0.77). The types of contracting institutions, residents' age, marital status, occupation, and whether chronic diseases are confirmed are the influencing factors of PCAT scores.

CONCLUSION

The family doctors' contracted services in the city has achieved certain results. At the same time, there are still some problems, such as difficult access to outpatient services during non-working hours, incomplete service items, an imperfect referral system, and inadequate utilization of traditional Chinese medicine services, it is recommended that the government continue to enhance and increase its investment in relevant policies and funds. Primary medical institutions should improve the compensation mechanism for family doctors and increase their work enthusiasm, improve and effectively implement the two-way referral system, gradually form an orderly hierarchical pattern of medical treatment, provide diversified health services in accordance with their own service capacity and the actual needs of residents, and improve the utilization rate of traditional Chinese medicine services in primary medical institutions.

Measuring Regional Quality of Health Care Using Unsolicited Online Data: Text Analysis Study

Background

Regional population management (PM) health initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for this use. This study explored the addition of comments accompanying unsolicited online ratings to regional analyses.

Objective

The goal was to create additional insight for each PM initiative as well as overall comparisons between these initiatives by attempting to determine the reasoning and rationale behind a rating.

Methods

The Dutch Zorgkaart database provided the unsolicited ratings from 2008 to 2017 for the analyses. All ratings included both quantitative ratings as well as qualitative text comments. Nine PM regions were used to aggregate ratings geographically. Sentiment analyses were performed by categorizing ratings into negative, neutral, and positive ratings. Per category, as well as per PM initiative, word frequencies (ie, unigrams and bigrams) were explored. Machine learning—naïve Bayes and random forest models—was applied to identify the most important predictors for rating overall sentiment and for identifying PM initiatives.

Results

A total of 449,263 unsolicited ratings were available in the Zorgkaart database: 303,930 positive ratings, 97,739 neutral ratings, and 47,592 negative ratings. Bigrams illustrated that feeling like not being “taken seriously” was the dominant bigram in negative ratings, while bigrams in positive ratings were mostly related to listening, explaining, and perceived knowledge. Comparing bigrams between PM initiatives showed a lot of overlap but several differences were identified. Machine learning was able to predict sentiments of comments but was unable to distinguish between specific PM initiatives.

Conclusions

Adding information from text comments that accompany online ratings to regional evaluations provides insight for PM initiatives into the underlying reasons for ratings. Text comments provide useful overarching information for health care policy makers but due to a lot of overlap, they add little region-specific information. Specific outliers for some PM initiatives are insightful.

The Association Between Residency Learning Climate and Inpatient Care Experience in Clinical Teaching Departments in the Netherlands

PURPOSE

To examine the association between residency learning climate and inpatient care experience.

METHOD

The authors analyzed 1,201 evaluations of the residency learning climate (using the Dutch Residency Educational Climate Test questionnaire) and 6,689 evaluations of inpatient care experience (using the Consumer Quality Index Inpatient Hospital Care questionnaire) from 86 departments across 15 specialties in 18 hospitals in the Netherlands between 2013 and 2014. The authors used linear hierarchical panel analyses to study the associations between departments' overall and subscale learning climate scores and inpatient care experience global ratings and subscale scores, controlling for respondent- and department-level characteristics and correcting for multiple testing.

RESULTS

Overall learning climate was not associated with global department ratings (b = 0.03; 95% confidence interval -0.17 to 0.23) but was positively associated with specific inpatient care experience domains, including communication with doctors (b = 0.11; 0.02 to 0.20) and feeling of safety (b = 0.09; 0.01 to 0.17). Coaching and assessment was positively associated with communication with doctors (b = 0.22; 0.08 to 0.37) and explanation of treatment (b = 0.22; 0.08 to 0.36). Formal education was negatively associated with pain management (b = -0.16; -0.26 to -0.05), while peer collaboration was positively associated with pain management (b = 0.14; 0.03 to 0.24).

CONCLUSIONS

Optimizing the clinical learning environment is an important step toward ensuring high-quality residency training and patient care. These findings could help clinical teaching departments address those aspects of the learning environment that directly affect patient care.

Harvesting the wisdom of the crowd: using online ratings to explore care experiences in regions

Background

Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient’s experiences at a (regional) population level.

Methods

Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008–2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).

Results

In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3–8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.

Conclusions

Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.

Patient-centered primary care and self-rated health in 6 Latin American and Caribbean countries: Analysis of a public opinion cross-sectional survey

BACKGROUND

Despite the substantial attention to primary care (PC), few studies have addressed the relationship between patients' experience with PC and their health status in low-and middle-income countries. This study aimed to (1) test the association between overall patient-centered PC experience (OPCE) and self-rated health (SRH) and (2) identify specific features of patient-centered PC associated with better SRH (i.e., excellent or very good SRH) in 6 Latin American and Caribbean countries.

METHODS AND FINDINGS

We conducted a secondary analysis of a 2013 public opinion cross-sectional survey on perceptions and experiences with healthcare systems in Brazil, Colombia, El Salvador, Jamaica, Mexico, and Panama; the data were nationally representative for urban populations. We analyzed 9 features of patient-centered PC. We calculated OPCE score as the arithmetic mean of the PC features. OPCE score ranged from 0 to 1, where 0 meant that the participant did not have any of the 9 patient-centered PC experiences, while 1 meant that he/she reported having all these experiences. After testing for interaction on the additive scale, we analyzed countries pooled for aim 1, with an interaction term for Mexico, and each country separately for aim 2. We used multiple Poisson regression models double-weighted by survey and inverse probability weights to deal with the survey design and missing data. The study included 6,100 participants. The percentage of participants with excellent or very good SRH ranged from 29.5% in Mexico to 52.4% in Jamaica. OPCE was associated with reporting excellent or very good SRH in all countries: adjusting for socio-demographic and health covariates, patients with an OPCE score of 1 in Brazil, Colombia, El Salvador, Jamaica, and Panama were more likely to report excellent or very good SRH than those with a score of 0 (adjusted prevalence ratio [aPR] 1.61, 95% CI 1.37-1.90, p < 0.001); in Mexico, this association was even stronger (aPR 4.27, 95% CI 2.34-7.81, p < 0.001). The specific features of patient-centered PC associated with better SRH differed by country. The perception that PC providers solve most health problems was associated with excellent or very good SRH in Colombia (aPR 1.38, 95% CI 1.01-1.91, p = 0.046) and Jamaica (aPR 1.21, 95% CI 1.02-1.43, p = 0.030). Having a provider who knows relevant medical history was positively associated with better SRH in Mexico (aPR 1.47, 95% CI 1.03-2.12, p = 0.036) but was negatively associated with better SRH in Brazil (aPR 0.71, 95% CI 0.56-0.89, p = 0.003). Finally, easy contact with PC facility (Mexico: aPR 1.35, 95% CI 1.04-1.74, p = 0.023), coordination of care (Mexico: aPR 1.53, 95% CI 1.19-1.98, p = 0.001), and opportunity to ask questions (Brazil: aPR 1.42, 95% CI 1.11-1.83, p = 0.006) were each associated with better SRH. The main study limitation consists in the analysis being of cross-sectional data, which does not allow making causal inferences or identifying the direction of the association between the variables.

CONCLUSIONS

Overall, a higher OPCE score was associated with better SRH in these 6 Latin American and Caribbean countries; associations between specific characteristics of patient-centered PC and SRH differed by country. The findings underscore the importance of high-quality, patient-centered PC as a path to improved population health.

Impact of patient characteristics on the Canadian Patient Experiences Survey-Inpatient Care: survey analysis from an academic tertiary care centre

OBJECTIVE

To determine the role of patient demographics, care domains and self-perceived health status in the analysis and interpretation of results from the Canadian Patient Experience Survey-Inpatient Care.

DESIGN

Cross-sectional survey.

SETTING

Single large Canadian two campus tertiary care academic centre.

PARTICIPANTS

Random sampling of hospital patients postdischarge.

INTERVENTION AND MAIN OUTCOME MEASURES

Logistic regression models were developed to analyse topbox scoring on four questions of global care (rate experience, recommend hospital, rate hospital, overall helped). Means of each composite domain were correlated to the four overall scores at the patient level to determine Spearman's rank correlation coefficients which were plotted against the overall (hospital) domain score for the key driver analysis.

RESULTS

Topbox scoring was decreased with worse degrees of perceived physical and mental health in all four global questions (p<0.05). Female gender and higher levels of education were associated with worse scoring on rate experience, recommend hospital and rate hospital (p<0.001). Whereas there was a significant difference between hospital departments in unadjusted measures, these differences were no longer evident after adjustment with patient covariates. Key driver analysis identified person-centred care, care transition and the domain related to emergency admission as areas of highest potential for improvement.

CONCLUSIONS

Global measures of overall care are influenced by patient-perceived physical and mental health. Caution should be exercised in using patient-satisfaction surveys to compare performance between different healthcare provision entities, as apparent differences could be explained by variation in patient mix rather than variation in performance.

Closing the patient experience chasm: A two-level validation of the Consumer Quality Index Inpatient Hospital Care

BACKGROUND

Evaluation of patients' health care experiences is central to measuring patient-centred care. However, different instruments tend to be used at the hospital or departmental level but rarely both, leading to a lack of standardization of patient experience measures.

OBJECTIVE

To validate the Consumer Quality Index (CQI) Inpatient Hospital Care for use on both department and hospital levels.

DESIGN

Using cross-sectional observational data, we investigated the internal validity of the questionnaire using confirmatory factor analyses (CFA), and the generalizability of the questionnaire for use at the department and hospital levels using generalizability theory.

SETTING AND PARTICIPANTS

22924 adults hospitalized for ≥24 hours between 1 January 2013 and 31 December 2014 in 23 Dutch hospitals (515 department evaluations).

MAIN VARIABLE

CQI Inpatient Hospital Care questionnaire.

RESULTS

CFA results showed a good fit on individual level (CFI=0.96, TLI=0.95, RMSEA=0.04), which was comparable between specialties. When scores were aggregated to the department level, the fit was less desirable (CFI=0.83, TLI=0.81, RMSEA=0.06), and there was a significant overlap between communication with doctors and explanation of treatment subscales. Departments and hospitals explained ≤5% of total variance in subscale scores. In total, 4-8 departments and 50 respondents per department are needed to reliably evaluate subscales rated on a 4-point scale, and 10 departments with 100-150 respondents per department for binary subscales.

DISCUSSION AND CONCLUSIONS

The CQI Inpatient Hospital Care is a valid and reliable questionnaire to evaluate inpatient experiences in Dutch hospitals provided sufficient sampling is done. Results can facilitate meaningful comparisons and guide quality improvement activities in individual departments and hospitals.

Investigating the Potential Contribution of Patient Rating Sites to Hospital Supervision: Exploratory Results From an Interview Study in the Netherlands

Background

Over the last decades, the patient perspective on health care quality has been unconditionally integrated into quality management. For several years now, patient rating sites have been rapidly gaining attention. These offer a new approach toward hearing the patient’s perspective on the quality of health care.

Objective

The aim of our study was to explore whether and how patient reviews of hospitals, as reported on rating sites, have the potential to contribute to health care inspector’s daily supervision of hospital care.

Methods

Given the unexplored nature of the topic, an interview study among hospital inspectors was designed in the Netherlands. We performed 2 rounds of interviews with 10 senior inspectors, addressing their use and their judgment on the relevance of review data from a rating site.

Results

All 10 Dutch senior hospital inspectors participated in this research. The inspectors initially showed some reluctance to use the major patient rating site in their daily supervision. This was mainly because of objections such as worries about how representative they are, subjectivity, and doubts about the relevance of patient reviews for supervision. However, confrontation with, and assessment of, negative reviews by the inspectors resulted in 23% of the reviews being deemed relevant for risk identification. Most inspectors were cautiously positive about the contribution of the reviews to their risk identification.

Conclusions

Patient rating sites may be of value to the risk-based supervision of hospital care carried out by the Health Care Inspectorate. Health care inspectors do have several objections against the use of patient rating sites for daily supervision. However, when they are presented with texts of negative reviews from a hospital under their supervision, it appears that most inspectors consider it as an additional source of information to detect poor quality of care. Still, it should always be accompanied and verified by other quality and safety indicators. More research on the value and usability of patient rating sites in daily hospital supervision and other health settings is needed.

The Net Promoter Score--an asset to patient experience surveys?

BACKGROUND

In the search for more straightforward ways of summarizing patient experiences and satisfaction, there is growing interest in the Net Promoter Score (NPS): How likely is it that you would recommend our company to a friend or colleague?

OBJECTIVE

To assess what the NPS adds to patient experience surveys. The NPS was tested against three other constructs already used in current surveys to summarize patient experiences and satisfaction: global ratings, recommendation questions and overall scores calculated from patient experiences. To establish whether the NPS is a valid measure for summarizing patient experiences, its association with these experiences should be assessed.

METHODS

Associations between the NPS and the three other constructs were assessed and their distributions were compared. Also, the association between the NPS and patient experiences was assessed. Data were used from patient surveys of inpatient hospital care (N = 6018) and outpatient hospital care (N = 10 902) in six Dutch hospitals.

RESULTS

Analyses showed that the NPS was moderately to strongly correlated with the other three constructs. However, their distributions proved distinctly different. Furthermore, the patient experiences from the surveys showed weaker associations with the NPS than with the global rating and the overall score.

CONCLUSIONS

Because of the limited extent to which the NPS reflects the survey results, it seems less valid as a summary of patient experiences than a global rating, the existing recommendation question or an overall score calculated from patient experiences. In short, it is still unclear what the NPS specifically adds to patient experience surveys.

A short generic patient experience questionnaire: howRwe development and validation

Background

Patient experience is a key quality outcome for modern health services, but most existing survey methods are long and setting-specific. We identified the need for a short generic questionnaire for tracking patient experience.

Methods

We describe the development and validation of the howRwe questionnaire. This has two items relating to clinical care (treat you kindly; listen and explain) and two items relating to the organisation of care (see you promptly; well organised) as perceived by patients. Each item has four responses (excellent, good, fair and poor). The questionnaire was trialled in 828 patients in an orthopaedic pre-operative assessment clinic (PAC).

Results

The howRwe questionnaire is shorter (29 words) and more readable (Flesch-Kincaid grade score 2.2) than other questionnaires with broadly similar objectives. Psychometric properties in this sample are good with Cronbach’s α=0.82. Following a change to the appointments system in the clinic, howRwe showed improvement in promptness and organisation, but not in kindness and communication, showing that it can distinguish between the clinical and organisational aspects of patient experience.

Conclusions

howRwe meets the criteria for a short generic patient experience questionnaire that is suitable for frequent use. In the validation study of PAC patients, it showed good psychometric properties and concurrent, construct and discriminant validity.