Evaluating Digital Maturity and Patient Acceptability of Real-Time Patient Experience Feedback Systems: Systematic Review

Implementing a real-time patient experience feedback in inpatient rehabilitation: Process evaluation informed by the normalisation process theory

PURPOSE

Near real-time patient experience feedback (NRTPEF) can enable a patient-centric, immediate service recovery but has not been widely used in inpatient rehabilitation. We 1) assess the utility, feasibility, and acceptability of implementing a new NRTPEF, perceived by patients and providers; and 2) understand how the NRTPEF became embedded into routine provider practices.

MATERIALS AND METHODS

Mixed methods process evaluation of the 8-month implementation of an innovative NRTPEF in an inpatient rehabilitation unit, using interviews and focus groups with all the service-unit leaders and interviews with a randomised sample of patients. Beyond descriptive statistics and content analysis, the Normalisation Process Theory (NPT) informed a framework analysis.

RESULTS

Patients and service-unit leaders perceived high utility in the NRTPEF (median: 9 for both; 0-10 scale) and qualitative comments emphasised the value of providing/obtaining timely feedback. The system was found feasible and acceptable for patients (median: 9.5), but with an improvement margin for providers (median: 7.3). Suggestions include strengthening the data-relay format. Even in the pilot form, providers found the NRTPEF became embedded into practice (median 10; average: 8.6). The analysis based on the NPT shows how providers saw differential value, engaged with, and used the patient feedback into reconfigured practices.

CONCLUSION

An innovative NRTPEF was found useful, feasible and acceptable, but with refinement opportunities before scale-up.

Near real-time patient experience feedback with data relay to providers: a systematic review of its effectiveness

Near Real-Time Feedback (NRTF) on the patient's experience with care, coupled with data relay to providers, can inform quality-of-care improvements, including at the point of care. The objective is to systematically review contemporary literature on the impact of the use of NRTF and data relay to providers on standardized patient experience measures. Six scientific databases and five specialty journals were searched supplemented by snowballing search strategies, according to the registered study protocol. Eligibility included studies in English (2015-2023) assessing the impact of NRTF and data relay on standardized patient-reported experience measures as a primary outcome. Eligibility and quality appraisals were performed by two independent reviewers. An expert former patient (Patient and Family Advisory Council and communication sciences background) helped interpret the results. Eight papers met review eligibility criteria, including three randomized controlled trials (RCTs) and one non-randomized study. Three of these studies involved in-person NRTF prior to data relay (patient-level data for immediate corrective action or aggregated and peer-compared) and led to significantly better results in all or some of the experience measures. In turn, a kiosk-based NRTF achieved no better experience results. The remaining studies were pre-post designs with mixed or neutral results and greater risks of bias. In-person NRTF on the patient experience followed by rapid data relay to their providers, either patient-level or provider-level as peer-compared, can improve the patient experience of care. Reviewed kiosk-based or self-reported approaches combined with data relay were not effective. Further research should determine which approach (e.g. who conducts the in-person NRTF) will provide better, more efficient improvements and under which circumstances.

The use of patient experience feedback in rehabilitation quality improvement and codesign activities: Scoping review of the literature

Objective

To characterize the literature, reported enablers, and gaps on the use of patient experience feedback for person-centered rehabilitation quality improvement and codesign activities.

Design

Scoping Review.

Data sources

Scientific databases (PubMed, CINAHL, Rehabdata, Scopus, Web of Science, ProQuest), website searches (e.g. Beryl Institute), snowballing, and key-informant recommendations.

Methods

Two independent reviewers performed title and abstract screenings and full-text reviews. Eligibility focused on English-language, peer-reviewed (all time) and gray literature (last five years) that used patient experience feedback in rehabilitation improvement activities. The aims, settings, methods, findings, implications, and reported limitations were extracted, followed by content analyses identifying reported enablers and gaps.

Results

Among the 901 unique references and 52 full texts reviewed, ten were included: four used patient experience surveys for improving patient experiences; six used codesign methodologies to engage patient feedback in service improvement activities. Implementation enablers included securing managerial support, having a structured methodology and facilitator, using efficient processes, engaging staff experiences, and using appreciative inquiry. Reported study gaps included limited follow-up, low sample sizes, analytical limitations, lack of reported limitations, or narrow range of perspectives (e.g. not from people with severe impairments).

Conclusion

Few examples of the use of patient experience feedback in quality improvement or codesign activities were found in the rehabilitation literature. Patient experience improvement activities relied exclusively on retrospective survey data, which were not combined with often more actionable forms (e.g. qualitative, real time) of patient experience feedback. Further research might consider design of activities that collect and use patient experience feedback for rehabilitation service improvements.

Synthesizing Dimensions of Digital Maturity in Hospitals: Systematic Review

BACKGROUND

Digital health in hospital settings is viewed as a panacea for achieving the "quadruple aim" of health care, yet the outcomes have been largely inconclusive. To optimize digital health outcomes, a strategic approach is necessary, requiring digital maturity assessments. However, current approaches to assessing digital maturity have been largely insufficient, with uncertainty surrounding the dimensions to assess.

OBJECTIVE

The aim of this study was to identify the current dimensions used to assess the digital maturity of hospitals.

METHODS

A systematic literature review was conducted of peer-reviewed literature (published before December 2020) investigating maturity models used to assess the digital maturity of hospitals. A total of 29 relevant articles were retrieved, representing 27 distinct maturity models. The articles were inductively analyzed, and the maturity model dimensions were extracted and consolidated into a maturity model framework.

RESULTS

The consolidated maturity model framework consisted of 7 dimensions: strategy; information technology capability; interoperability; governance and management; patient-centered care; people, skills, and behavior; and data analytics. These 7 dimensions can be evaluated based on 24 respective indicators.

CONCLUSIONS

The maturity model framework developed for this study can be used to assess digital maturity and identify areas for improvement.

Effect of Real-Time Feedback Devices on Primary Care Patient Experience Scores: A Cluster-Randomized Trial

Patient experience is a critical measure for ambulatory primary care, although it is unclear how to best improve patient experience scores. This study aimed to determine whether use of a real-time feedback (RTF) device improved patient experience scores in a cluster-randomized trial. The primary outcomes were change from baseline in 9 Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) question and domain scores most closely related to the RTF questions asked in a linear mixed effects model. There were no observed statistically significant intervention-related differences in CG-CAHPS scores in any of the 9 CG-CAHPS questions or domains (P = .12-.99). In intervention clinics, there were no statistically significant correlation between CG-CAHPS top box scores and RTF device scores (P = .23-.98). Clinics in an urban primary care network randomized to receive RTF devices did not significantly improve related CG-CAHPS question or domain scores nor were those scores correlated with RTF device scores. More research is needed to identify effective interventions to improve ambulatory primary care patient experience.

Differences in Mode Preferences, Response Rates, and Mode Effect Between Automated Email and Phone Survey Systems for Patients of Primary Care Practices: Cross-Sectional Study

BACKGROUND

A growing number of health care practices are adopting software systems that link with their existing electronic medical records to generate outgoing phone calls, emails, or text notifications to patients for appointment reminders or practice updates. While practices are adopting this software technology for service notifications to patients, its use for collection of patient-reported measures is still nascent.

OBJECTIVE

This study assessed the mode preferences, response rates, and mode effect for a practice-based automated patient survey using phone and email modalities to patients of primary care practices.

METHODS

This cross-sectional study analyzed responses and respondent demographics for a short, fully automated, telephone or email patient survey sent to individuals within 72 hours of a visit to their regular primary care practice. Each survey consisted of 5 questions drawn from a larger study's patient survey that all respondents completed in the waiting room at the time of their visit. Automated patient survey responses were linked to self-reported sociodemographic information provided on the waiting room survey including age, sex, reported income, and health status.

RESULTS

A total of 871 patients from 87 primary care practices in British Columbia, Ontario, and Nova Scotia, Canada, agreed to the automated patient survey and 470 patients (45.2%) completed all 5 questions on the automated survey. Email administration of the follow-up survey was preferred over phone-based administration, except among patients aged 75 years and older (P<.001). Overall, response rates for those who selected an emailed survey (369/606, 60.9%) were higher (P<.001) than those who selected the phone survey (101/265, 38.1%). This held true irrespective of age, sex, or chronic disease status of individuals. Response rates were also higher for email (range 57.4% [58/101] to 66.3% [108/163]) compared with phone surveys (range 36% [23/64] to 43% [10/23]) for all income groups except the lowest income quintile, which had similar response rates (email: 29/63, 46%; phone: 23/50, 46%) for phone and email modes. We observed moderate (range 64.6% [62/96] to 78.8% [282/358]) agreement between waiting room survey responses and those obtained in the follow-up automated survey. However, overall agreement in responses was poor (range 45.3% [43/95] to 46.2% [43/93]) for 2 questions relating to care coordination.

CONCLUSIONS

An automated practice-based patient experience survey achieved significantly different response rates between phone and email and increased response rates for email as income group rose. Potential mode effects for the different survey modalities may limit multimodal survey approaches. An automated minimal burden patient survey could facilitate the integration of patient-reported outcomes into care planning and service organization, supporting the move of our primary care practices toward a more responsive, patient-centered, continual learning system. However, practices must be attentive to furthering inequities in health care by underrepresenting the experience of certain groups in decision making based on the reach of different survey modes.