Protocol for implementation of the 'AusPROM' recommendations for elective surgery patients: a mixed-methods cohort study

Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation

BACKGROUND

Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients' experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM.

OBJECTIVE

This study examined the usability of an electronic real-time measurement system to monitor surgical patients' experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction.

METHODS

We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients' experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction).

RESULTS

A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access.

CONCLUSIONS

A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients' experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2023-079155.

Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review

OBJECTIVE

This systematic literature review aimed to identify factors that influence the implementation of electronic patient-reported outcome measures (ePROMs) and patient-reported experience measures (ePREMs) in healthcare settings.

INTRODUCTION

Improvements in health care through increased patient engagement have gained traction in recent years. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are tools used to improve the quality of care from the patient perspective. The influence of implementing PROMs and PREMs using electronic information systems (ePROMs and ePREMs) is not well understood.

INCLUSION CRITERIA

Studies with information related to the implementation of ePROMs and/or ePREMs with a focus on health-related services, irrespective of provider type, were included.

METHODS

A literature search of peer-reviewed databases was conducted on the 24th of January 2022 for articles about barriers and facilitators of the implementation of ePROMs/ePREMs in healthcare settings. Two reviewers independently extracted relevant findings from the included studies and performed a descriptive code-based synthesis before collaboratively creating a final consensus set of code categories, which were then mapped to the consolidated framework of implementation research (CFIR). Study quality was appraised using a mixed-methods appraisal tool (MMAT).

RESULTS

24 studies were eligible for inclusion in the screening of 626 nonduplicate studies. Quality assessment using the MMAT revealed that 20/24 studies met at least 60% of the MMAT criteria. Ninety-six code categories were identified and mapped to the constructs across all CFIR domains.

CONCLUSION

To guide the effective implementation of ePROMs/ePREMs in healthcare settings, factors shown to influence their implementation have been summarised as an implementation checklist for adoption and use by clinicians, organisations, and policymakers.

Implementing PROMS for elective surgery patients: feasibility, response rate, degree of recovery and patient acceptability

BACKGROUND

Patient reported outcome measures (PROMs) engage patients in co-evaluation of their health and wellbeing outcomes. This study aimed to determine the feasibility, response rate, degree of recovery and patient acceptability of a PROM survey for elective surgery.

METHODS

We sampled patients with a broad range of elective surgeries from four major Australian hospitals to evaluate (1) feasibility of the technology used to implement the PROMs across geographically dispersed sites, (2) response rates for automated short message service (SMS) versus email survey delivery formats, (3) the degree of recovery at one and four weeks post-surgery as measured by the Quality of Recovery 15 Item PROM (QoR-15), and (4) patient acceptability of PROMS based on survey and focus group results. Feasibility and acceptability recommendations were then co-designed with stakeholders, based on the data.

RESULTS

Over three months there were 5985 surveys responses from 20,052 surveys (30% response rate). Feasibility testing revealed minor and infrequent technical difficulties in automated email and SMS administration of PROMs prior to surgery. The response rate for the QoR-15 was 34.8% (n = 3108/8919) for SMS and 25.8% (n = 2877/11,133) for email. Mean QoR-15 scores were 122.1 (SD 25.2; n = 1021); 113.1 (SD 27.7; n = 1906) and 123.4 (SD 26.84; n = 1051) for pre-surgery and one and four weeks post-surgery, respectively. One week after surgery, 825 of the 1906 responses (43%) exceeded 122.6 (pre-surgery average), and at four weeks post-surgery, 676 of the 1051 responses (64%) exceeded 122.6 (pre-surgery average). The PROM survey was highly acceptable with 76% (n = 2830/3739) of patients rating 8/10 or above for acceptability. Fourteen patient driven recommendations were then co-developed.

CONCLUSION

Administering PROMS electronically for elective surgery hospital patients was feasible, acceptable and discriminated changes in surgical recovery over time. Patient co-design and involvement provided innovative and practical solutions to implementation and new recommendations for implementation. Trial Registration and Ethical Approval ACTRN12621000298819 (Phase I and II) and ACTRN12621000969864 (Phase III). Ethics approval has been obtained from La Trobe University (Australia) Human Research Ethics Committee (HEC20479).

KEY POINTS

Patient reported outcome measures (PROMs) help to engage patients in understanding their health and wellbeing outcomes. This study aimed to determine how patients feel about completing a PROM survey before and after elective surgery, and to develop a set of recommendations on how to roll out the survey, based on patient feedback. We found that implementing an electronic PROM survey before and after elective surgery was relatively easy to do and was well accepted by patients. Consumer feedback throughout the project enabled co-design of innovative and practical solutions to PROM survey administration.

Patient Judgement of Change with Elective Surgery Correlates with Patient Reported Outcomes and Quality of Life

Obtaining pre-surgery PROM measures is not always feasible. The aim of this study was to examine if self-reports of change following elective surgery correlate with change scores from a validated PROM (15-item Quality of Recovery (QoR-15)). This cross-sectional study across 29 hospitals enrolled elective surgery patients. PROMs were collected one-week pre-surgery, as well as one- and four-weeks post-surgery via an electronic survey. We examined associations between patient “judgement of change” at one and four-weeks after surgery and the actual pre-to post-surgery PROM change scores. A total of 4177 surveys were received. The correlation between patient judgement of change, and the actual change score was moderately strong at one-week (n = 247, rs = 0.512, p < 0.001), yet low at four-weeks (n = 241, rs = 0.340, p < 0.001). Patient judgement was aligned to the direction of the PROM change score from pre- to post-surgery. We also examined the correlation between the QoR-15 (quality of recovery) and the EQ-5D-5L (QOL). There was a moderately strong positive correlation between the two PROMs (n = 356, rs = 0.666, p < 0.001), indicating that change in quality of recovery was related to change in QOL. These findings support the use of a single “judgement of change” recall question post-surgery.