Innovating From Within: A Process Model for User-Centered Digital Development in Academic Medical Centers

The development and pilot of a novel mobile application to assess clinician perception of workload and work environment

BACKGROUND

Traditional measures of workload such as wRVUs may not be adequate to understand the impact of workload on key outcomes.

OBJECTIVE

The objective of this study was to develop a mobile application to assess, in near real time, clinicians' perception of workload and work environment.

DESIGNS, SETTINGS AND PARTICIPANTS

We developed the GrittyWork™ application (GW App) using the Chokshi and Mann process model for user-centered digital development. Study occured at a single academic medical center with hospitalist clinicians.

MAIN OUTCOME MEASURES AND MEASURES

Measures included the System Usability Scale (SUS), use measures from GW App, electronic health record (EHR) event log data and note counts, and qualitative interviews.

RESULTS

From October 28, 2022 to November 3, 2022, six hospitalist clinicians provided feedback on the early prototype of the GW App, and from February 28, 2023 to June 8, 2023, 30 hospitalist clinicians participated in the pilot while on clinical service. All 30 clinicians (100%) participated in the pilot submitting data for a total of 122 shifts. Participants reported working 10 ± 1 h per day (mean ± SD) and were responsible for an average of 11 ± 3 patients per day. The postpilot evaluation of the GW App showed a SUS score of 86 ± 11 and a participant preference toward mobile application-based surveys (73% of participants). Regarding workload measures, EHR event log data and notes data correlated with physician-reported workloads. Applying user-centered design techniques, we successfully developed a mobile application with high usability. These data can be paired with EHR event log data and outcomes to provide insights into the impact of workloads and work environments on outcomes.

A pre-implementation examination of barriers and facilitators of an electronic prospective surveillance model for cancer rehabilitation: a qualitative study

BACKGROUND

An electronic Prospective Surveillance Model (ePSM) uses patient-reported outcomes to monitor symptoms along the cancer pathway for timely identification and treatment. Randomized controlled trials show that ePSMs can effectively manage treatment-related adverse effects. However, an understanding of optimal approaches for implementing these systems into routine cancer care is limited. This study aimed to identify barriers and facilitators prior to the implementation of an ePSM to inform the selection of implementation strategies.

METHODS

A qualitative study using virtual focus groups and individual interviews was conducted with cancer survivors, oncology healthcare providers, and clinic leadership across four cancer centres in Canada. The Consolidated Framework for Implementation Research (CFIR) guided the interviews and analysis of barriers and facilitators based on five domains (intervention characteristics, individual characteristics, inner setting, outer setting, and process).

RESULTS

We conducted 13 focus groups and nine individual interviews with 13 patient participants and 56 clinic staff. Of the 39 CFIR constructs, 18 were identified as relevant determinants to the implementation. The adaptability, relative advantage, and complexity of an ePSM emerged as key intervention-level factors that could influence implementation. Knowledge of the system was important at the individual level. Within the inner setting, major determinants were the potential fit of an ePSM with clinical workflows (compatibility) and the resources that could be dedicated to the implementation effort (readiness for implementation). In the outer setting, meeting the needs of patients and the availability of rehabilitation supports were key determinants. Engaging various stakeholders was critical at the process level.

CONCLUSIONS

Improving the implementation of ePSMs in routine cancer care has the potential to facilitate early identification and management of treatment-related adverse effects, thereby improving quality of life. This study provides insight into important factors that may influence the implementation of an ePSM, which can be used to select appropriate implementation strategies to address these factors.

Applying Customer Discovery Method to a Chronic Disease Self-Management Mobile App: Qualitative Study

BACKGROUND

A significant health challenge is evident in the United States, with 6 in 10 adults having a chronic disease and 4 in 10 adults having 2 or more. Chronic disease self-management aims to prevent or delay disease progression and disability and reduce mortality risk. The evidence to support the use of information technology tools, including mobile apps, web-based portals, and web-based educational interventions, that support disease self-management and improve clinical outcomes is growing. Customer discovery and value proposition design methodology is a form of stakeholder engagement and is based on marketing and lean start-up business methods. As applied in health care, customer discovery and value proposition methodology can be used to understand the clinical problem and articulate the product's hypothesized unique value proposition relative to alternative options that are available to end users.

OBJECTIVE

This study aims to describe the experience and findings of academic researchers applying the customer discovery and value proposition methodology to identify stakeholders, needs, adaptability, and sustainability of a chronic disease self-management mobile app (CDapp). The motivation of the work is to make mobile health app interventions accessible and acceptable for all segments of patients' chronic diseases.

METHODS

Data were obtained through key informant interviews and analyzed using rapid qualitative analysis techniques. The value proposition framework was used to build the interview guide. The aim was to identify the needs, challenges (pains), and potential benefits (gains) of the CDapp for our stakeholders.

RESULTS

Our results showed that the primary consumers (end users) of a CDapp were the patients. The app adopters (decision makers) can be medical center leaders including population health department managers or insurance providers, while the consumer adoption influencers (influencers or saboteurs) are clinicians and patient caregivers. We developed an ecosystem map to visualize the clinical practice workflow and how an app for chronic disease management might integrate within an academic health care center or system. A value proposition for the identified customer segments was generated. Each stakeholder segment was working within a different framework to improve patient self-management. Patients needed help to adhere to self-care activities and they needed tailored health education. Health care leaders aim to improve the quality of care while reducing costs and workload. Clinicians wanted to improve patient education and care while reducing the time burden. Our results also showed that within academic medical centers, there were variations regarding patients' self-reported abilities to manage their diseases.

CONCLUSIONS

Customer discovery is a useful form of stakeholder engagement when designing studies that seek to implement, adapt, and sustain an intervention. The customer discovery and value proposition methodology can be used as an alternative or complementary approach to formative research to generate valuable information in a brief period.

Human-Centered Design of a Clinical Decision Support for Anemia Screening in Children with Inflammatory Bowel Disease

BACKGROUND

Inflammatory bowel disease (IBD) commonly leads to iron deficiency anemia (IDA). Rates of screening and treatment of IDA are often low. A clinical decision support system (CDSS) embedded in an electronic health record could improve adherence to evidence-based care. Rates of CDSS adoption are often low due to poor usability and fit with work processes. One solution is to use human-centered design (HCD), which designs CDSS based on identified user needs and context of use and evaluates prototypes for usefulness and usability.

OBJECTIVES

this study aimed to use HCD to design a CDSS tool called the IBD Anemia Diagnosis Tool, IADx.

METHODS

Interviews with IBD practitioners informed creation of a process map of anemia care that was used by an interdisciplinary team that used HCD principles to create a prototype CDSS. The prototype was iteratively tested with "Think Aloud" usability evaluation with clinicians as well as semi-structured interviews, a survey, and observations. Feedback was coded and informed redesign.

RESULTS

Process mapping showed that IADx should function at in-person encounters and asynchronous laboratory review. Clinicians desired full automation of clinical information acquisition such as laboratory trends and analysis such as calculation of iron deficit, less automation of clinical decision selection such as laboratory ordering, and no automation of action implementation such as signing medication orders. Providers preferred an interruptive alert over a noninterruptive reminder.

CONCLUSION

Providers preferred an interruptive alert, perhaps due to the low likelihood of noticing a noninterruptive advisory. High levels of desire for automation of information acquisition and analysis with less automation of decision selection and action may be generalizable to other CDSSs designed for chronic disease management. This underlines the ways in which CDSSs have the potential to augment rather than replace provider cognitive work.

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: a Prospective Improvement Study

BACKGROUND

The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete of clinically significant "signals". There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating.

OBJECTIVE

To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation.

METHODS

This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System. A group of physician leaders representing each campus met biweekly for six months. The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS). Primary outcomes measures after go-live include template utilization rate and note length in characters.

RESULTS

In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use. 68% strongly agreed that they would like to use the template frequently. In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions.

CONCLUSIONS

A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes.

CLINICALTRIAL

User-centered design of a telehealth-enhanced hybrid cardiac rehabilitation program as hospital quality improvement

Background Innovative program designs and strategies are needed to support the widespread uptake of cardiac rehabilitation (CR) programs in the post-COVID19 era. We combined user-centered design (UCD) and implementation science (ImS) principles to design a novel telehealth-enhanced hybrid (home and clinic-based) CR (THCR) program. Methods As part of a New York Presbyterian Hospital (NYPH) quality improvement initiative (March 2020-February 2022), we designed a THCR program using an iterative 3 step UCD process informed by the Theoretical Domains Framework and Consolidated Framework for Implementation Research to: 1) identify user and contextual barriers to CR uptake (stakeholder interviews), 2) design an intervention prototype (design workshops and journey mapping), and 3) refine the prototype (usability testing). The process was optimized for usability and implementation outcomes. Results Step 1: Semi-structured interviews with stakeholders (n = 9) at 3 geographically diverse academic medical centers revealed behavioral (e.g., self-efficacy, knowledge) and contextual (e.g., social distancing guidelines, physical space, staffing, reimbursement) barriers to uptake. Step 2: Design workshops (n = 20) and journey-mapping sessions (n = 3) with multi-disciplinary NYPH stakeholders (e.g., digital health team, CR clinicians, creative director) yielded a THCR prototype that leveraged NYPH's investment in their remote patient monitoring (RPM) platform to optimize feasibility of home-based CR sessions. Step 3: Usability testing with CR clinicians (n = 2) administering and CR patients (n = 3) participating in home-based sessions revealed usability challenges (e.g., RPM devices/exercise equipment usability; Wi-Fi/Bluetooth connectivity/syncing; patient safety/knowledge and protocol flexibility). Design workshops (n = 24) and journey-mapping sessions (n = 3) yielded design solutions (e.g., onboarding sessions, safety surveys, fully supervised remote sessions) and a refined THCR prototype. Conclusion Combining UCD and ImS methods while engaging multi-disciplinary stakeholders in an iterative process yielded a theory-informed telehealth-enhanced hybrid CR program targeting user and contextual barriers to real-world CR implementation. We provide a detailed summary of the process, and guidance for incorporating UCD and ImS methods in early-stage intervention development. THCR may shrink the evidence-to-practice gap in CR implementation. A future hybrid type I effectiveness-implementation trial will determine its feasibility, acceptability, and effectiveness.

Provider Perspectives on Patient- and Provider-Facing High Blood Pressure Clinical Decision Support

BACKGROUND

Hypertension, persistent high blood pressures (HBP) leading to chronic physiologic changes, is a common condition that is a major predictor of heart attacks, strokes, and other conditions. Despite strong evidence, care teams and patients are inconsistently adherent to HBP guideline recommendations. Patient-facing clinical decision support (CDS) could help improve recommendation adherence but must also be acceptable to clinicians and patients.

OBJECTIVE

This study aimed to partly address the challenge of developing a patient-facing CDS application, we sought to understand provider variations and rationales related to HBP guideline recommendations and perceptions regarding patient role and use of digital tools.

METHODS

We engaged hypertension experts and primary care respondents to iteratively develop and implement a pilot survey and a final survey which presented five clinical cases that queried clinicians' attitudes related to actions; variations; prioritization; patient input; importance; and barriers for HBP diagnosis, monitoring, and treatment. Analysis of Likert's scale scores was descriptive with content analysis for free-text answers.

RESULTS

Fifteen hypertension experts and 14 providers took the pilot and final version of the surveys, respectively. The majority (>80%) of providers felt the recommendations were important, yet found them difficult to follow-up to 90% of the time. Perceptions of relative amounts of patient input and patient work for effective HBP management ranged from 22 to 100%. Stated reasons for variation included adverse effects of treatment, patient comorbidities, shared decision-making, and health care cost and access issues. Providers were generally positive toward patient use of electronic CDS applications but worried about access to health care, nuance of recommendations, and patient understanding of the tools.

CONCLUSION

At baseline, provider management of HBP is heterogeneous. Providers were accepting of patient-facing CDS but reported preferences for that CDS to capture the complexity and nuance of guideline recommendations.

A Scalable Service to Improve Health Care Quality Through Precision Audit and Feedback: Proposal for a Randomized Controlled Trial

BACKGROUND

Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels.

OBJECTIVE

This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium.

METHODS

We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes.

RESULTS

The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024.

CONCLUSIONS

The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/34990.

Development of a computer-aided text message platform for user engagement with a digital Diabetes Prevention Program: a case study

Digital Diabetes Prevention Programs (dDPP) are novel mHealth applications that leverage digital features such as tracking and messaging to support behavior change for diabetes prevention. Despite their clinical effectiveness, long-term engagement to these programs remains a challenge, creating barriers to adherence and meaningful health outcomes. We partnered with a dDPP vendor to develop a personalized automatic message system (PAMS) to promote user engagement to the dDPP platform by sending messages on behalf of their primary care provider. PAMS innovates by integrating into clinical workflows. User-centered design (UCD) methodologies in the form of iterative cycles of focus groups, user interviews, design workshops, and other core UCD activities were utilized to defined PAMS requirements. PAMS uses computational tools to deliver theory-based, automated, tailored messages, and content to support patient use of dDPP. In this article, we discuss the design and development of our system, including key requirements and features, the technical architecture and build, and preliminary user testing.

The Effectiveness of a Multidisciplinary Electronic Discharge Readiness Tool: Prospective, Single-Center, Pre-Post Study

Background

In the face of hospital capacity strain, hospitals have developed multifaceted plans to try to improve patient flow. Many of these initiatives have focused on the timing of discharges and on lowering lengths of stay, and they have met with variable success. We deployed a novel tool in the electronic health record to enhance discharge communication.

Objective

The aim of this study is to evaluate the effectiveness of a discharge communication tool.

Methods

This was a prospective, single-center, pre-post study. Hospitalist physicians and advanced practice providers (APPs) used the Discharge Today Tool to update patient discharge readiness every morning and at any time the patient status changed throughout the day. Primary outcomes were tool use, time of day the clinician entered the discharge order, time of day the patient left the hospital, and hospital length of stay. We used linear mixed modeling and generalized linear mixed modeling, with team and discharging provider included in all the models to account for patients cared for by the same team and the same provider.

Results

During the pilot implementation period from March 5, 2019, to July 31, 2019, a total of 4707 patients were discharged (compared with 4558 patients discharged during the preimplementation period). A total of 352 clinical staff had used the tool, and 84.85% (3994/4707) of the patients during the pilot period had a discharge status assigned at least once. In a survey, most respondents reported that the tool was helpful (32/34, 94% of clinical staff) and either saved time or did not add additional time to their workflow (21/24, 88% of providers, and 34/34, 100% of clinical staff). Although improvements were not observed in either unadjusted or adjusted analyses, after including starting morning census per team as an effect modifier, there was a reduction in the time of day the discharge order was entered into the electronic health record by the discharging physician and in the time of day the patient left the hospital (decrease of 2.9 minutes per additional patient, P=.07, and 3 minutes per additional patient, P=.07, respectively). As an effect modifier, for teams that included an APP, there was a significant reduction in the time of day the patient left the hospital beyond the reduction seen for teams without an APP (decrease of 19.1 minutes per patient, P=.04). Finally, in the adjusted analysis, hospital length of stay decreased by an average of 3.7% (P=.06).

Conclusions

The Discharge Today tool allows for real time documentation and sharing of discharge status. Our results suggest an overall positive response by care team members and that the tool may be useful for improving discharge time and length of stay if a team is staffed with an APP or in higher-census situations.

What Makes for a Successful Digital Health Integrated Program of Work? Lessons Learnt and Recommendations From the Melbourne Children's Campus

Embedding digital technologies in healthcare has the potential to streamline and personalize medical care. However, healthcare systems are often fragmented, and therefore achieving a truly integrated digital health program can be challenging. To promote a streamlined, evidence-based approach to implementing digital health solutions in a healthcare system, the Murdoch Children's Research Institute (MCRI) established the Digital Health Translation and Implementation Program (DHTI) bringing together clinicians, researchers and digital health experts. From the program commencement, frontline clinical innovators have collaborated with DHTI team members to develop and implement digital solutions to address pain-points in the healthcare system. Throughout this program, important lessons have been learnt relating to the development, evaluation and implementation of digital solutions in the healthcare system. This paper explores these lessons and makes recommendations for the successful implementation of digital health solutions in healthcare systems under five main categories: (1) design and usability, (2) stakeholder engagement and uptake, (3) project management and resourcing, (4) process and implementation, and (5) evaluation. Recommendations suggested here are designed to support future healthcare-based digital health programs to maximize the impact digital solutions can have on the healthcare system and patients.

Mobile app helps trainees manage emergencies at the bedside

BACKGROUND

Although emergency departments (ED) have standardized guidelines for low-frequency, high-acuity diagnoses, they are not immediately accessible at the bedside, and this can cause anxiety in trainees and delay patient care. This problem is exacerbated during events like COVID-19 that require the rapid creation, iteration, and dissemination of new guidelines.

METHODS

Physician innovators used design thinking principles to develop EM Protocols (EMP), a mobile application that clinicians can use to immediately view guidelines, contact consultants (e.g., cath lab activation), and access code-running tools. The project became an institutional high priority, because it helps EM trainees and off-service rotators manage low-frequency, high-acuity emergencies at the point of care, and its COVID-19 guidelines can be rapidly updated and disseminated in real time.

RESULTS

This intervention was deployed across two academic medical centers during the COVID-19 surge. Nearly 300 ED clinicians have downloaded EMP, and they have interacted with the app over 5,400 times. It continues to be used regularly, over 12 months after the initial surge. Since the app was received positively, there are efforts to build in additional adult and pediatric guidelines.

DISCUSSION

Digital health tools like EMP can serve as invaluable adjuncts for managing acute, life-threatening emergencies at the point of care. They can benefit trainees during normal day-to-day operations as well as scenarios that cause large-scale operational disruptions, such as natural disasters, mass casualty events, and future pandemics.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center

Objective

To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations.

Materials and Methods

We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations.

Results

University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape.

Discussion

Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation.

Conclusion

EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Examining Challenges to the Incorporation of End Users in the Design of Digital Health Interventions: Protocol for a Systematic Review

Background

The process of designing a digital health intervention (DHI)—also referred to as mobile health or eHealth—spans needs assessments, technical functionality and feasibility, user satisfaction, effectiveness, impact, and value. These interventions are causing a rapid evolution in the landscape of health care. Multiple studies have shown their propensity to extend both the quality and reach of interventions. However, failure to improve DHI design is linked to failed uptake and health outcomes. This dilemma is further conflicted by the colliding backdrops of the digital and health industries, both of which approach, understand, and involve end users differently in the framing of a DHI.

Objective

The objective of this systematic review is to assess the challenges to incorporating end users in the design stage of digital health interventions, to identify key pain points, and to identify limitations and gaps for areas of future investigation.

Methods

The PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) checklist will be used to structure this protocol. A systematic search of the PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Scopus, and IEEE Xplore databases will be conducted. Additionally, the PerSPEcTiF guidelines for complex interventions will be consulted. Two reviewers will independently screen the titles and abstracts of the identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. Two reviewers will independently extract and validate data from the included studies into a standardized form and conduct quality appraisal.

Results

As of February 2021, we have completed a preliminary literature search examining challenges to the incorporation of end users in the design stage of DHIs. Systematic searches, data extraction and analysis, and writing of the systematic review are expected to be completed by December 2021.

Conclusions

This systematic review aims to provide an effective summary of key pain points toward incorporating end users in DHIs. Results from this review will provide an evidence base for a better approach to end user involvement in the interest of improving efficacy and uptake of DHIs.

Trial Registration

PROSPERO International Prospective Register of Systematic Reviews CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=238164

International Registered Report Identifier (IRRID)

PRR1-10.2196/28083

Digital Support to Multimodal Community-Based Prehabilitation: Looking for Optimization of Health Value Generation

Prehabilitation has shown its potential for most intra-cavity surgery patients on enhancing preoperative functional capacity and postoperative outcomes. However, its large-scale implementation is limited by several constrictions, such as: i) unsolved practicalities of the service workflow, ii) challenges associated to change management in collaborative care; iii) insufficient access to prehabilitation; iv) relevant percentage of program drop-outs; v) need for program personalization; and, vi) economical sustainability. Transferability of prehabilitation programs from the hospital setting to the community would potentially provide a new scenario with greater accessibility, as well as offer an opportunity to effectively address the aforementioned issues and, thus, optimize healthcare value generation. A core aspect to take into account for an optimal management of prehabilitation programs is to use proper technological tools enabling: i) customizable and interoperable integrated care pathways facilitating personalization of the service and effective engagement among stakeholders; ii) remote monitoring (i.e. physical activity, physiological signs and patient-reported outcomes and experience measures) to support patient adherence to the program and empowerment for self-management; and, iii) use of health risk assessment supporting decision making for personalized service selection. The current manuscript details a proposal to bring digital innovation to community-based prehabilitation programs. Moreover, this approach has the potential to be adopted by programs supporting long-term management of cancer patients, chronic patients and prevention of multimorbidity in subjects at risk.

Development and Implementation of a Multidisciplinary Electronic Discharge Readiness Tool: User-Centered Design Approach

BACKGROUND

Typical solutions for improving discharge planning often rely on one-way communication mechanisms, static data entry into the electronic health record (EHR), or in-person meetings. Lack of timely and effective communication can adversely affect patients and their care teams.

OBJECTIVE

Applying robust user-centered design strategies, we aimed to design an innovative EHR-based discharge readiness communication tool (the Discharge Today tool) to enable care teams to communicate any barriers to discharge, the status of patient discharge readiness, and patient discharge needs in real time across hospital settings.

METHODS

We employed multiple user-centered design strategies, including exploration of the current state for documenting discharge readiness and directing discharge planning, iterative low-fidelity prototypes, multidisciplinary stakeholder meetings, a brainwriting premortem exercise, and preproduction user testing. We iteratively collected feedback from users via meetings and surveys.

RESULTS

We conducted 28 meetings with 20 different stakeholder groups. From these stakeholder meetings, we developed 14 low-fidelity prototypes prior to deploying the Discharge Today tool for our pilot study. During the pilot study, stakeholders requested 46 modifications, of which 25 (54%) were successfully executed. We found that most providers who responded to the survey reported that the tool either saved time or did not change the amount of time required to complete their discharge workflow (21/24, 88%). Responses to open-ended questions offered both positive feedback and opportunities for improvement in the domains of efficiency, integration into workflow, avoidance of redundancies, expedited communication, and patient-centeredness.

CONCLUSIONS

Survey data suggest that this electronic discharge readiness tool has been successfully adopted by providers and clinical staff. Frequent stakeholder engagement and iterative user-centered design were critical to the successful implementation of this tool.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials

BACKGROUND

Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.

OBJECTIVE

This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.

METHODS

A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.

RESULTS

To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.

CONCLUSIONS

These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.

TRIAL REGISTRATION

Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

An Ecological Momentary Intervention for people with social anxiety: A descriptive case study

This study describes the development and pilot evaluation of a smartphone- delivered Ecological Momentary Intervention (EMI) for people with social anxiety symptoms. Using a software engineering framework (agile modeling, model-driven development, bottom-up development), mental health experts and software developers collaborated to develop a 4-module EMI app designed to reduce social anxiety in real-time. Fifty-five participants with social anxiety were randomly allocated to the EMI or a wait-list control arm. App downloads, usage and user satisfaction data were collected and mental health outcomes assessed at baseline and post-intervention. Software development practices allowed mental health experts to distil core elements of a psychological intervention into discrete software components but there were challenges in engaging mental health experts in the process. Relative to control there was no significant reduction in social anxiety among the EMI participants in the pilot trial. However, post-test data were available for only 4 intervention and 10 control participants and only 2 (4.0%) of the EMI participants downloaded the app. The two participants who both accessed the app and completed the post-test reported being satisfied with the intervention. Future research should address managing resources and providing additional training to support ongoing engagement with key stakeholders.

Development of a Health Information Technology Tool for Behavior Change to Address Obesity and Prevent Chronic Disease Among Adolescents: Designing for Dissemination and Sustainment Using the ORBIT Model

Health information technology (HIT) has not been broadly adopted for use in outpatient healthcare settings to effectively address obesity in youth, especially among disadvantaged populations that face greater barriers to good health. A well-designed HIT tool can deliver behavior change recommendations and provide community resources to address this gap, and the Obesity-Related Behavioral Intervention Trials (ORBIT) model can guide its development and refinement. This article reports the application of the ORBIT model to (1) describe the characteristics and design of a novel HIT tool (the PREVENT tool) using behavioral theory, (2) illustrate the use of stakeholder-centered "designing for dissemination and sustainability" principles, and (3) discuss the practical implications and directions for future research. Two types of stakeholder engagement (customer discovery and user testing) were conducted with end users (outpatient healthcare teams). Customer discovery interviews (n = 20) informed PREVENT tool components and intervention targets by identifying (1) what healthcare teams (e.g., physicians, dietitians) identified as their most important "jobs to be done" in helping adolescents who are overweight/obese adopt healthy behaviors, (2) their most critical "pains" and "gains" related to overweight/obesity treatment, and (3) how they define success compared to competing alternatives. Interviews revealed the need for a tool to help healthcare teams efficiently deliver tailored, evidence-based behavior change recommendations, motivate patients, and follow-up with patients within the constraints of clinic schedules and workflows. The PREVENT tool was developed to meet these needs. It facilitates prevention discussions, delivers tailored, evidence-based recommendations for physical activity and food intake, includes an interactive map of community resources to support behavior change, and automates patient follow-up. Based on Self-Determination Theory, the PREVENT tool engages the patient to encourage competence and autonomy to motivate behavior change. The use of this intentional, user-centered design process should increase the likelihood of the intended outcomes (e.g., behavior change, weight stabilization/loss) and ultimately increase uptake, implementation success, and long-term results. After initial tool development, user-testing interviews (n = 13) were conducted using a think-aloud protocol that provided insight into users' (i.e., healthcare teams) cognitive processes, attitudes, and challenges when using the tool. Overall, the PREVENT tool was perceived to be useful, well-organized, and visually appealing.

Development and evaluation of a decision aid for family surrogate decision-makers for patients with acute kidney injury requiring renal replacement therapy (RRT) in ICUs: a study protocol

INTRODUCTION

Shared decision making is endorsed by guidelines for both acute kidney injury and critical care medicine. However, there is still a huge need for effective interventions, especially those focusing on decisions about renal replacement therapy for intensive care unit (ICU) patients with acute kidney injury. The decision aids provide evidence-based support for shared decision making, to achieve better decisions through enhanced knowledge of treatment options and treatment aligns with patients' preferences and values. Therefore, our objectives are to develop and evaluate a decision aid systematically and rigorously for family surrogate decision makers of ICU patients with acute kidney injury who need renal replacement therapy.

METHODS AND ANALYSIS

We will use a systematic development process that focuses on user-centred design to develop and evaluate the decision aid in three phases: (1) development of a draft prototype for the decision aid based on extensive literature reviews, interviews with key stakeholders and evidence synthesis; (2) alpha testing ('near live' usability) the decision aid during simulated clinical encounters to test its comprehensibility, acceptability and usability and (3) beta testing ('live' usability) to examine the aid's clinical feasibility. User testing will be conducted using mixed-methods approach to support iterative revision of the decision aid. The IPDASi (V.4.0) will be used for following qualitative assessment. All interviews will be analysed by Colaizzi's seven-step approach to qualitative analysis. The coding scheme will use to analyse user interactions. Questionnaire surveys will be analysed using paired sample t-tests when related to the before-and-after survey, otherwise using one-sample t-test.

ETHICS AND DISSEMINATION

Ethical approval for this research was obtained from the Ethics Committee of the First Affiliated Hospital of Army Medical University, PLA (Ref: KY2020104). All participants will sign a formal informed consent form. The findings will be published in peer-reviewed journals and reported in appropriate meetings.

TRIAL REGISTRATION NUMBER

ChiCTR2000031613.

Effectiveness of an Integrated Engagement Support System to Facilitate Patient Use of Digital Diabetes Prevention Programs: Protocol for a Randomized Controlled Trial

Background

Digital diabetes prevention programs (dDPPs) are effective behavior change tools to prevent disease progression in patients at risk for diabetes. At present, these programs are poorly integrated into existing health information technology infrastructure and clinical workflows, resulting in barriers to provider-level knowledge of, interaction with, and support of patients who use dDPPs. Tools that can facilitate patient-provider interaction around dDPPs may contribute to improved patient engagement and adherence to these programs and improved health outcomes.

Objective

This study aims to use a rigorous, user-centered design (UCD) methodology to develop a theory-driven system that supports patient engagement with dDPPs and their primary care providers with their care.

Methods

This study will be conducted in 3 phases. In phase 1, we will use systematic UCD, Agile software development, and qualitative research methods to identify key user (patients, providers, clinical staff, digital health technologists, and content experts) requirements, constraints, and prioritization of high-impact features to design, develop, and refine a viable intervention prototype for the engagement system. In phase 2, we will conduct a single-arm feasibility pilot of the engagement system among patients with prediabetes and their primary care providers. In phase 3, we will conduct a 2-arm randomized controlled trial using the engagement system. Primary outcomes will be weight, BMI, and A_1c at 6 and 12 months. Secondary outcomes will be patient engagement (use and activity) in the dDPP. The mediator variables (self-efficacy, digital health literacy, and patient-provider relationship) will be measured.

Results

The project was initiated in 2018 and funded in September 2019. Enrollment and data collection for phase 1 began in September 2019 under an Institutional Review Board quality improvement waiver granted in July 2019. As of December 2020, 27 patients have been enrolled and first results are expected to be submitted for publication in early 2021. The study received Institutional Review Board approval for phases 2 and 3 in December 2020, and phase 2 enrollment is expected to begin in early 2021.

Conclusions

Our findings will provide guidance for the design and development of technology to integrate dDPP platforms into existing clinical workflows. This will facilitate patient engagement in digital behavior change interventions and provider engagement in patients’ use of dDPPs. Integrated clinical tools that can facilitate patient-provider interaction around dDPPs may contribute to improved patient adherence to these programs and improved health outcomes by addressing barriers faced by both patients and providers. Further evaluation with pilot testing and a clinical trial will assess the effectiveness and implementation of these tools.

Trial Registration

ClinicalTrials.gov NCT04049500; https://clinicaltrials.gov/ct2/show/NCT04049500

International Registered Report Identifier (IRRID)

DERR1-10.2196/26750

Building Health Services in a Rapidly Changing Landscape: Lessons in Adaptive Leadership and Pivots in a COVID-19 Remote Monitoring Program

Adaptive leadership has become an essential skill for leaders in health systems to respond to the COVID-19 pandemic as new knowledge emerges and case counts rise, fall, and rise again. This leadership approach has been described as an iterative process of taking a wide view of the situation, interpreting the meaning of incoming data from multiple directions, and taking real-time action. This process is also common in start-ups, which attempt to create new products or services of uncertain value for consumer markets that may not yet exist. Start-ups manage uncertainty through "pivots," which can include changes in the target group, need, features, or intended benefit of a product or service. Pivots are large changes that account for the high likelihood of getting something wrong during development, and they are distinct from the "tweaks" or small tests of change that define quality improvement methodology. This case study describes three pivots in the launch of a remote monitoring program for COVID-19. Adaptive leadership helped inform strategic decisions, with pivots providing a framework for internal and external stakeholders to articulate options for changes to address shifting needs. There is considerable uncertainty in the appropriate design and implementation of health services, and although this case example focuses on the use of adaptive leadership and pivots during a pandemic, these strategies are relevant for health care leaders at any time.

Impact of entrepreneurship training on clinician engagement in innovation creation: an evaluation of the Johns Hopkins Hexcite programme

Background

Academic health science centres are an ideal location to translate innovative discoveries into clinical practice. However, increased cost, decreased time and encroaching technology are few of the challenges that academic clinicians face in an increasingly digitised healthcare industry. Academic health science centres have begun creating training to involve clinicians in developing and deploying innovative solutions. Few of these programmes engage clinicians in interactive and interdisciplinary activities.

Approach

Hexcite is a 16-week entrepreneurship training programme at Johns Hopkins. During the programme, clinicians with innovative clinical software ideas learn how to launch start-ups. Clinicians accepted into the programme team up with a business expert, design expert and technical expert. Teams participate in 15 expert-led interactive 3-hour workshops, interview potential customers, regularly pitch their ideas to industry experts and iteratively refine their products.

Methods

This report examined anonymous participant feedback, quantitative data from team productivity reports, and interview responses between 2015 and 2019. Outcomes were assessed using the Kirkpatrick Model.

Results and conclusion

Many clinicians reported improved understanding of team building, design thinking and marketing communications as well as increased involvement in innovation. Many teams received funding after Hexcite. Outcomes from previous cohorts will guide more robust evaluation measures for future cohorts.

Giving Children With Osteogenesis Imperfecta a Voice: Participatory Approach for the Development of the Interactive Assessment and Communication Tool Sisom OI

BACKGROUND

Children with osteogenesis imperfecta (OI) experience acute and chronic symptoms that expose them to physical, mental, and social challenges. Empowering these children by involving them in their care can help them to cope with OI. Sisom is an interactive assessment and communication tool designed to help children aged 6-12 years with chronic illnesses express their symptoms. This tool has not yet been adapted to the unique needs of OI.

OBJECTIVE

The aim of this study was to develop a Sisom OI paper prototype by seeking the perspectives of end users.

METHODS

A participatory approach was adopted to develop the prototype overseen by an expert panel of 9 clinicians at a university-affiliated pediatric hospital. Purposive sampling was used to recruit 12 children with OI who were aged 6-12 years. The study was carried out over the course of 3 feedback cycles. Data were deductively interpreted using content analysis techniques.

RESULTS

Overall, 64% (57/89) of the Sisom symptoms were deemed relevant for inclusion in Sisom OI, with 42% (37/89) directly incorporated and 22% (20/89) incorporated with changes. In total, 114 symptoms were used to create the prototype, of which 57 were newly generated. The relevant symptoms addressed children's thoughts and feelings about hospitalization and their wishes for participation in their own care. The new symptoms addressed fractures, body image, and social isolation related to difficulties with accessibility and intimidation.

CONCLUSIONS

Once developed, Sisom OI will offer clinicians an innovative and child-centered approach to capture children's perspectives on their condition.

Investigation of a Mobile Health Texting Tool for Embedding Patient-Reported Data Into Diabetes Management (i-Matter): Development and Usability Study

Background

Patient-reported outcomes (PROs) are increasingly being used in the management of type 2 diabetes (T2D) to integrate data from patients’ perspective into clinical care. To date, the majority of PRO tools have lacked patient and provider involvement in their development, thus failing to meet the unique needs of end users, and lack the technical infrastructure to be integrated into the clinic workflow.

Objective

This study aims to apply a systematic, user-centered design approach to develop i-Matter (investigating a mobile health [mHealth] texting tool for embedding patient-reported data into diabetes management), a theory-driven, mobile PRO system for patients with T2D and their primary care providers.

Methods

i-Matter combines text messaging with dynamic data visualizations that can be integrated into electronic health records (EHRs) and personalized patient reports. To build i-Matter, we conducted semistructured group and individual interviews with patients with T2D and providers, a design thinking workshop to refine initial ideas and design the prototype, and user testing sessions of prototypes using a rapid-cycle design (ie, design-test-modify-retest).

Results

Using an iterative user-centered process resulted in the identification of 6 PRO messages that were relevant to patients and providers: medication adherence, dietary behaviors, physical activity, sleep quality, quality of life, and healthy living goals. In user testing, patients recommended improvements to the wording and timing of the PRO text messages to increase clarity and response rates. Patients also recommended including motivational text messages to help sustain engagement with the program. The personalized report was regarded as a key tool for diabetes self-management by patients and providers because it aided in the identification of longitudinal patterns in the PRO data, which increased patient awareness of their need to adopt healthier behaviors. Patients recommended adding individualized tips to the journal on how they can improve their behaviors. Providers preferred having a separate tab built into the EHR that included the personalized report and highlighted key trends in patients’ PRO data over the past 3 months.

Conclusions

PRO tools that capture patients’ well-being and the behavioral aspects of T2D management are important to patients and providers. A clinical trial will test the efficacy of i-Matter in 282 patients with uncontrolled T2D.

Trial Registration

ClinicalTrials.gov NCT03652389; https://clinicaltrials.gov/ct2/show/NCT03652389

Use of a human-centered design approach to adapt a nurse-led cardiovascular disease prevention intervention in HIV clinics

Stakeholder-informed strategies addressing cardiovascular disease (CVD) burden among people living with HIV (PWH) are needed within healthcare settings. This study provides an assessment of how human-centered design (HCD) guided the adaptation of a nurse-led intervention to reduce CVD risk among PWH. Using a HCD approach, research staff guided two multidisciplinary "design teams" in Ohio and North Carolina, with each having five HCD meetings. We conducted acceptability and feasibility testing. Six core recommendations were produced by two design teams of key stakeholders and further developed after the acceptability and feasibility testing to produce a final list of 14 actionable areas of adaptation. Acceptability and feasibility testing revealed areas for adaptation, e.g. patient preferences for communication and the benefit of additional staff to support patient follow-up. In conclusion, along with acceptability and feasibility testing, HCD led to the production of 14 key recommendations to enhance the effectiveness and scalability of an integrated HIV/CVD intervention.

User-Centered Development of a Behavioral Economics Inspired Electronic Health Record Clinical Decision Support Module

Changing physician behaviors is difficult. Electronic health record (EHR) clinical decision support (CDS) offers an opportunity to promote guideline adherence. Behavioral economics (BE) has shown success as an approach to supporting evidence-based decision-making with little additional cognitive burden. We applied a user-centered approach to incorporate BE “nudges” into a CDS module in two “vanguard” sites utilizing: (1) semi-structured interviews with key informants (n=8); (2) a design thinking workshop; and (3) semi-structured group interviews with clinicians. In the 133 day development phase at two clinics, the navigator section fired 299 times for 27 unique clinicians. The inbasket refill alert fired 124 times for 22 clinicians. Fifteen prescriptions for metformin were written by 11 clinicians. Our user-centered approach yielded a BE- driven CDS module with relatively high utilization by clinicians. Next steps include the addition of two modules and continued tracking of utilization, and assessment of clinical impact of the module.

Building digital innovation capacity at a large academic medical center

Academic medical centers (AMCs) today prioritize digital innovation. In efforts to develop and disseminate the best technology for their institutions, challenges arise in organizational structure, cross-disciplinary collaboration, and creative and agile problem solving that are essential for successful implementation. To address these challenges, the Digital DesignLab was created at NYU Langone Health to provide structured processes for assessing and supporting the capacity for innovative digital development in our research and clinical community. Digital DesignLab is an enterprise level, multidisciplinary, digital development team that guides faculty and student innovators through a digital development “pipeline”, which consists of intake, discovery, bootcamp, development. It also provides a framework for digital health innovation and dissemination at the institution. This paper describes the Digital DesignLab’s creation and processes, and highlights key lessons learned to support digital health innovation at AMCs.

Adaptive design of a clinical decision support tool: What the impact on utilization rates means for future CDS research

OBJECTIVE

We employed an agile, user-centered approach to the design of a clinical decision support tool in our prior integrated clinical prediction rule study, which achieved high adoption rates. To understand if applying this user-centered process to adapt clinical decision support tools is effective in improving the use of clinical prediction rules, we examined utilization rates of a clinical decision support tool adapted from the original integrated clinical prediction rule study tool to determine if applying this user-centered process to design yields enhanced utilization rates similar to the integrated clinical prediction rule study.

MATERIALS & METHODS: We conducted pre-deployment usability testing and semi-structured group interviews at 6 months post-deployment with 75 providers at 14 intervention clinics across the two sites to collect user feedback. Qualitative data analysis is bifurcated into immediate and delayed stages; we reported on immediate-stage findings from real-time field notes used to generate a set of rapid, pragmatic recommendations for iterative refinement. Monthly utilization rates were calculated and examined over 12 months.

RESULTS

We hypothesized a well-validated, user-centered clinical decision support tool would lead to relatively high adoption rates. Then 6 months post-deployment, integrated clinical prediction rule study tool utilization rates were substantially lower than anticipated based on the original integrated clinical prediction rule study trial (68%) at 17% (Health System A) and 5% (Health System B). User feedback at 6 months resulted in recommendations for tool refinement, which were incorporated when possible into tool design; however, utilization rates at 12 months post-deployment remained low at 14% and 4% respectively.

DISCUSSION

Although valuable, findings demonstrate the limitations of a user-centered approach given the complexity of clinical decision support.

CONCLUSION

Strategies for addressing persistent external factors impacting clinical decision support adoption should be considered in addition to the user-centered design and implementation of clinical decision support.

Bridging the Gap Between Academic Research and Pragmatic Needs in Usability: A Hybrid Approach to Usability Evaluation of Health Care Information Systems

BACKGROUND

Technology is increasingly embedded into the full spectrum of health care. This movement has benefited from the application of software development practices such as usability testing and agile development processes. These practices are frequently applied in both commercial or operational and academic settings. However, the relative importance placed on rapid iteration, validity, reproducibility, generalizability, and efficiency differs between the 2 settings and the needs and objectives of academic versus pragmatic usability evaluations.

OBJECTIVE

This paper explores how usability evaluation typically varies on key dimensions in pragmatic versus academic settings that impact the rapidity, validity, and reproducibility of findings and proposes a hybrid approach aimed at satisfying both pragmatic and academic objectives.

METHODS

We outline the characteristics of pragmatic versus academically oriented usability testing in health care, describe the tensions and gaps resulting from differing contexts and goals, and present a model of this hybrid process along with 2 case studies of digital development projects in which we demonstrate this integrated approach to usability evaluation.

RESULTS

The case studies presented illustrate design choices characteristic of our hybrid approach to usability evaluation.

CONCLUSIONS

Designed to leverage the strengths of both pragmatically and academically focused usability studies, a hybrid approach allows new development projects to efficiently iterate and optimize from usability data as well as preserves the ability of these projects to produce deeper insights via thorough qualitative analysis to inform further tool development and usability research by way of academically focused dissemination.