Usability of a Human Factors-based Clinical Decision Support in the Emergency Department: Lessons Learned for Design and Implementation

How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study

Clinical tools for use in practice-such as medicine reconciliation charts, diagnosis support tools and track-and-trigger charts-are endemic in healthcare, but relatively little attention is given to how to optimise their design. User-centred design approaches and co-design principles offer potential for improving usability and acceptability of clinical tools, but limited practical guidance is currently available. We propose a framework (FRamework for co-dESign of Clinical practice tOols or 'FRESCO') offering practical guidance based on user-centred methods and co-design principles, organised in five steps: (1) establish a multidisciplinary advisory group; (2) develop initial drafts of the prototype; (3) conduct think-aloud usability evaluations; (4) test in clinical simulations; (5) generate a final prototype informed by workshops. We applied the framework in a case study to support co-design of a prototype track-and-trigger chart for detecting and responding to possible fetal deterioration during labour. This started with establishing an advisory group of 22 members with varied expertise. Two initial draft prototypes were developed-one based on a version produced by national bodies, and the other with similar content but designed using human factors principles. Think-aloud usability evaluations of these prototypes were conducted with 15 professionals, and the findings used to inform co-design of an improved draft prototype. This was tested with 52 maternity professionals from five maternity units through clinical simulations. Analysis of these simulations and six workshops were used to co-design the final prototype to the point of readiness for large-scale testing. By codifying existing methods and principles into a single framework, FRESCO supported mobilisation of the expertise and ingenuity of diverse stakeholders to co-design a prototype track-and-trigger chart in an area of pressing service need. Subject to further evaluation, the framework has potential for application beyond the area of clinical practice in which it was applied.

Improving Healthcare Practice Through the Implementation of Human Factors and Ergonomics Principles

OBJECTIVE

Present a collection of papers focusing on improving healthcare practice through the implementation of human factors and ergonomics principles that were presented at the International Ergonomics Association (IEA) 2021 international conference.

BACKGROUND

The mission of the IEA is to elaborate and advance ergonomics science and practice and to expand its scope of application.

METHOD

We reviewed papers that were submitted for presentation at the IEA 2021 international conference and focused on improving healthcare practice through the implementation of human factors and ergonomics principles.

RESULTS

The eight papers that are included in this special issue cover varied aspects of human factors application and implementation.

CONCLUSION

This special issue provides clear evidence that the science of human factors is relevant and is continuing to grow and so is its implementation in healthcare.

APPLICATION

This special issue offers a selection of applied works, providing a wide scope of human factors guidelines, methods, and theories in healthcare work environments.

Post-implementation usability evaluation of a human factors-based clinical decision support for pulmonary embolism (PE) diagnosis (Dx): PE Dx Study Part 1

While there is promise for health IT, such as Clinical Decision Support (CDS), to improve patient safety and clinician efficiency, poor usability has hindered widespread use of these tools. Human Factors (HF) principles and methods remain the gold standard for health IT design; however, there is limited information on how HF methods and principles influence CDS usability "in the wild". In this study, we explore the usability of an HF-based CDS used in the clinical environment; the CDS was designed according to a human-centered design process, which is described in Carayon et al. (2020). In this study, we interviewed 12 emergency medicine physicians, identifying 294 excerpts of barriers and facilitators of the CDS. Sixty-eight percent of excerpts related to the HF principles applied in the human-centered design of the CDS. The remaining 32% of excerpts related to 18 inductively-created categories, which highlight gaps in the CDS design process. Several barriers were related to the physical environment and organization work system elements as well as physicians' broader workflow in the emergency department (e.g., teamwork). This study expands our understanding of the usability outcomes of HF-based CDS "in the wild". We demonstrate the value of HF principles in the usability of CDS and identify areas for improvement to future human-centered design of CDS. The relationship between these usability outcomes and the HCD process is explored in an accompanying Part 2 manuscript.

Retrospective analysis of the human-centered design process used to develop a clinical decision support in the emergency department: PE Dx Study Part 2

With the growing implementation and use of health IT such as Clinical Decision Support (CDS), there is increasing attention on the potential negative impact of these technologies on patients (e.g., medication errors) and clinicians (e.g., increased workload, decreased job satisfaction, burnout). Human-Centered Design (HCD) and Human Factors (HF) principles are recommended to improve the usability of health IT and reduce its negative impact on patients and clinicians; however, challenges persist. The objective of this study is to understand how an HCD process influences the usability of health IT. We conducted a systematic retrospective analysis of the HCD process used in the design of a CDS for pulmonary embolism diagnosis in the emergency department (ED). Guided by the usability outcomes (e.g., barriers and facilitators) of the CDS use "in the wild" (see Part 1 of this research in the accompanying manuscript), we performed deductive content analysis of 17 documents (e.g., design session transcripts) produced during the HCD process. We describe if and how the design team considered the barriers and facilitators during the HCD process. We identified 7 design outcomes of the HCD process, for instance designing a workaround and making a design change to the CDS. We identify gaps in the current HCD process and demonstrate the need for a continuous health IT design process.

Physicians' Perceptions of Clinical Decision Support to Treat Patients With Heart Failure in the ED

Importance

Clinical decision support (CDS) could help emergency department (ED) physicians treat patients with heart failure (HF) by estimating risk, collating relevant history, and assisting with medication prescribing if physicians' perspectives inform its design and implementation.

Objective

To evaluate CDS usability and workflow integration in the hands of ED physician end users who use it in clinical practice.

Design, Setting, and Participants

This mixed-methods qualitative study administered semistructured interviews to ED physicians from 2 community EDs of Kaiser Permanente Northern California in 2023. The interview guide, based on the Usability Heuristics for User Interface Design and the Sociotechnical Environment models, yielded themes used to construct an electronic survey instrument sent to all ED physicians.

Main Outcomes and Measures

Main outcomes were physicians' perceptions of using CDS to complement clinical decision-making, usability, and integration into ED clinical workflow.

Results

Seven key informant physicians (5 [71.4%] female, median [IQR] 15.0 [9.5-15.0] years in practice) were interviewed and survey responses from 51 physicians (23 [45.1%] female, median [IQR] 14.0 [9.5-17.0] years in practice) were received from EDs piloting the CDS intervention. Response rate was 67.1% (51 of 76). Physicians suggested changes to CDS accessibility, functionality, and workflow integration. Most agreed that CDS would improve patient care and fewer than half of physicians expressed hesitation about their capacity to consistently comply with its recommendations, citing workload concerns. Physicians preferred a passive prompt that encouraged, but did not mandate, interaction with the CDS.

Conclusions and Relevance

In this qualitative study of physicians who were using a novel CDS intervention to assist with ED management of patients with acute HF, several opportunities were identified to improve usability as well as several key barriers and facilitators to CDS implementation.

Usability and Utility of Human Immunodeficiency Virus Pre-exposure Prophylaxis Clinical Decision Support to Increase Knowledge and Pre-exposure Prophylaxis Initiations among Pediatric Providers

OBJECTIVES

 An effective clinical decision support system (CDSS) may address the current provider training barrier to offering preexposure prophylaxis (PrEP) to youth at risk for human immunodeficiency virus (HIV) infection. This study evaluated change in provider knowledge and the likelihood to initiate PrEP after exposure to a PrEP CDSS. A secondary objective explored perceived provider utility of the CDSS and suggestions for improving CDSS effectiveness.

METHODS

 This was a prospective study using survey responses from a convenience sample of pediatric providers who launched the interruptive PrEP CDSS when ordering an HIV test. McNemar's test evaluated change in provider PrEP knowledge and likelihood to initiate PrEP. Qualitative responses on CDSS utility and suggested improvements were analyzed using framework analysis and were connected to quantitative analysis elements using the merge approach.

RESULTS

 Of the 73 invited providers, 43 had available outcome data and were included in the analysis. Prior to using the CDSS, 86% of participants had never been prescribed PrEP. Compared to before CDSS exposure, there were significant increases in the proportion of providers who were knowledgeable about PrEP (p = 0.0001), likely to prescribe PrEP (p < 0.0001) and likely to refer their patient for PrEP (p < 0.0001). Suggestions for improving the CDSS included alternative "triggers" for the CDSS earlier in visit workflows, having a noninterruptive CDSS, additional provider educational materials, access to patient-facing PrEP materials, and additional CDSS support for adolescent confidentiality and navigating financial implications of PrEP.

CONCLUSION

 Our findings suggest that an interruptive PrEP CDSS attached to HIV test orders can be an effective tool to increase knowledge and likelihood to initiate PrEP among pediatric providers. Continual improvement of the PrEP CDSS based on provider feedback is required to optimize usability, effectiveness, and adoption. A highly usable PrEP CDSS may be a powerful tool to close the gap in youth PrEP access and uptake.