Example of a Human Factors Engineering approach to a medication administration work system: Potential impact on patient safety

Post-implementation usability evaluation of a human factors-based clinical decision support for pulmonary embolism (PE) diagnosis (Dx): PE Dx Study Part 1

While there is promise for health IT, such as Clinical Decision Support (CDS), to improve patient safety and clinician efficiency, poor usability has hindered widespread use of these tools. Human Factors (HF) principles and methods remain the gold standard for health IT design; however, there is limited information on how HF methods and principles influence CDS usability "in the wild". In this study, we explore the usability of an HF-based CDS used in the clinical environment; the CDS was designed according to a human-centered design process, which is described in Carayon et al. (2020). In this study, we interviewed 12 emergency medicine physicians, identifying 294 excerpts of barriers and facilitators of the CDS. Sixty-eight percent of excerpts related to the HF principles applied in the human-centered design of the CDS. The remaining 32% of excerpts related to 18 inductively-created categories, which highlight gaps in the CDS design process. Several barriers were related to the physical environment and organization work system elements as well as physicians' broader workflow in the emergency department (e.g., teamwork). This study expands our understanding of the usability outcomes of HF-based CDS "in the wild". We demonstrate the value of HF principles in the usability of CDS and identify areas for improvement to future human-centered design of CDS. The relationship between these usability outcomes and the HCD process is explored in an accompanying Part 2 manuscript.

Assessing the Impact of a New Pediatric Healthcare Facility on Medication Administration: A Human Factors Approach

OBJECTIVE

This observational descriptive study was designed to measure the effect a new evidence-based design (EBD) hospital has on pediatric medication safety.

BACKGROUND

Medication safety is a priority for nurse leaders. Controlling system design by increasing the understanding of the impact human factors have could improve medication delivery.

METHODS

Medication administration data from 2 studies conducted at the same hospital, 1 at an older facility in 2015 and the other at a new EBD facility in 2019, were compared using a similar research design.

RESULTS

Results indicate that rates of distractions per 100 drug administrations were all statistically significant, favoring the 2015 data regardless of the EBD. No statistically significant differences were observed in error rates of any type when comparing the data collected in the older facility versus the newer EBD facility.

CONCLUSION

This study demonstrated that EBD alone does not ensure the absence of medication errors. By comparing 2 data sets, unanticipated associations were found that could impact safety. Despite the new facility's contemporary design, distractions persisted that could inform nurse leaders in developing interventions to support a safer patient care environment using a human factors approach.

Implementation of An Antibiotic Stewardship Intervention to Reduce Prescription of Fluoroquinolones: A Human Factors Analysis in Two Intensive Care Units

Antibiotic use is often the target of interventions in health care organizations that aim to decrease healthcare-associated infections (HAI) such as Clostridioides difficile (CDI); this is particularly important for fluoroquinolones (FQ), which are frequently used in critical care settings. In this study, using a multiple case study research approach, we conduct an in-depth analysis of an intervention aimed at limiting ICU prescriber access to FQ in two ICUs of two hospitals. The data collection and analysis were guided by a human factors engineering approach based on the SEIPS (Systems Engineering Initiative for Patient Safety) model and evidence-based implementation principles. Our results show some differences in the implementation of the FQ intervention between the two ICUs, such as level and method of FQ restriction, and training and communication with physicians and pharmacists. In both ICUs, several organizational learning mechanisms helped to quickly identify problems with the intervention and ensure that changes were made in a just-in-time manner (e.g. just-in-time training, removal of FQ in order set for pneumonia). Despite their organizational differences, both sites developed strategies to successfully implement the FQ intervention.

Identifying Electronic Medication Administration Record (eMAR) Usability Issues from Patient Safety Event Reports

BACKGROUND

Improving our understanding of the association between medication errors and health information technology (health IT) usability has the potential to reduce errors and improve patient safety. This study used patient safety event reports (PSEs) to investigate the contribution of usability challenges associated with the electronic medication administration record (eMAR) to medication errors.

METHODS

Free-text descriptions of 849 medication-related PSEs selected from 2.3 million reports were analyzed. Coders identified the specific health IT components, usability challenge categories, and nuanced usability themes that contributed to each PSE. Thematic analysis was conducted to refine categorizations and identify emerging themes. Final analysis was limited to PSEs involving a contribution from eMAR, either as the point of origin or as a downstream contributor to error.

RESULTS

eMAR contributed to 473 PSEs. eMAR was the point of origin for 84 (17.8% of 473) PSEs. Usability challenge categories included Workflow support (n = 52, 11.0%) and Display/Visual clutter (n = 30, 6.3%). eMAR contributed downstream from the point of origin in 389 (82.2% of 473) PSEs, with errors stemming primarily from Pharmacy IT and computerized provider order entry (CPOE). Prominent secondary eMAR-associated usability challenges included Display/Visual clutter (n = 327, 69.1%) and Alerting (n = 32, 6.8%).

CONCLUSION

This study identified several eMAR usability challenges, through the analysis of PSEs, that contribute to medication errors. Findings highlight the critical need for improving the eMAR user interface. Improved interface design, better vendor usability testing, eMAR-focused certification testing, consideration of work system factors, and eMAR-focused usability and safety testing by health care facilities can improve eMAR technology and patient safety.

A work system analysis of the medication administration process in a Norwegian nursing home ward

Nursing home patients often have multiple diagnoses and a high prevalence of polypharmacy and are at risk of experiencing adverse drug events. The study aims to explore the dynamic interactions of stakeholders and work system elements in the medication administration process in a nursing home ward. Data were collected using observations and interviews. A deductive content analysis led to a SEIPS-based process map and an accompanying work system analysis. The study increases knowledge of the complexity of the medication administration process by portraying the dynamic interactions between the major stakeholders in the work system, and the temporal flow of the activities involved. Secondly, it identifies facilitators and barriers in the work system linked to the medication administration process. Most barriers and facilitators are associated with the work system elements - tools & technology, organisation and tasks - and occur early in the medication administration process.

Evaluating information security core human error causes (IS-CHEC) technique in public sector and comparison with the private sector

BACKGROUND

The number of reported public sector information security incidents has significantly increased recently including 22% related to the UK health sector. Over two thirds of these incidents pertain to human error, but despite this, there are limited published related works researching human error as it affects information security.

METHOD

This research conducts an empirical case study into the feasibility and implementation of the Information Security Core Human Error Causes (IS-CHEC) technique which is an information security adaptation of Human Error Assessment and Reduction Technique (HEART). We analysed 12 months of reported information security incidents for a participating public sector organisation providing healthcare services and mapped them to the IS-CHEC technique.

RESULTS

The results show that the IS-CHEC technique is applicable to the field of information security but identified that the underpinning HEART human error probability calculations did not align to the recorded incidents. The paper then proposes adaptation of the IS-CHEC technique based on the feedback from users during the implementation. We then compared the results against those of a private sector organisation established using the same approach.

CONCLUSIONS

The research concluded that the proportion of human error is far higher than reported in current literature. The most common causes of human error within the participating public sector organisation were lack of time for error detection and correction, no obvious means of reversing an unintended action and people performing repetitious tasks.

Medication adherence: staying within the boundaries of safety

An important domain of patient safety is the management of medications in home and community settings by patients and their caregiving network. This study applied human factors/ergonomics theories and methods to data about medication adherence collected from 61 patients with heart failure accompanied by 31 informal caregivers living in the US. Seventy non-adherence events were identified, described, and analysed for performance shaping factors. Half were classified as errors and half as violations. Performance shaping factors included elements of the person or team (e.g. patient limitations), task (e.g. complexity), tools and technologies (e.g. tool quality) and organisational, physical, and social context (e.g. resources, support, social influence). Study findings resulted in a dynamic systems model of medication safety applicable to patient medication adherence and the medication management process. Findings and the resulting model offer implications for future research on medication adherence, medication safety interventions, and resilience in home and community settings. Practitioner Summary: We describe situational and habitual errors and violations in medication use among older patients and their family members. Multiple factors pushed performance towards risk and harm. These factors can be the target for redesign or various forms of support, such as education, changes to the plan of care, and technology design.

An appraisal of published usability evaluations of electronic health records via systematic review

OBJECTIVE

In this systematic review, we aimed to evaluate methodological and reporting trends present in the current literature by investigating published usability studies of electronic health records (EHRs).

METHODS

A literature search was conducted for articles published through January 2015 using MEDLINE (Ovid), EMBASE, Scopus, and Web of Science, supplemented by citation and reference list reviews. Studies were included if they tested the usability of hospital and clinic EHR systems in the inpatient, outpatient, emergency department, or operating room setting.

RESULTS

A total of 4848 references were identified for title and abstract screening. Full text screening was performed for 197 articles, with 120 meeting the criteria for study inclusion.

CONCLUSION

A review of the literature demonstrates a paucity of quality published studies describing scientifically valid and reproducible usability evaluations at various stages of EHR system development. A lack of formal and standardized reporting of EHR usability evaluation results is a major contributor to this knowledge gap, and efforts to improve this deficiency will be one step of moving the field of usability engineering forward.

What do we know about developing patient portals? a systematic literature review

OBJECTIVE

Numerous articles have reported on the development of patient portals, including development problems and solutions. We review these articles to inform future patient portal development efforts and to provide a summary of the evidence base that can guide future research.

MATERIALS AND METHODS

We performed a systematic review of relevant literature to answer 5 questions: (1) What categories of problems related to patient portal development have been defined? (2) What causal factors have been identified by problem analysis and diagnosis? (3) What solutions have been proposed to ameliorate these causal factors? (4) Which proposed solutions have been implemented and in which organizational contexts? (5) Have implemented solutions been evaluated and what learning has been generated? Through searches on PubMed, ScienceDirect and LISTA, we included 109 articles.

RESULTS

We identified 5 main problem categories: achieving patient engagement, provider engagement, appropriate data governance, security and interoperability, and a sustainable business model. Further, we identified key factors contributing to these problems as well as solutions proposed to ameliorate them. While about half (45) of the 109 articles proposed solutions, fewer than half of these solutions (18) were implemented, and even fewer (5) were evaluated to generate learning about their effects.

DISCUSSION

Few studies systematically report on the patient portal development processes. As a result, the review does not provide an evidence base for portal development.

CONCLUSION

Our findings support a set of recommendations for advancement of the evidence base: future research should build on existing evidence, draw on principles from design sciences conveyed in the problem-solving cycle, and seek to produce evidence within various different organizational contexts.

A systematic review of human factors and ergonomics (HFE)-based healthcare system redesign for quality of care and patient safety

Healthcare systems need to be redesigned to provide care that is safe, effective and efficient, and meets the multiple needs of patients. This systematic review examines how human factors and ergonomics (HFE) is applied to redesign healthcare work systems and processes and improve quality and safety of care. We identified 12 projects representing 23 studies and addressing different physical, cognitive and organisational HFE issues in a variety of healthcare systems and care settings. Some evidence exists for the effectiveness of HFE-based healthcare system redesign in improving process and outcome measures of quality and safety of care. We assessed risk of bias in 16 studies reporting the impact of HFE-based healthcare system redesign and found varying quality across studies. Future research should further assess the impact of HFE on quality and safety of care, and clearly define the mechanisms by which HFE-based system redesign can improve quality and safety of care.

Human factors and health information technology: current challenges and future directions

OBJECTIVES

Recent federal mandates and incentives have spurred the rapid growth, development and adoption of health information technology (HIT). While providing significant benefits for better data integration, organization, and availability, recent reports have raised questions regarding their potential to cause medication errors, decreased clinician performance, and lowered efficiency. The goal of this survey article is to (a) examine the theoretical and foundational models of human factors and ergonomics (HFE) that are being advocated for achieving patient safety and quality, and their use in the evaluation of healthcare systems; (b) and the potential for macroergonomic HFE approaches within the context of current research in biomedical informatics.

METHODS

We reviewed literature (2007-2013) on the use of HFE approaches in healthcare settings, from databases such as Pubmed, CINAHL, and Cochran.

RESULTS

Based on the review, we discuss the systems-oriented models, their use in the evaluation of HIT, and examples of their use in the evaluation of EHR systems, clinical workflow processes, and medication errors. We also discuss the opportunities for better integrating HFE methods within biomedical informatics research and its potential advantages.

CONCLUSIONS

The use of HFE methods is still in its infancy - better integration of HFE within the design lifecycle, and quality improvement efforts can further the ability of informatics researchers to address the key concerns regarding the complexity in clinical settings and develop HIT solutions that are designed within the social fabric of the considered setting.

Tree testing of hierarchical menu structures for health applications

To address the need for greater evidence-based evaluation of Health Information Technology (HIT) systems we introduce a method of usability testing termed tree testing. In a tree test, participants are presented with an abstract hierarchical tree of the system taxonomy and asked to navigate through the tree in completing representative tasks. We apply tree testing to a commercially available health application, demonstrating a use case and providing a comparison with more traditional in-person usability testing methods. Online tree tests (N=54) and in-person usability tests (N=15) were conducted from August to September 2013. Tree testing provided a method to quantitatively evaluate the information structure of a system using various navigational metrics including completion time, task accuracy, and path length. The results of the analyses compared favorably to the results seen from the traditional usability test. Tree testing provides a flexible, evidence-based approach for researchers to evaluate the information structure of HITs. In addition, remote tree testing provides a quick, flexible, and high volume method of acquiring feedback in a structured format that allows for quantitative comparisons. With the diverse nature and often large quantities of health information available, addressing issues of terminology and concept classifications during the early development process of a health information system will improve navigation through the system and save future resources. Tree testing is a usability method that can be used to quickly and easily assess information hierarchy of health information systems.

Human factors and ergonomics as a patient safety practice

BACKGROUND

Human factors and ergonomics (HFE) approaches to patient safety have addressed five different domains: usability of technology; human error and its role in patient safety; the role of healthcare worker performance in patient safety; system resilience; and HFE systems approaches to patient safety.

METHODS

A review of various HFE approaches to patient safety and studies on HFE interventions was conducted.

RESULTS

This paper describes specific examples of HFE-based interventions for patient safety. Studies show that HFE can be used in a variety of domains.

CONCLUSIONS

HFE is a core element of patient safety improvement. Therefore, every effort should be made to support HFE applications in patient safety.

The fungibility of time in claims of efficiency: the case of making transmission of prescriptions electronic in English general practice

OBJECTIVES

This paper presents a study of the effects of the implementation of the NHS Electronic Prescription Service (EPS) on time spent on repeat prescribing in English general practice. EPS is a new network service for the electronic transmission of primary care prescriptions, principally between GP practices and community pharmacies. This service is promoted on the basis of the importance of safe and timely supply of medicines, and the level of medicines use by many patients with treatable chronic conditions. The service is also based on presumptions of significant time-savings and efficiency gains for general practices and GPs. Our objective was to assess the time-related changes (including time savings) conditioned by digital transmission of prescriptions, specifically for repeat prescribing activity in primary care practices.

METHODS

As part of the official evaluation of EPS in the English NHS we undertook a qualitative research design with field studies in four of the first GP practices adopting EPS. This research was based on interviews with clinical and administrative staff, and non-participant observation of repeat prescribing related activities.

RESULTS

We found that the use of EPS reduced turnaround time and conditioned changes in the workflow, with time-savings found mainly in relation to administrative tasks. But the use of this technology also created additional tasks and shifted existing tasks and responsibilities. Thus elimination of tasks did not automatically correspond to potential staff savings or cost savings. Tasks that were eliminated and new tasks that were created were not equivalent in terms of time spent, quality of attention required, and roles involved.

CONCLUSIONS

The wider claim that healthcare information technology saves time and increases efficiency is often based on assumptions of the fungibility of time and people - i.e. that units of time added or saved on different steps of the workflow can be summed up as if they were all of the same kind, and thus reveal any net efficiency gain. But workflow time savings involve changes in the quality of tasks, redistribution of work and responsibility that mean that time can hardly be added or subtracted to obtain 'efficiency totals'.

Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA

In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems. After a careful review of the literature and of vendor experiences with EHR design and implementation, the task force developed 10 recommendations in four areas: (1) human factors health information technology (IT) research, (2) health IT policy, (3) industry recommendations, and (4) recommendations for the clinician end-user of EHR software. These AMIA recommendations are intended to stimulate informed debate, provide a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

Assessment of efficiency and safety of the comprehensive Chemotherapy Assistance Program for ordering oncology medications

PURPOSE

To assess whether the newly developed Comprehensive Chemotherapy Assistance Program (CAP) decreases errors and increases performance efficiency in ordering oncology medications as compared to the existing Computerized Physician Order Entry (CPOE) system, the Order Communication System (OCS).

METHODS

All chemotherapy prescriptions ordered using OCS and CAP systems during the first year of CAP implementation were prospectively analyzed to identify near-miss cases. The efficiency outcomes were determined in a trial setting with eight predefined hypothetical chemotherapy regimens, each with four measures of the average time, movement distance on the screen, maximum input interval time, and number of mouse or keystrokes per order. A total of 14 physicians participated in the study, and each regimen was ordered twice, once using CAP and once using OCS.

RESULTS

For all near-miss types, CAP orders showed significant improvements as compared with OCS orders (p<0.0001). For CAP orders, incorrect dose and agent errors were reduced by 43.9% and 31.6%, respectively. Compared with OCS orders, regimen defects, drug omissions, and incorrect data input errors were reduced by more than 70% for CAP orders. For all four efficiency measures, a statistically significant competence was identified when physicians ordered a chemotherapy prescription with CAP as compared with OCS (p<0.0001).

CONCLUSIONS

These results demonstrate the superiority of CAP over the existing CPOE system for ordering oncology medications with regard to safety and efficiency.

E-prescribing: a focused review and new approach to addressing safety in pharmacies and primary care

E-prescribing, the health information technology (HIT) that enables prescribers to electronically transmit prescriptions to community pharmacies, has been touted as a solution for improving patient safety and overall quality of care. However, the impact of HIT, such as e-prescribing on medication errors in acute care settings, has been widely studied and shows that if poorly designed or implemented, HIT can pose a risk to patient safety by introducing a source of medication errors. Unlike acute care settings, safety issues related to e-prescribing in primary care settings (where e-prescriptions are generated and transmitted) and pharmacies (where e-prescriptions are received) have not received as much attention in the literature. This paper provides a focused review of patient safety issues related to using e-prescribing systems in primary care and pharmacies. In addition, the paper proposes using human factors engineering concepts to study e-prescribing safety in pharmacies and primary care settings to identify safety problems and possible mechanisms for improvement.

Using human factors techniques to design text message reminders for childhood immunization

This study engaged parents to develop concise, informative, and comprehensible text messages for an immunization reminder system using Human Factors techniques. Fifty parents completed a structured interview including demographics, technology questions, willingness to receive texts from their child's doctor, and health literacy. Each participant was assigned to one user-centered design test: card sort, needs analysis, or comprehension. The majority of respondents were female (90%), White non-Hispanic (62%), and averaged 29 years (SD = 5). Nearly all (96%) had "adequate" health literacy. The card sort, an activity in which end users organize information into categories, identified six pieces of critical information. These were compiled into eight example texts, which were ranked in the needs assessment. The top two were assessed for comprehension, with 100% of participants able to understand the content and describe intention to act. Using user-centered design methods, the authors developed concise, informative text messages that parents indicated would prompt them to schedule their child's immunization appointment.

Integration of the enterprise electronic health record and anesthesia information management systems

Fewer than 5% of anesthesia departments use an electronic medical record (EMR) that is anesthesia specific. Many anesthesia information management systems (AIMS) have been developed with a focus only on the unique needs of anesthesia providers, without being fully integrated into other electronic health record components of the entire enterprise medical system. To understand why anesthesia providers should embrace health information technology (HIT) on a health system-wide basis, this article reviews recent HIT history and reviews HIT concepts. The author explores current developments in efforts to expand enterprise HIT, and the pros and cons of full enterprise integration with an AIMS.

Clinicians satisfaction with CPOE ease of use and effect on clinicians' workflow, efficiency and medication safety

OBJECTIVES

To study the satisfaction of end-users of a computerized physician order entry (CPOE) system concerning ease of use and the effect on users' workflow, efficiency, and medication safety and to seek users' opinions regarding required improvements of the system. Usability evaluation had shown that this system, which was in use for almost a decade, contained a number of severe usability problems. So another objective of the study was to determine whether there was a direct relation between user satisfaction and the results of a usability evaluation of the system.

METHODS

Two survey questionnaires were distributed to CPOE system users (physicians and nurses) working in inpatient departments of a university hospital. Questionnaires included items that were rated using a five point Likert scale. Multiple choice questions with space for free text additions also were used to collect qualitative data concerning the use of the CPOE system and the users' opinion concerning system requirements for improvement. Data were analyzed using descriptive statistics and by the use of Mann-Whitney U and Kruskal Wallis tests.

RESULTS

Two hundred seventeen physicians and 587 nurses were eligible to participate in this study (response rate 49% and 56% respectively). Physicians were satisfied with the CPOE ease of use (median 3.8, interquartile range [IQR] 3.3-4), and the effect on workflow (median 3.7, IQR 3.3-4), medication safety (median 3.75, IQR 3-4), and efficiency (median 4, IQR 3-4). Nurses also had a positive attitude towards CPOE ease of use (median 3.6, IQR 3-4), and its effect on workflow (median 3, IQR 3-3.6), medication safety (median 3, IQR 2.5-3.5), and efficiency (median 3.5, IQR 3-4). Users mainly indicated that the system needs: supplementary functionalities (e.g. alerts for allergies), improvement of current functionalities, integration with other hospital information systems and improvement of information presentation (e.g. a clear medication overview). Users did not use some current functionalities because of lack of awareness of the functionalities or having difficulty in using them.

CONCLUSIONS

Users of this CPOE system, which was used for almost a decade, were satisfied with the system's ease of use and its effect on efficiency, workflow and medication safety although the system showed many usability problems and lacked some functionalities. In this case study, therefore, there seems no direct relation between the results of the earlier performed usability evaluation and user satisfaction as determined in the current study.

The use of human factors methods to identify and mitigate safety issues in radiation therapy

BACKGROUND AND PURPOSE

New radiation therapy technologies can enhance the quality of treatment and reduce error. However, the treatment process has become more complex, and radiation dose is not always delivered as intended. Using human factors methods, a radiotherapy treatment delivery process was evaluated, and a redesign was undertaken to determine the effect on system safety.

MATERIAL AND METHODS

An ethnographic field study and workflow analysis was conducted to identify human factors issues of the treatment delivery process. To address specific issues, components of the user interface were redesigned through a user-centered approach. Sixteen radiation therapy students were then used to experimentally evaluate the redesigned system through a usability test to determine the effectiveness in mitigating use errors.

RESULTS

According to findings from the usability test, the redesigned system successfully reduced the error rates of two common errors (p<.04 and p<.01). It also improved the mean task completion time by 5.5% (p<.02) and achieved a higher level of user satisfaction.

CONCLUSIONS

These findings demonstrated the importance and benefits of applying human factors methods in the design of radiation therapy systems. Many other opportunities still exist to improve patient safety in this area using human factors methods.