Modeling patient-related workload in the emergency department using electronic health record data

INTRODUCTION

Understanding and managing clinician workload is important for clinician (nurses, physicians and advanced practice providers) occupational health as well as patient safety. Efforts have been made to develop strategies for managing clinician workload by improving patient assignment. The goal of the current study is to use electronic health record (EHR) data to predict the amount of work that individual patients contribute to clinician workload (patient-related workload).

METHODS

One month of EHR data was retrieved from an emergency department (ED). A list of workload indicators and five potential workload proxies were extracted from the data. Linear regression and four machine learning classification algorithms were utilized to model the relationship between the indicators and the proxies.

RESULTS

Linear regression proved that the indicators explained a substantial amount of variance of the proxies (four out of five proxies were modeled with R² > 0.80). Classification algorithms also showed success in classifying a patient as having high or low task demand based on data from early in the ED visit (e.g. 80 % accurate binary classification with data from the first hour).

CONCLUSION

The main contribution of this study is demonstrating the potential of using EHR data to predict patient-related workload automatically in the ED. The predicted workload can potentially help in managing clinician workload by supporting decisions around the assignment of new patients to providers. Future work should focus on identifying the relationship between workload proxies and actual workload, as well as improving prediction performance of regression and multi-class classification.

Lessons from the Glass Cockpit: Innovation in Alarm Systems to Support Cognitive Work

Nurses working in the hospital setting increasingly have become overburdened by managing alarms that, in many cases, provide low information value regarding patient health. The current trend, aided by disposable, wearable technologies, is to promote patient monitoring that does not require entering a patient's room. The development of telemetry alarms and middleware escalation devices adds to the continued growth of auditory, visual, and haptic alarms to the hospital environment but can fail to provide a more complete understanding of patient health. As we begin to innovate to both address alarm overload and improve patient management, perhaps using fundamentally different integration architectures, lessons from the aviation flight deck are worth considering. Commercial jet transport systems and their alarms have evolved slowly over many decades and have developed integration methods that account for operational context, provide multiple response protocol levels, and present a more integrated view of the airplane system state. We articulate three alarm system objectives: (1) supporting hazard management, (2) establishing context, and (3) supporting alarm prioritization. More generally, we present the case that alarm design in aviation can spur directions for innovation for telemetry monitoring systems in hospitals.

Design and Evaluation of an Integrated, Patient-Focused Electronic Health Record Display for Emergency Medicine

BACKGROUND

Hospital emergency departments (EDs) are dynamic environments, involving coordination and shared decision making by staff who care for multiple patients simultaneously. While computerized information systems have been widely adopted in such clinical environments, serious issues have been raised related to their usability and effectiveness. In particular, there is a need to support clinicians to communicate and maintain awareness of a patient's health status, and progress through the ED plan of care.

OBJECTIVE

This study used work-centered usability methods to evaluate an integrated patient-focused status display designed to support ED clinicians' communication and situation awareness regarding a patient's health status and progress through their ED plan of care. The display design was informed by previous studies we conducted examining the information and cognitive support requirements of ED providers and nurses.

METHODS

ED nurse and provider participants were presented various scenarios requiring patient-prioritization and care-planning tasks to be performed using the prototype display. Participants rated the display in terms of its cognitive support, usability, and usefulness. Participants' performance on the various tasks, and their feedback on the display design and utility, was analyzed.

RESULTS

Participants provided ratings for usability and usefulness for the display sections using a work-centered usability questionnaire-mean scores for nurses and providers were 7.56 and 6.6 (1 being lowest and 9 being highest), respectively. General usability scores, based on the System Usability Scale tool, were rated as acceptable or marginally acceptable. Similarly, participants also rated the display highly in terms of support for specific cognitive objectives.

CONCLUSION

A novel patient-focused status display for emergency medicine was evaluated via a simulation-based study in terms of work-centered usability and usefulness. Participants' subjective ratings of usability, usefulness, and support for cognitive objectives were encouraging. These findings, including participants' qualitative feedback, provided insights for improving the design of the display.

Cognitive engineering and health informatics: Applications and intersections

Cognitive engineering is an applied field with roots in both cognitive science and engineering that has been used to support design of information displays, decision support, human-automation interaction, and training in numerous high risk domains ranging from nuclear power plant control to transportation and defense systems. Cognitive engineering provides a set of structured, analytic methods for data collection and analysis that intersect with and complement methods of Cognitive Informatics. These methods support discovery of aspects of the work that make performance challenging, as well as the knowledge, skills, and strategies that experts use to meet those challenges. Importantly, cognitive engineering methods provide novel representations that highlight the inherent complexities of the work domain and traceable links between the results of cognitive analyses and actionable design requirements. This article provides an overview of relevant cognitive engineering methods, and illustrates how they have been applied to the design of health information technology (HIT) systems. Additionally, although cognitive engineering methods have been applied in the design of user-centered informatics systems, methods drawn from informatics are not typically incorporated into a cognitive engineering analysis. This article presents a discussion regarding ways in which data-rich methods can inform cognitive engineering.

Evaluating the Effectiveness of a Joint Cognitive System: Metrics, Techniques, and Frameworks

An implication of Cognitive Systems Engineering is that joint cognitive systems (JCS; also known as complex socio-technical systems) need to be evaluated for its effectiveness in performing the complex cognitive work requirements. This requires using measures that go well beyond “typical” performance metrics such as the number of subtask goals achieved per person per unit of time and the corresponding simple baseline comparisons or workload assessment metrics. This JCS perspective implies that the system must be designed and evaluated from the perspective of the shift in role of the human supervisor. This imposes new types of requirements on the human operator. Previous research in CSE and our own experience has lead us to identify a set of generic JCS support requirements that apply to cognitive work by any cognitive agent or any set of cognitive agents, including teams of people and machine agents. Metrics will have to reflect such phenomena as “teamwork” or “resilience” of a JCS. This places new burdens on evaluation techniques and frameworks, since metrics should be generated from a principled approach and based on fundamental principles of interest to the designers of the JCS. An implication of the JCS perspective is that complex and cognitive systems need to be evaluated for usability, usefulness, and understandability; each of which goes well beyond raw performance. However, conceptually-grounded evaluation frameworks, corresponding operational techniques, and corresponding measures for these are limited. Therefore, in order to advance the state of the field, we have gathered a set of researchers and practitioners to present recent evaluation work to stimulate discussion.

Using a Benchmark-Referenced Approach for Validating a Power Plant Control Room: Results of the Baseline Study

When performing a validation study of a new system, a question arises as to what constitutes acceptable performance. One approach to setting acceptance criteria is to compare performance obtained with the new system to performance with an existing system that serves as a ‘benchmark‘. This paper describes an integrated system validation for a modernized power plant control room that takes the benchmark-referenced approach. In preparation for the planned integrated system validation, a baseline study of crew performance with the existing control room technology was performed using a high fidelity simulator. This paper reports the results of the baseline study and discusses their implications for validation studies of complex systems more generally. Most notably, the results establish an empirical link between process measures of performance (e.g., target detection, diagnosis, workload, teamwork, and situation awareness) and operationally significant outcome measures of performance, which reinforces the value of including process measures in validation studies.

Human in the Loop Evaluation of a Mixed-Initiative System for Planning and Control of Multiple UAV Teams

This paper describes a ‘Human in the Loop' evaluation of an early prototype mixed-initiative control system that generated plans for single operator supervision of multiple unmanned aerial vehicles (UAV) missions. The objectives of the evaluation were to assess to what extent human operators were able to understand the plans generated by the automated controllers as well as to point to additional support requirements to drive further development of mixed-initiative planning systems. Six former fighter pilots served as test participants. Multiple convergent measures were utilized to evaluate the ability of users to understand and evaluate the plans generated by the automated controller. While test participants were able to understand the plans, the results provided compelling evidence for the need to communicate more effectively the rationale behind plan elements proposed by the automated controller and to provide ‘levers' to enable the user to modify the plan. These results point to important challenges for design of ‘mixed-initiative’ controllers to enable the human and automated controller to function as effective ‘partners’.

Assessment of Display Attributes for Displaying Metainformation on Maps

In many domains, operators need to understand and act on large volumes of information from a variety of sources. Operators are particularly challenged by the need to reason about the qualifiers of that information. These qualifiers, or “meta-information”, include characteristics such as the uncertainty associated with data, the age of the data, and the source of the data. Often, these critical data qualifiers are not presented, or are not incorporated into the primary information displays used by operators. In this research, we conducted a controlled experiment to investigate the utility of four common color display attributes (hue, saturation, brightness, and transparency) for displaying meta-information under different map background, task, and meta-information-type conditions. Results indicated that participants could rank and rate display elements which varied based on saturation, transparency and brightness similarly to expected ranks and ratings. Background effects were limited; but task type and framing effects indicated that the “natural” direction for ranking may be context-dependent.

Supporting Situation Awareness of Individuals and Teams Using Group View Displays

This paper describes a project to design and test a large wall-mounted group view display intended to support situation awareness of individuals and multi-person teams in an advanced power plant control room. The design process began with an analysis of the requirements for situation awareness of individuals and teams to establish the design basis. A function-based cognitive task analysis was then used to define the plant state information to be included in the group view display. A phased test program was then initiated to establish the adequacy of the design concept. The first study employed sequences of static displays representing “snapshots” of evolving normal and emergency events. The second study tested dynamic group view displays (three 100 inch wall mounted displays) driven by a high-fidelity plant simulation. The study compared operator performance with the function-based group view display to operator performance using a more traditional physical mimic overview display. Dependent measures included (1) objective measures of performance, (2) operator ratings of the displays, and (3) workload measures. The measures provided statistically significant, converging evidence of the superiority of the function-based group view display. The implication of the results for design of group view displays to support individuals and teams are discussed.

A Framework for Work-Centered Product Evaluation

We describe a comprehensive work-centered evaluation framework for assessing the value of new technology intended to support human performance. A key feature of the framework is that it spans three types of evaluation: (1) usability — How easy is the aid to use? (2) usefulness — How flexible and effective is it in aiding work? and (3) impact — How (much) does it contribute value within the work organization? We argue that all three of these dimensions are important for advancing the product development and facilitating rapid transition of the product to the customer for deployment. A recent evaluation of a work-centered support system prototype is used to illustrate the multi-level work-centered evaluation approach, the types of insights that can be drawn from it, and a way to employ it in a cost effective manner.

A Framework for Integrating Cognitive Task Analysis into the System Development Process

This paper describes a process that orchestrates different types of specific CTA techniques to provide design relevant CTA results and integrates CTA results into the software development process. Two fundamental premises underlie the approach. First, CTA is more than the application of any single CTA technique. Instead, developing a meaningful understanding of a field of practice relies on multiple converging techniques in a bootstrapping process. The important issue from a CTA perspective is to evolve a model of the interconnections between the demands of the domain, the strategies and knowledge of practitioners, the cooperative interactions across human and machine agents, and how artifacts shape these strategies and coordinative activities across a series of different specific techniques. Second, since CTA is a means to support the design of computer-based artifacts that enhance human and team performance, CTA must be integrated into the software and system development process. Thus, the vision of CTA as an initial, self-contained technique that is handed-off to system designers is reconceived as an incremental process of uncovering the cognitive demands imposed on the operator(s) by the complexities and constraints of the domain.

Understanding how Train Dispatchers Manage and Control Trains: A Cognitive Task Analysis of a Distributed Team Planning Task

A Cognitive Task Analysis was conducted to examine how experienced train dispatchers manage track use. The results reveal the cognitive complexities faced by dispatchers and the cognitive and collaborative strategies developed in response to those demands, including strategies to support anticipation and planning, and proactive strategies to exploit windows of opportunity to satisfy the multiple demands on track use. In many cases these strategies depend on communication and cooperation among individuals distributed across time and space (i.e., multiple dispatchers, engineers, maintenance of way personnel). The ability to “listen in” on communications directed at others that have a bearing on achievement of your own goals and to recognize when information in your possession is of relevance to others, are important contributors to safe and efficient track use. The results reinforce findings from other domains (e.g., space shuttle mission control, air traffic control) regarding the role of a shared communication channel in supporting anticipation and contingency planning. Implication for the introduction of advanced “data-link” communication technologies, as well as for dispatcher training are discussed.

Exploring the Impact of Advanced Alarms, Displays, and Computerized Procedures on Teams

New support technologies embody hypotheses about what constitutes effective support, and how technological change is expected to shape cognition and collaboration. Introduction of these technologies into a field of practice provides an opportunity to test and revise our understanding. An illustrative case is presented that examined the impact of introducing advanced human-system interfaces (HSIs), including a computer-based procedure system, an advanced alarm system, and a graphic display system, into a conventional power plant control room. Crews were observed using the HSIs to handle simulated plant disturbances in a full-scope simulator. The HSIs generally provided positive support and reduced workload as intended. However, unanticipated impacts were also identified. Of particular note was the effect on team structure and communication. The computerized systems altered the distribution of information across crew members and communication patterns, affecting the situation awareness of individual crew members and the crew as a whole. While in a traditional control room, the open environment, and explicit communication requirements foster shared situation awareness, maintaining shared situation awareness in the computerized environment required active work on the part of crew members. The results point to requirements for effectivecollaboration and help define future research direction.

Handoffs During Nursing Shift Changes in Acute Care

Handoffs during the nursing shift change were directly observed on two acute care wards each of a private and public hospital, for a total of 236 patient updates by 49 nurses during 14 shift changes. Data from the three wards which conducted audio-taped updates were transcribed. The transcriptions and field notes were analyzed for the existence and frequency of 21 strategies used in high reliability organizations. In addition, we iteratively categorized the interruptions, questions, and statements made during the updates. Finally, we iteratively categorized stances towards decisions communicated during the updates. Implications of the findings are discussed.

Fostering Shared Situation Awareness and on-Track Safety Across Distributed Teams in Railroad Operations

Cooperative strategies of individuals within a distributed organizational structure can contribute to increased efficiency of operations and safety. We describe selected results of a cognitive task analysis that examined the cognitive and collaborative demands and activities of railroad roadway workers. The findings highlight the informal cooperative strategies that railroad workers have developed across distributed teams consisting of roadway workers, train crews, and railroad dispatchers that foster shared situation awareness and enhance on-track safety. We discuss design implications for leveraging new digital technologies and location finding systems to more effectively support these informal strategies to improve efficiency and enhance on-track safety are discussed.

Modeling Operator Performance in Emergencies

Human performance frequently has been shown to be a substantial contributor to the overall reliability of complex dynamic systems. As a consequence, there has been a great emphasis in human factors to develop models of human cognition relevant to these situations as a tool to better predict human performance and to help in the design of improved person-machine systems. This paper describes the Cognitive Environment Simulation strategy towards human performance modeling and summarizes one comparison between system behavior and operator behavior in nuclear power emergency operations.

An Expert “Critiquer” for Propulsion Gear Design: A Case Study in Intelligent Decision Support

This paper describes an “intelligent” designer's aid that was developed to support the design of marine propulsion gears. Key elements of the system include: conversion of gear design formulas from a procedural to a declarative form to facilitate inspection; a direct-manipulation interface; and encoding of “expert” design constraint knowledge. The case study demonstrates that delivering “expert knowledge” is often only a small element of an “intelligent” support system, and provides a concrete illustration of the importance of a cognitive task analysis in defining the elements of an effective support system. This design solution should have applicability to other engineering design tasks.

Navigating through Large Display Networks in Dynamic Control Applications

There is an increasing trend to use computer display systems as the primary “window” by which users see and interact with complex dynamic processes (e.g., air traffic control; computerized control rooms for process control). These kinds of applications offer special challenges to the design of computer based display systems. In particular, the large scope of these applications necessitates large display structures involving thousands of displays. Further, the dynamic nature of the tasks mean that users need to be able to move rapidly through the display structure to keep pace with temporally evolving situations and to be able to respond to new events as they occur. As a result, special display navigation challenges arise in computer based display systems for monitoring and controlling dynamic processes.

Shared Situation Awareness as a Contributor to High Reliability Performance in Railroad Operations

Cooperative strategies of individuals within a distributed organization can contribute to increased efficiency of operations and safety. We examine these processes in the context of a particular work domain: railroad operations. Analyses revealed a variety of informal cooperative strategies that railroad workers have developed that span across multiple railroad crafts including roadway workers, train crews, and railroad dispatchers. These informal, proactive communications foster shared situation awareness across the distributed organization, facilitate work, and contribute to the overall efficiency, safety, and resilience to error of railroad operations. We discuss design implications for leveraging new digital technologies and location-finding systems to more effectively support these informal strategies, enhance shared situation awareness, and promote high reliability performance.

Discussion Panel

Cognitive Task Analysis (CTA) has become part of the standard tool set of cognitive engineering. CTAs are routinely used to understand the cognitive and collaborative demands that contribute to performance problems, the basis of expertise, as well as the opportunities to improve performance through new forms of training, user interfaces, or decision aids. While the need to conduct CTAs has become well established, there is little in the way of available guidance with respect to ‘best practice’ for how to conduct a CTA or how to evaluate the quality of a CTA that has been conducted by others. This is an important gap as the range of consumers of CTAs is expanding to include program managers and regulators who may need to make decisions based on CTA findings. This panel brings together some of the leaders in development and application of CTA methods to address the question: Given the variety of methods available, and the lack of rigid guidance on how to perform a CTA, how does one judge the quality of a CTA?” The goal of the panel is to explore points of consensus with respect to ‘best practice’ in conducting and evaluating a CTA, in spite of differences in particular CTA methods, as well as to draw insights from unique and provocative perspectives.

Deconstructing intraoperative communication failures

BACKGROUND

Communication failure is a common contributor to adverse events. We sought to characterize communication failures during complex operations.

METHODS

We video recorded and transcribed six complex operations, representing 22 h of patient care. For each communication event, we determined the participants and the content discussed. Failures were classified into four types: audience (key individuals missing), purpose (issue nonresolution), content (insufficient/inaccurate information), and/or occasion (futile timing). We added a systems category to reflect communication occurring at the organizational level. The impact of each identified failure was described.

RESULTS

We observed communication failures in every case (mean 29, median 28, range 13-48), at a rate of one every 8 min. Cross-disciplinary exchanges resulted in failure nearly twice as often as intradisciplinary ones. Discussions about or mandated by hospital policy (20%), personnel (18%), or other patient care (17%) were most error prone. Audience and purpose each accounted for >40% of failures. A substantial proportion (26%) reflected flawed systems for communication, particularly those for disseminating policy (29% of system failures), coordinating personnel (27%), and conveying the procedure planned (27%) or the equipment needed (24%). In 81% of failures, inefficiency (extraneous discussion and/or work) resulted. Resource waste (19%) and work-arounds (13%) also were frequently seen.

CONCLUSIONS

During complex operations, communication failures occur frequently and lead to inefficiency. Prevention may be achieved by improving synchronous, cross-disciplinary communication. The rate of failure during discussions about/mandated by policy highlights the need for carefully designed standardized interventions. System-level support for asynchronous perioperative communication may streamline operating room coordination and preparation efforts.

War stories: a qualitative analysis of narrative teaching strategies in the operating room

BACKGROUND

"War stories" are commonplace in surgical education, yet little is known about their purpose, construct, or use in the education of trainees.

METHODS

Ten complex operations were videotaped and audiotaped. Narrative stories were analyzed using grounded theory to identify emergent themes in both the types of stories being told and the teaching objectives they illustrated.

RESULTS

Twenty-four stories were identified in 9 of the 10 cases (mean, 2.4/case). They were brief (mean, 58 seconds), illustrative of multiple teaching points (mean, 1.5/story), and appeared throughout the operations. Anchored in personal experience, these stories taught both clinical (eg, operative technique, decision making, error identification) and programmatic (eg, resource management, professionalism) topics.

CONCLUSIONS

Narrative stories are used frequently and intuitively by physicians to emphasize a variety of intraoperative teaching points. They socialize trainees in the culture of surgery and may represent an underrecognized approach to teaching the core competencies. More understanding is needed to maximize their potential.

Person-in-the-Loop Testing of a Digital Power Plant Control Room

There is renewed interest in building new commercial nuclear power plants. Unlike existing plants that have traditional control rooms with large control boards, new plants will have compact digital control rooms. The transition to digital control rooms introduce opportunities for enhanced support (e.g., integrated displays; improved alarm systems; computerized procedures) as well as potential new challenges (e.g., shift from open to private workspaces; changes in workload distribution resulting from reduced control room crew size). This paper describes two simulator tests that were conducted as part of a person-in-the-loop test program to support development and validation of a control room for the US-APWR evolutionary pressurized water reactor plant. While the results are presented in the context of evaluation of a particular digital control room design, they have applicability to design of compact digital control rooms more generally, and point to areas where more research is needed.

Uncovering the requirements of cognitive work

OBJECTIVE

In this article, the author provides an overview of cognitive analysis methods and how they can be used to inform system analysis and design.

BACKGROUND

Human factors has seen a shift toward modeling and support of cognitively intensive work (e.g., military command and control, medical planning and decision making, supervisory control of automated systems). Cognitive task analysis and cognitive work analysis methods extend traditional task analysis techniques to uncover the knowledge and thought processes that underlie performance in cognitively complex settings.

METHODS

The author reviews the multidisciplinary roots of cognitive analysis and the variety of cognitive task analysis and cognitive work analysis methods that have emerged.

RESULTS

Cognitive analysis methods have been used successfully to guide system design, as well as development of function allocation, team structure, and training, so as to enhance performance and reduce the potential for error.

CONCLUSIONS

A comprehensive characterization of cognitive work requires two mutually informing analyses: (a) examination of domain characteristics and constraints that define cognitive requirements and challenges and (b) examination of practitioner knowledge and strategies that underlie both expert and error-vulnerable performance. A variety of specific methods can be adapted to achieve these aims within the pragmatic constraints of particular projects.

APPLICATION

Cognitive analysis methods can be used effectively to anticipate cognitive performance problems and specify ways to improve individual and team cognitive performance (be it through new forms of training, user interfaces, or decision aids).

Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses

OBJECTIVE

Computerized clinical reminders (CRs) were designed to reduce clinicians' reliance on their memory and to present evidence-based guidelines at point of care. However, the literature indicates that CR adoption and effectiveness has been variable. We examined the impact of four design modifications to CR software on learnability, efficiency, usability, and workload for intake nursing personnel in an outpatient clinic setting. These modifications were included in a redesign primarily to address barriers to effective CR use identified during a previous field study.

DESIGN

In a simulation experiment, 16 nurses used prototypes of the current and redesigned system in a within-subject comparison for five simulated patient encounters. Prior to the experimental session, participants completed an exploration session, where "learnability" of the current and redesigned systems was assessed.

MEASUREMENTS

Time, performance, and survey data were analyzed in conjunction with semi-structured debrief interview data.

RESULTS

The redesign was found to significantly increase learnability for first-time users as measured by time to complete the first CR, efficiency as measured by task completion time for two of five patient scenarios, usability as determined by all three groupings of questions taken from a commonly used survey instrument, and two of six workload subscales of the NASA Task Load Index (TLX) survey: mental workload and frustration.

CONCLUSION

Modest design modifications to existing CR software positively impacted variables that likely would increase the willingness for first-time nursing personnel to adopt and consistently use CRs.

Making the operating room of the future safer

There is an increasing demand for interventions to improve patient safety, but there is limited data to guide such reform. In particular, because much of the existing research is outcome-driven, we have a limited understanding of the factors and process variations that influence safety in the operating room. In this article, we start with an overview of safety terminology, suggesting a model that emphasizes "safety" rather than "error" and that can encompass the spectrum of events occurring in the operating room. Next, we provide an introduction to techniques that can be used to understand safety at the point of care and we review the data that exists relating such studies to improved outcomes. Future work in this area will need to prospectively study the processes and factors that impact patient safety and vulnerability in the operating room.

Making the Operating Room of the Future Safer

There is an increasing demand for interventions to improve patient safety, but there is limited data to guide such reform. In particular, because much of the existing research is outcome-driven, we have a limited understanding of the factors and process variations that influence safety in the operating room. In this article, we start with an overview of safety terminology, suggesting a model that emphasizes “safety” rather than “error” and that can encompass the spectrum of events occurring in the operating room. Next, we provide an introduction to techniques that can be used to understand safety at the point of care and we review the data that exists relating such studies to improved outcomes. Future work in this area will need to prospectively study the processes and factors that impact patient safety and vulnerability in the operating room.

A prospective study of patient safety in the operating room

BACKGROUND

To better understand the operating room as a system and to identify system features that influence patient safety, we performed an analysis of operating room patient care using a prospective observational technique.

METHODS

A multidisciplinary team comprised of human factors experts and surgeons conducted prospective observations of 10 complex general surgery cases in an academic hospital. Minute-to-minute observations were recorded in the field, and later coded and analyzed. A qualitative analysis first identified major system features that influenced team performance and patient safety. A quantitative analysis of factors related to these systems features followed. In addition, safety-compromising events were identified and analyzed for contributing and compensatory factors.

RESULTS

Problems in communication and information flow, and workload and competing tasks were found to have measurable negative impact on team performance and patient safety in all 10 cases. In particular, the counting protocol was found to significantly compromise case progression and patient safety. We identified 11 events that potentially compromised patient safety, allowing us to identify recurring factors that contributed to or mitigated the overall effect on the patient's outcome.

CONCLUSIONS

This study demonstrates the role of prospective observational methods in exposing critical system features that influence patient safety and that can be the targets for patient safety initiatives. Communication breakdown and information loss, as well as increased workload and competing tasks, pose the greatest threats to patient safety in the operating room.

Handoff strategies in settings with high consequences for failure: lessons for health care operations

OBJECTIVE

To describe strategies employed during handoffs in four settings with high consequences for failure.

DESIGN

ANALYSIS

of observational data for evidence of use of 21 handoff strategies.

SETTING

NASA Johnson Space Center in Texas, nuclear power generation plants in Canada, a railroad dispatch center in the United States, and an ambulance dispatch center in Toronto.

MAIN MEASURE

Evidence of 21 handoff strategies from observations and interviews.

RESULTS

Nineteen of 21 strategies were used in at least one domain, on at least an 'as needed' basis.

CONCLUSIONS

An understanding of how handoffs are conducted in settings with high consequences for failure can jumpstart endeavors to modify handoffs to improve patient safety.

Lessons from a comparison of work domain models: representational choices and their implications

As methods in cognitive work analysis become more widely applied, questions regarding the impact of modeling choices and similarities in modeling efforts across projects and domains are increasingly relevant. However, no explicit comparison of models of similar systems has been reported. This paper compares independently developed work domain analysis (WDA) models of two command and control environments. Similarities in model content and the types of nodes included provide evidence that WDA techniques can capture fundamental elements regarding purposes and constraints. These points of agreement provide a common starting point for developing work domain representations of military command and control systems. The comparison also revealed differences between the models. Although differences in content reflected differences in scope of coverage and level of detail, other differences corresponded to more fundamental choices in modeling approach. These included the treatment of sensors, level of integration in the model, and representation of particular abstract constraints. Examination of these more fundamental differences pointed to important degrees of freedom in how to represent a WDA and clarified the implications of these modeling choices for guiding design. Actual or potential applications of this research include aiding analysts in making work domain modeling choices as well as producing work domain models of command and control environments.

Coronary perforation during percutaneous coronary intervention in the era of abciximab platelet glycoprotein IIb/IIIa blockade: an algorithm for percutaneous management

Coronary perforation is an uncommon but potentially life-threatening complication of percutaneous coronary intervention. The use of both atheroablative technologies for coronary intervention and adjunctive platelet glycoprotein blockade pharmacology may increase the incidence of or risk for life-threatening bleeding complications following the occurrence of coronary artery perforation. The interventional database for 6,214 percutaneous coronary interventions performed between January 1995 and June 1999 was analyzed. Hospital charts and cine angiograms for all patients identified in the database as having had coronary perforation were reviewed. Coronary perforation complicated 0.58% of all procedures and was more commonly observed in patients with a history of congestive heart failure and following use of atheroablative interventional technologies (2.8%). There was no association of abciximab therapy with either the incidence of or classification for coronary perforation. Adverse clinical outcomes (death, emergency surgical exploration) were related to the angiographic classification of perforation and were more frequently observed in patients who experienced a class 3 coronary perforation. These data suggest that specific clinical and procedural demographic factors are associated with the occurrence and severity of angiographic coronary perforation. An angiographic perforation class-specific algorithm for treatment of coronary perforation is proposed.

Abciximab provides cost-effective survival advantage in high-volume interventional practice

BACKGROUND

Placebo-controlled randomized trials of platelet glycoprotein (GP) IIb/IIIa blockade during percutaneous coronary intervention have demonstrated efficacy of these agents for reducing the risk of periprocedural ischemic events. However, cost-effectiveness of this adjunctive pharmacotherapy has been scrutinized. Extrapolation of cost-efficacy observations from clinical trials to "real world" interventional practice is problematic.

METHODS

Consecutive percutaneous coronary interventions (n = 1472) performed by Ohio Heart Health Center operators at The Christ Hospital, Cincinnati, Ohio, in 1997 were analyzed for procedural and long-term (6-month) outcomes and charges. Observations on cost and efficacy (survival) were adjusted for nonrandomized abciximab allocation by means of "propensity scoring" methods.

RESULTS

Abciximab therapy was associated with a survival advantage to 6 months after percutaneous coronary intervention. The average reduction in mortality rate at 6 months was 3.4% (unadjusted) and 4.9% when adjusted for nonrandomization. The average charge increment to 6 months was $1512 (unadjusted) and $950 when adjusted for nonrandomization. Patients deriving the greatest reduction in mortality rates also had a reduction in total cardiovascular charges to 6 months. Distinguishing demographics of this population included multivessel coronary intervention, coronary stent deployment, intervention within 1 week of myocardial infarction, and lower left ventricular ejection fraction. The average cost per life-year gained in this study was $2875 for all patients (unadjusted) and $1243 when adjusted for nonrandomization.

CONCLUSIONS

Abciximab provides a cost-effective survival advantage in high-volume interventional practice that compares favorably with currently accepted standards. Clinical and procedural demographics associated with increased cost-effectiveness included multivessel coronary intervention, stent deployment, recent (<1 week) myocardial infarction, and impaired left ventricular function.

High platelet count in platelet-rich plasma reduces measured platelet inhibition by abciximab but not tirofiban nor eptifibatide glycoprotein IIb/IIIa receptor antagonists

We evaluated the differential effect of platelet count in platelet-rich plasma (PRP) on the level of ex vivo inhibition of platelet aggregation provided by abciximab, eptifibatide, and tirofiban as part of a randomized, comparative trial of these agents on platelet function in patients with unstable angina pectoris undergoing percutaneous coronary intervention. Platelet count <350 K/microL in PRP reduced measured platelet inhibition by abciximab, but not eptifibatide nor tirofiban. This observation suggests the need for standardized, uniform platelet counts in PRP during future comparisons of the degree of platelet inhibition by these agents.

There is more to monitoring a nuclear power plant than meets the eye

A fundamental challenge in studying cognitive systems in context is how to move from the specific work setting studied to a more general understanding of distributed cognitive work and how to support it. We present a series of cognitive field studies that illustrate one response to this challenge. Our focus was on how nuclear power plant (NPP) operators monitor plant state during normal operating conditions. We studied operators at two NPPs with different control room interfaces. We identified strong consistencies with respect to factors that made monitoring difficult and the strategies that operators have developed to facilitate monitoring. We found that what makes monitoring difficult is not the need to identify subtle abnormal indications against a quiescent background, but rather the need to identify and pursue relevant findings against a noisy background. Operators devised proactive strategies to make important information more salient or reduce meaningless change, create new information, and off-load some cognitive processing onto the interface. These findings emphasize the active problem-solving nature of monitoring, and highlight the use of strategies for knowledge-driven monitoring and the proactive adaptation of the interface to support monitoring. Potential applications of this research include control room design for process control and alarm systems and user interfaces for complex systems.

Time course, magnitude, and consistency of platelet inhibition by abciximab, tirofiban, or eptifibatide in patients with unstable angina pectoris undergoing percutaneous coronary intervention

Adjunctive platelet glycoprotein IIb/IIIa blockade during percutaneous coronary intervention (PCI) reduces platelet-mediated adverse ischemic outcomes. Although abciximab, eptifibatide, and tirofiban have received U.S. Food and Drug Administration approval for use, these agents differ in their pharmacodynamic profiles. Each of these agents has been compared in randomized trials with placebo for patients undergoing PCI, but no randomized comparative studies of these agents have been performed. We compared ex vivo platelet function by both standard light transmission aggregometry and rapid platelet function assay during and after administration of abciximab, eptifibatide, or tirofiban in approved dose regimens on a randomized basis at the time of PCI in patients with unstable angina pectoris. A reduced intensity of platelet inhibition by light transmission aggregometry was observed for tirofiban compared with either eptifibatide or abciximab. In addition, the 30-minute bolus strategy used for tirofiban was associated with delayed onset of maximal platelet inhibition relative to the initiation of bolus infusion. Whether the trends in platelet function observed in this study will be translated into differences in clinical outcomes awaits definition by larger scale randomized clinical trials comparing these platelet glycoprotein IIb/IIIa inhibitors.

Efficacy of abciximab induced platelet blockade using a rapid point of care assay

Anciximab provides potent, but variable degrees of platelet inhibition both during the duration of intravenous administration and at 12 hours following therapy. Platelet function was assessed using the PC-RPFA system in 78 patients scheduled for percutaneous coronary revascularization who were administered the standard abciximab weight-adjusted bolus and 12-hour infusion. The PC-RPFA system is a cartridge-based, semiautomated point-of-care whole-blood assay that incorporates fibrinogen-coated polystyrene beads, buffers, and a modified thrombin receptor activating peptide (Isotrap) in lyophilized form. The instrument detects the agglutination rate between the stimulated platelets and the fibrinogen-coated beads, and provides a quantitative digital display in less than 2 minutes. No differences in the level of platelet inhibition were observed in these abciximab-treated patients by diabetic status, gender, smoking, diagnosis (unstable angina, chronic stable angina, recent myocardial infarction), or abciximab treatment status (first time vs. retreatment). Nocorrelation of the PC-RPFA rate of platelet aggregation with clinical demographic factors was observed, with the exception of baseline hematocrit (r2 = 0.4556). The relationship between the PC-RPFA rate of aggregation and hematocrit reflects light absorbance by erythrocytes and is specific to the PC-RPFA system. The absolute rate of platelet aggregation (slope) reported by the PC-RPFA is correlated with percent aggregation, thus making it potentially possible to predict the level of aggregation without reference to a baseline (pretreatment) measure of platelet function. This correlation was closest for patients having <40% baseline aggregation (r2 = 0.55). Thus, PC-RPFA provides a rapid point-of-care assessment of platelet function that could allow for adjustment of abciximab dosing to achieve targeted levels of platelet inhibition. The utility of this device to optimize therapy with platelet glycoprotein IIb/IIIa inhibitors is currently being evaluated.