Evaluation of user acceptance of a clinical expert system

Real-Time Emergency Department Electronic Notifications Regarding High-Risk Patients: A Systematic Review

Background: To systematically review evidence on the feasibility and efficacy of real-time electronic notifications about patients at high risk of emergency department (ED) recidivism. Methods: Eight electronic databases were searched for empirical studies of real-time ED-based electronic tools, identifying adult patients at high risk of frequent utilization. Study selection and data extraction were performed independently by two reviewers. Qualitative data synthesis and assessment of strength of evidence were conducted through consensus discussion. Results: Of 2,256 records found through the search, 210 were duplicates, 2,004 were excluded based on abstract review, and 31 were excluded after full text review. The final sample consisted of 10 studies described in 11 articles describing the effect of real-time ED-based electronic notifications for high-risk patients. Three were randomized controlled trials (RCTs). All notifications were based on prespecified markers of risk. Seven studies integrated complex care plans into the electronic health record. Effect on ED use and length of stay (LOS) was mixed: nine studies reported decreased ED use, although results were statistically significant in only three studies; for LOS, one study reported a statistically significant reduction. Impact on cost and financial metrics was promising, with three (of three studies reporting this metric) showing improved organizational financial metrics. Three RCTs reported a reduction in opioid prescriptions. Conclusions: Real-time electronic notifications of ED providers regarding patients at high risk of ED recidivism are feasible. They may help reduce resource utilization and costs. Large knowledge gaps remain regarding patient- and provider-centered outcomes.

Assessing the Safety of Custom Web-Based Clinical Decision Support Systems in Electronic Health Records: A Case Study

BACKGROUND

With the widespread adoption of vendor-supplied electronic health record (EHR) systems, clinical decision support (CDS) customization efforts beyond those anticipated by the vendor may require the use of technologies external to the EHR such as web services. Pursuing such customizations, however, is not without risk. Validating the expected behavior of a customized CDS system in the high-volume, complex environment of the live EHR is a challenging problem.

OBJECTIVE

This article identifies technology failures that impacted clinical care related to web service-based advanced custom CDS systems embedded in the complex sociotechnical context of a production EHR.

METHODS

In an academic health system's primary care network, we performed an inventory of incidents between January 1, 2008 and December 31, 2016 related to a customized CDS system and performed a targeted review of changes in the CDS source code. Additional feedback on the root cause of individual incidents was obtained through interviews with members of the CDS project teams.

RESULTS

We identified five CDS malfunctions that impaired clinical workflow. The mechanisms for these failures are mapped to four characteristics of well-behaved applications: (1) system integrity; (2) data integrity; (3) reliability; and (4) scalability. Over the 9-year period, two malfunctions of the customized CDS significantly impaired clinical workflow for a total of 5 hours. Lesser impacts-loss of individual features with straightforward workarounds-arose from three malfunctions, which affected users on 53 days.

DISCUSSION

Advanced customization of EHRs for the purpose of CDS can present significant risks to clinical workflow.

CONCLUSION

This case study highlights that advanced customization of CDS within a commercial EHR may support care for complex patient populations, but ongoing monitoring and support is required to ensure its safe use.

Clinical Information Systems - From Yesterday to Tomorrow

OBJECTIVES

To review the history of clinical information systems over the past twenty-five years and project anticipated changes to those systems over the next twenty-five years.

METHODS

Over 250 Medline references about clinical information systems, quality of patient care, and patient safety were reviewed. Books, Web resources, and the author's personal experience with developing the HELP system were also used.

RESULTS

There have been dramatic improvements in the use and acceptance of clinical computing systems and Electronic Health Records (EHRs), especially in the United States. Although there are still challenges with the implementation of such systems, the rate of progress has been remarkable. Over the next twenty-five years, there will remain many important opportunities and challenges. These opportunities include understanding complex clinical computing issues that must be studied, understood and optimized. Dramatic improvements in quality of care and patient safety must be anticipated as a result of the use of clinical information systems. These improvements will result from a closer involvement of clinical informaticians in the optimization of patient care processes.

CONCLUSIONS

Clinical information systems and computerized clinical decision support have made contributions to medicine in the past. Therefore, by using better medical knowledge, optimized clinical information systems, and computerized clinical decision, we will enable dramatic improvements in both the quality and safety of patient care in the next twenty-five years.

Selecting anti-epileptic drugs: a pediatric epileptologist's view, a computer's view

OBJECTIVE

To identify which clinical characteristics are important to include in clinical decision support systems developed for Antiepileptic Drug (AEDs) selection.

METHODS

Twenty-three epileptologists from the Childhood Absence Epilepsy network completed a survey related to AED selection. Using cluster analysis their responses where classified into subject matter groups and weighted for importance.

RESULTS

Five distinct subject matter groups were identified and their relative weighting for importance were determined: disease characteristics (weight 4.8 ± 0.049), drug toxicities (3.82 ± 0.098), medical history (3.12 ± 0.102), systemic characteristics (2.57 ± 0.048) and genetic characteristics (1.08 ± 0.046).

CONCLUSION

Research about prescribing patterns exists but research on how such data can be used to train advanced technology is novel. As machine learning algorithms becomes more and more prevalent in clinical decisions support systems, developing methods for determining which data should be part of those algorithms is equally important.

Clinicians satisfaction with CPOE ease of use and effect on clinicians' workflow, efficiency and medication safety

OBJECTIVES

To study the satisfaction of end-users of a computerized physician order entry (CPOE) system concerning ease of use and the effect on users' workflow, efficiency, and medication safety and to seek users' opinions regarding required improvements of the system. Usability evaluation had shown that this system, which was in use for almost a decade, contained a number of severe usability problems. So another objective of the study was to determine whether there was a direct relation between user satisfaction and the results of a usability evaluation of the system.

METHODS

Two survey questionnaires were distributed to CPOE system users (physicians and nurses) working in inpatient departments of a university hospital. Questionnaires included items that were rated using a five point Likert scale. Multiple choice questions with space for free text additions also were used to collect qualitative data concerning the use of the CPOE system and the users' opinion concerning system requirements for improvement. Data were analyzed using descriptive statistics and by the use of Mann-Whitney U and Kruskal Wallis tests.

RESULTS

Two hundred seventeen physicians and 587 nurses were eligible to participate in this study (response rate 49% and 56% respectively). Physicians were satisfied with the CPOE ease of use (median 3.8, interquartile range [IQR] 3.3-4), and the effect on workflow (median 3.7, IQR 3.3-4), medication safety (median 3.75, IQR 3-4), and efficiency (median 4, IQR 3-4). Nurses also had a positive attitude towards CPOE ease of use (median 3.6, IQR 3-4), and its effect on workflow (median 3, IQR 3-3.6), medication safety (median 3, IQR 2.5-3.5), and efficiency (median 3.5, IQR 3-4). Users mainly indicated that the system needs: supplementary functionalities (e.g. alerts for allergies), improvement of current functionalities, integration with other hospital information systems and improvement of information presentation (e.g. a clear medication overview). Users did not use some current functionalities because of lack of awareness of the functionalities or having difficulty in using them.

CONCLUSIONS

Users of this CPOE system, which was used for almost a decade, were satisfied with the system's ease of use and its effect on efficiency, workflow and medication safety although the system showed many usability problems and lacked some functionalities. In this case study, therefore, there seems no direct relation between the results of the earlier performed usability evaluation and user satisfaction as determined in the current study.

The impact of electronic medical record systems on outpatient workflows: a longitudinal evaluation of its workflow effects

CONTEXT

The promise of the electronic medical record (EMR) lies in its ability to reduce the costs of health care delivery and improve the overall quality of care--a promise that is realized through major changes in workflows within the health care organization. Yet little systematic information exists about the workflow effects of EMRs. Moreover, some of the research to-date points to reduced satisfaction among physicians after implementation of the EMR and increased time, i.e., negative workflow effects. A better understanding of the impact of the EMR on workflows is, hence, vital to understanding what the technology really does offer that is new and unique.

OBJECTIVE

(i) To empirically develop a physician centric conceptual model of the workflow effects of EMRs; (ii) To use the model to understand the antecedents to the physicians' workflow expectation from the new EMR; (iii) To track physicians' satisfaction overtime, 3 months and 20 months after implementation of the EMR; (iv) To explore the impact of technology learning curves on physicians' reported satisfaction levels.

DESIGN

The current research uses the mixed-method technique of concept mapping to empirically develop the conceptual model of an EMR's workflow effects. The model is then used within a controlled study to track physician expectations from a new EMR system as well as their assessments of the EMR's performance 3 months and 20 months after implementation.

SETTING

The research tracks the actual implementation of a new EMR within the outpatient clinics of a large northeastern research hospital.

PARTICIPANTS

The pre-implementation survey netted 20 physician responses; post-implementation Time 1 survey netted 22 responses, and Time 2 survey netted 26 physician responses.

INTERVENTION

The implementation of the actual EMR served as the intervention. Since the study was conducted within the same setting and tracked a homogenous group of respondents, the overall study design ensured against extraneous influences on the results.

MAIN OUTCOME MEASURES

Outcome measures were derived empirically from the conceptual model. They included 85 items that measured physician perceptions of the EMR's workflow effect on the following eight issues: (1) administration, (2) efficiency in patient processing, (3) basic clinical processes, (4) documentation of patient encounter, (5) economic challenges and reimbursement, (6) technical issues, (7) patient safety and care, and (8) communication and confidentiality. The items were used to track expectations prior to implementation and they served as retrospective measures of satisfaction with the EMR in post-implementation Time 1 and Time 2.

RESULTS

The findings suggest that physicians conceptualize EMRs as an incremental extension of older computerized provider order entries (CPOEs) rather than as a new innovation. The EMRs major functional advantages are seen to be very similar to, if not the same as, those of CPOEs. Technology learning curves play a statistically significant though minor role in shaping physician perceptions.

CONCLUSIONS

The physicians' expectations from the EMR are based on their prior beliefs rather than on a rational evaluation of the EMR's fit, functionality, or performance. Their decision regarding the usefulness of the EMR is made very early, within the first few months of use of the EMR. These early perceptions then remain stable and become the lens through which subsequent experience with the EMR is interpreted. The findings suggest a need for communication based interventions aimed at explaining the value, fit, and usefulness of EMRs to physicians early in the pre- and immediate post-EMR implementation stages.

Computerized clinical decision support for prescribing: provision does not guarantee uptake

There is wide variability in the use and adoption of recommendations generated by computerized clinical decision support systems (CDSSs) despite the benefits they may bring to clinical practice. We conducted a systematic review to explore the barriers to, and facilitators of, CDSS uptake by physicians to guide prescribing decisions. We identified 58 studies by searching electronic databases (1990-2007). Factors impacting on CDSS use included: the availability of hardware, technical support and training; integration of the system into workflows; and the relevance and timeliness of the clinical messages. Further, systems that were endorsed by colleagues, minimized perceived threats to professional autonomy, and did not compromise doctor-patient interactions were accepted by users. Despite advances in technology and CDSS sophistication, most factors were consistently reported over time and across ambulatory and institutional settings. Such factors must be addressed when deploying CDSSs so that improvements in uptake, practice and patient outcomes may be achieved.

EHR acceptance factors in ambulatory care: a survey of physician perceptions

With the U.S. government calling for electronic health records (EHRs) for all Americans by the year 2014, adoption of an interoperable EHR is imminent in America's future. However, recent estimates for EHR implementation in the ambulatory care environment are just over 10 percent. This second part of a two-part study examines EHR acceptance factors in an academic-based healthcare system. Innovation diffusion theory and the Technology Acceptance Model provide a combined theoretical framework for this case study. An online questionnaire was administered to 802 faculty, fellow, and resident physicians to explore the factors affecting attitudes toward EHR adoption. In this study, age, years in practice, clinical specialty, health system relationship, and prior computer experience were not predictors of EHR acceptance. In order to facilitate successful adoption of health information systems, social and behavioral factors must be addressed during the EHR planning phase.

Practitioner approaches to the integration of clinical decision support system technology in critical care

OBJECTIVE

The aim of this study was to explore the experiences of nurses and physicians who use a clinical decision support system (CDSS) in the critical care area, focusing on clinicians' motives and values related to decisions to either use or not use this optional technology.

BACKGROUND

Information technology (IT) has been demonstrated to positively impact quality of patient care. Decision-support technology serves as an adjunct to, not as a replacement for, actual clinical decision making. Nurse administrators play an imperative role in the planning and implementation of IT projects and can benefit from understanding clinicians' affective considerations and approaches to the technology.

METHODS

This qualitative study used grounded theory methods. A total of 33 clinicians participated in in-depth structured interviews probing their professional concerns with how the technology is used. Data were analyzed using the constant comparative method.

RESULTS

Medical staff were frustrated by perceived lack of planning input before system implementation. Both nurse and physician cohort groups were dissatisfied with preimplementation education. Barriers to system use were identified in significant detail by the participants.

CONCLUSION

Both nurses and physicians should be involved in preimplementation planning and ongoing evaluation of CDSSs. There is a need for a systematic review or Cochrane meta-analysis describing the affective aspects of successful implementations of decisional technology in critical care, specifically from the perspective of nursing administrators.

A framework for predicting EHR adoption attitudes: a physician survey

With a focus on improving the quality of patient care, the President George W. Bush called for electronic health records (EHRs) for all Americans by the year 2014; however, recent estimates for EHR adoption in the ambulatory care environment are just over 10 percent. The objective of this study was to determine the individual characteristics and the social and technical factors that may contribute to physician acceptance of EHRs. This first paper of a two-part study presents a framework grounded in Diffusion of Innovations theory and the Technology Acceptance Model, which was developed using case study and survey methods, and tested using structural equation modeling (SEM). Model variables explained over 73 percent of the variance in attitude toward EHRs, and acceptable model fit was achieved. Lack of user acceptance continues to impede diffusion of EHRs, and this analysis supports the impact that leadership and other organizational changes have on user adoption.

Evaluating User Satisfaction with an Electronic Prescription System in a Primary Care Group

Introduction: Electronic prescribing has been proposed as an important strategy to reduce medication errors, improve the quality of patient care and create savings in health care costs. Despite these potential advantages, user satisfaction plays a significant role in the success of its implementation. Hence, this study aims to examine users’ satisfaction and factors associated with satisfaction regarding an electronic prescription system implemented in the National Healthcare Group Polyclinics in Singapore. Materials and Methods: An anonymous survey was administered in October 2007 to all physicians, pharmacists and pharmacy technicians working in the 9 National Healthcare Group Polyclinics. Results: Respondents included 118 doctors and 61 pharmacy staff. The overall level of satisfaction with electronic prescribing was high. Doctors and pharmacists reported a high degree of agreement that electronic prescribing reduces prescribing errors and interventions, and they did not want to go back to the paper-based system. Users were generally satisfied with the functionality of the system but there was some degree of workflow interference particularly for the pharmacy staff. Only 56.9% of the pharmacy respondents expressed satisfaction with the review function of the electronic prescription system and only 51.8% and 60% were satisfied when processing prescriptions that included items to be purchased from an external pharmacy or prescriptions with amendments. The results also revealed that satisfaction with the system was more associated with users’ perceptions about the electronic prescription system’s impact on productivity than quality of care. Conclusion: The survey results indicate that the implementation of the electronic prescription system has gone reasonably well. The survey findings provide opportunities for system and workflow enhancement, which is important as these issues could affect the acceptability of a new technology and the speed of diffusion within an organisation. Key words: Information technology, Primary healthcare, Prescriptions

The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review

Previous studies have shown the importance of workflow issues in the implementation of CPOE systems and patient safety practices. To understand the impact of CPOE on clinical workflow, we developed a conceptual framework and conducted a literature search for CPOE evaluations between 1990 and June 2007. Fifty-one publications were identified that disclosed mixed effects of CPOE systems. Among the frequently reported workflow advantages were the legible orders, remote accessibility of the systems, and the shorter order turnaround times. Among the frequently reported disadvantages were the time-consuming and problematic user-system interactions, and the enforcement of a predefined relationship between clinical tasks and between providers. Regarding the diversity of findings in the literature, we conclude that more multi-method research is needed to explore CPOE's multidimensional and collective impact on especially collaborative workflow.

Information technology from novice to expert: implementation implications

AIMS

This paper explores how the Novice-to-Expert Nursing Practice framework can illuminate the challenges of and opportunities in implementing information technology (IT), such as clinical decision support systems (CDSS), in nursing practice.

BACKGROUND

IT implementation in health care is increasing; however, substantial costs and risks remain associated with these projects.

EVALUATION

The theoretical framework of Novice-to-Expert Nursing Practice was applied to current design and implementation literature for CDSS.

KEY ISSUES

Organizational policies and CDSS design affect implementation and user adoption.

CONCLUSIONS

Nursing CDSS can improve the overall quality of care when designed for the appropriate end-user group and based on a knowledge base reflecting nursing expertise.

IMPLICATIONS FOR NURSING MANAGEMENT

Nurse administrators can positively influence CDSS function and end-user acceptance by participating in and facilitating staff nurse involvement in IT design, planning and implementation. Specific steps for nurse administrators and managers are included in this paper.

Critical care nurse practitioners and clinical nurse specialists interface patterns with computer-based decision support systems

PURPOSE

The purposes of this review are to examine the types of clinical decision support systems in use and to identify patterns of how critical care advanced practice nurses (APNs) have integrated these systems into their nursing care patient management practices. The decision-making process itself is analyzed with a focus on how automated systems attempt to capture and reflect human decisional processes in critical care nursing, including how systems actually organize and process information to create outcome estimations based on patient clinical indicators and prognosis logarithms. Characteristics of APN clinicians and implications of these characteristics on decision system use, based on the body of decision system user research, are introduced.

DATA SOURCES

A review of the Medline, Ovid, CINAHL, and PubMed literature databases was conducted using "clinical decision support systems,""computerized clinical decision making," and "APNs"; an examination of components of several major clinical decision systems was also undertaken.

CONCLUSIONS

Use patterns among APNs and other clinicians appear to vary; there is a need for original research to examine how APNs actually use these systems in their practices in critical care settings. Because APNs are increasingly responsible for admission to, and transfer from, critical care settings, more understanding is needed on how they interact with this technology and how they see automated decision systems impacting their practices.

IMPLICATIONS FOR PRACTICE

APNs who practice in critical care settings vary significantly in how they use the clinical decision systems that are in operation in their practice settings. These APNs must have an understanding of their use patterns with these systems and should critically assess whether their patient care decision making is affected by the technology.

Downtime procedures for a clinical information system: a critical issue

As computers become embedded in clinical workflow processes, disruptions to access can have serious consequences. The Health Evaluation through Logical Processing system at LDS Hospital is a computerized hospital information system that has been under continuous development for more than 30 years. The system maintains a 99.85% uptime and averages more than 17,000 logons per day. The first formal downtime plan for this system was developed in 1992 in anticipation of a major hardware installation. In early 2000 after a series of planned downtimes from which we did not recover smoothly, our Software Oversight Committee became interested in understanding downtime procedures. A downtime plan for clinical users was developed and tested and is discussed. A March 2000 downtime survey of 103 clinical staff provided additional information to refine the plan. The downtime plan now includes explicit instructions about the clinical data that must be reentered after a downtime and also includes a plan for a regularly scheduled downtime practice drill similar to a fire drill.

Can organizational support for technology influence job satisfaction?

The relationship between organizational support for telemedicine and nurses' job satisfaction in home health agencies (HHAs) is investigated in this study. Drawing from the theory of perceived organizational support, we propose that HHAs that provide strong support for telemedicine will have higher nurse job satisfaction levels than those that do not. Primary data were collected from 29 HHAs and 917 nurses from 2004 to 2005. The results of the analysis indicate a strong, positive, significant association between organizational support for telemedicine and home health nurses' job satisfaction. We conclude that technological implementation, with sufficient organizational support, can be a strong competitive strategy to attract and retain high-quality nurses.

Cognitive factors influencing perceptions of clinical documentation tools

Identifying healthcare providers' perceptions of clinical documentation methods can inform the design of computer-based documentation tools. The authors investigated the cognitive factors underlying such perceptions by performing a qualitative analysis that included open-ended in-depth interviews of a convenience sample of healthcare providers who use a variety of documentation methods. A total of 16 providers participated in the study; subjects included physicians and nurse practitioners from medical and surgical specialties who used paper- and computer-based documentation tools. Based on interview data, authors identified five factors that influenced satisfaction with clinical documentation tools: document system time efficiency, availability, expressivity, structure, and quality. These factors, if validated by subsequent investigations, can be used to develop a formal conceptual model of providers' perceptions of their satisfaction with various documentation systems.

Telephone triage, expert systems and clinical expertise

This paper reports on a qualitative study of the use of an expert system developed for the British telephone triage service NHS Direct. This system, known as CAS, is designed to standardise and control the interaction between NHS Direct nurses and callers. The paper shows, however, that in practice the nurses use CAS in a range of ways and, in so doing, privilege their own expertise and deliver an individualised service. The paper concludes by arguing that NHS Direct management's policy of using CAS as a means of standardising service delivery will achieve only limited success due not only to the professional ideology of nursing but also to the fact that rule-based expert systems capture only part of what 'experts' do.

Designing an EMR planning process based on staff attitudes toward and opinions about computers in healthcare

Electronic medical record (EMR) systems have been in use for more than 2 decades. Studies documenting nursing satisfaction with an EMR system, the benefits of an EMR, implementation barriers, user acceptance, the importance of staff buy-in, and the importance of attitudes toward and expectations from user buy-in are in the literature. Central to many studies is the importance of nursing staff buy-in to the successful implementation and ongoing use of an EMR, as well as the dependency of buy-in on staff attitudes and expectations. Buy-in is a precursor to effective use. Consequently, staff buy-in is a prerequisite to collecting and making optimum use of the data contained in an EMR. Data collected from an EMR containing rich, accurate documentation of nursing interventions and patient responses support evidence-based practice changes and documentation of the import of the care provided by nurses.

Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study

Despite benefits associated with the use of electronic health records (EHRs), one major barrier to adoption is the concern that EHRs may take longer for physicians to use than paper-based systems. To address this issue, we performed a time-motion study in five primary care clinics. Twenty physicians were observed and specific activities were timed during a clinic session before and after EHR implementation. Surveys evaluated physicians' perceptions regarding the EHR. Post-implementation, the adjusted mean overall time spent per patient during clinic sessions decreased by 0.5 min (p=0.86; 95% confidence interval [-5.05, 6.04]) from a pre-intervention adjusted average of 27.55 min (SE=2.1) to a post-intervention adjusted average of 27.05 min (SE=1.6). A majority of survey respondents believed EHR use results in quality improvement, yet only 29% reported that EHR documentation takes the same amount of time or less compared to the paper-based system. While the EHR did not require more time for physicians during a clinic session, further studies should assess the EHR's potential impact on non-clinic time.

Clinicians' perceptions of clinical decision support integrated into computerized provider order entry

Computerized provider order entry systems can improve patient care by integrating clinical decision support. Decision support is most effective when its content and delivery are acceptable. The authors report the results of a multifaceted survey of the attitudes of housestaff and medical student users of a provider order entry system with integrated decision support at an academic medical center. The survey contained 16 items covering four themes: efficiency, quality of care, results reporting and embedded guidelines. Responses were captured using a five point Likert scale, and were compared using ANOVA and Bartlett's testing. Out of 491 housestaff and 128 medical students, response rates were 47 and 29%, respectively. Among respondents, 72% agreed or strongly agreed that the provider order entry system improves the quality of care that they provide, 54% that the decision support usually help them to provide quality patient care, and 62% that it improves the efficiency of order entry. Respondents were least likely to agree that the display of prior laboratory results influenced their decision to order a subsequent test. There were no significant differences between subspecialties and by advancing years of training among housestaff. Respondents agreed that the integrated clinical decision support enhanced their medical training.

Heterogeneous effect of an Emergency Department Expert Charting System

STUDY OBJECTIVE

We compare results from different modules (occupational exposure to blood and body fluids, low back pain, and fever in children) of the Emergency Department Expert Charting System. Each module of this electronic medical record provides real-time advice based on clinical guidelines embedded in the software.

METHODS

We used a staggered off-on-off interrupted time-series design with an intent-to-treat analysis to implement the Emergency Department Expert Charting System in a university hospital emergency department for the treatment of fever in children, low back pain, and occupational exposure to blood and body fluids. We measured the quality of documentation as the percentage of essential items contained in the medical record and discharge instructions, the percentage of appropriate testing and treatment decisions, median charges per patient visit, physician satisfaction by pre-experiment and postexperiment questionnaires, and patient satisfaction by telephone questionnaire.

RESULTS

The Emergency Department Expert Charting System improved documentation rates for all modules. The Emergency Department Expert Charting System consistently improved the appropriateness of diagnostic testing and treatment decisions for patients with occupational exposure to blood and body fluids while decreasing median charges. For the low back pain and fever in children modules, improvements in appropriateness of testing and treatment were less consistent and did not result in a decrease in charges. Although physicians were generally supportive of the intervention, the physicians' use of the Emergency Department Expert Charting System and satisfaction with the modules were greatest with occupational exposure to blood and body fluids and least with fever in children. For each presenting complaint, mean patient satisfaction was highest during the Emergency Department Expert Charting System phase.

CONCLUSION

The delivery of guidelines through an electronic medical record with background decision support improved documentation, patient care, and patient satisfaction, although effects were heterogeneous across presenting complaints. The optimal guideline implementation strategy likely varies with the nature of the clinical problem and the type of health care delivery setting. For selected problems, delivering guidelines in this format through the Internet holds great promise for modifying physician behavior and improving care (see http://www.needlestick.mednet.ucla.edu ).

The use of computers for clinical care: a case series of advanced U.S. sites

OBJECTIVE

To describe advanced clinical information systems in the context in which they have been implemented and are being used.

DESIGN

Case series of five U.S. hospitals, including inpatient, ambulatory and emergency units. Descriptive study with data collected from interviews, observations, and document analysis.

MEASUREMENTS

The use of computerized results, notes, orders, and event monitors and the type of decision support; data capture mechanisms and data form; impact on clinician satisfaction and clinical processes and outcomes; and the organizational factors associated with successful implementation.

RESULTS

All sites have implemented a wide range of clinical information systems with extensive decision support. The systems had been well accepted by clinicians and have improved clinical processes. Successful implementation required leadership and long-term commitment, a focus on improving clinical processes, and gaining clinician involvement and maintaining productivity.

CONCLUSION

Despite differences in approach there are many similarities between sites in the clinical information systems in use and the factors important to successful implementation. The experience of these sites may provide a valuable guide for others who are yet to start, or are just beginning, the implementation of clinical information systems.

Towards integration of clinical decision support in commercial hospital information systems using distributed, reusable software and knowledge components

PROBLEM

Clinicians' acceptance of clinical decision support depends on its workflow-oriented, context-sensitive accessibility and availability at the point of care, integrated into the Electronic Patient Record (EPR). Commercially available Hospital Information Systems (HIS) often focus on administrative tasks and mostly do not provide additional knowledge based functionality. Their traditionally monolithic and closed software architecture encumbers integration of and interaction with external software modules. Our aim was to develop methods and interfaces to integrate knowledge sources into two different commercial hospital information systems to provide the best decision support possible within the context of available patient data.

METHODS

An existing, proven standalone scoring system for acute abdominal pain was supplemented by a communication interface. In both HIS we defined data entry forms and developed individual and reusable mechanisms for data exchange with external software modules. We designed an additional knowledge support frontend which controls data exchange between HIS and the knowledge modules. Finally, we added guidelines and algorithms to the knowledge library.

RESULTS

Despite some major drawbacks which resulted mainly from the HIS' closed software architectures we showed exemplary, how external knowledge support can be integrated almost seamlessly into different commercial HIS. This paper describes the prototypical design and current implementation and discusses our experiences.

Privacy: a review of the literature

The concept of privacy is used in many disciplines and is recognised as one of the important concepts also in nursing. In this review, a description about the perspectives and dimensions of the concept will be made and empirical studies in the area will be analysed. Perspectives include units experiencing privacy, desired-achieved state, and reactiveness. Dimensions are divided into physical, psychological, social and informational. In the empirical studies, the concept of the privacy has mainly been studied in hospital organisations using the physical dimension. The concept needs further clarification in future.

Physician satisfaction with two order entry systems

OBJECTIVES

In the wake of the Institute of Medicine report, To Err Is Human: Building a Safer Health System (LT Kohn, JM Corrigan, MS Donaldson, eds; Washington, DC: National Academy Press, 1999), numerous advisory panels are advocating widespread implementation of physician order entry as a means to reduce errors and improve patient safety. Successful implementation of an order entry system requires that attention be given to the user interface. The authors assessed physician satisfaction with the user interface of two different order entry systems-a commercially available product, and the Department of Veterans Affairs Computerized Patient Record System (CPRS).

DESIGN AND MEASUREMENT

A standardized instrument for measuring user satisfaction with physician order entry systems was mailed to internal medicine and medicine-pediatrics house staff physicians. The subjects answered questions on each system using a 0 to 9 scale.

RESULTS

The survey response rates were 63 and 64 percent for the two order entry systems. Overall, house staff were dissatisfied with the commercial system, giving it an overall mean score of 3.67 (95 percent confidence interval [95%CI], 3.37-3.97). In contrast, the CPRS had a mean score of 7.21 (95% CI, 7.00-7.43), indicating that house staff were satisfied with the system. Overall satisfaction was most strongly correlated with the ability to perform tasks in a "straightforward" manner.

CONCLUSIONS

User satisfaction differed significantly between the two order entry systems, suggesting that all order entry systems are not equally usable. Given the national usage of the two order entry systems studied, further studies are needed to assess physician satisfaction with use of these same systems at other institutions.

Modeling uncertainty in computerized guidelines using fuzzy logic

Computerized Clinical Practice Guidelines (CPGs) improve quality of care by assisting physicians in their decision making. A number of problems emerges since patients with close characteristics are given contradictory recommendations. In this article, we propose to use fuzzy logic to model uncertainty due to the use of thresholds in CPGs. A fuzzy classification procedure has been developed that provides for each message of the CPG, a strength of recommendation that rates the appropriateness of the recommendation for the patient under consideration. This work is done in the context of a CPG for the diagnosis and the management of hypertension, published in 1997 by the French agency ANAES. A population of 82 patients with mild to moderate hypertension was selected and the results of the classification system were compared to whose given by a classical decision tree. Observed agreement is 86.6% and the variability of recommendations for patients with close characteristics is reduced.

Evaluation of a data warehouse in an academic health sciences center

OBJECTIVES

The Clinical data repository (CDR) at the University of Virginia Health System is a data warehouse that provides direct access to data for clinical research and effective decision making. We undertook an evaluation of the CDR to understand factors affecting its adoption.

DESIGN

We used a theoretical framework that is based on diffusion of innovation theory. Building on validated survey instruments, we developed a questionnaire and conducted interviews of key executive leaders. Fifty-three individuals with logon ids to the CDR completed our questionnaire. Twelve executive leaders were interviewed.

MEASUREMENTS

The outcome variables were the initial and continued use of the CDR. Independent variables included attributes suggested by diffusion theory (i.e. relative advantage, complexity), knowledge and skills expected to correlate with computer usage, and the influence of communication channels.

RESULTS

Our overall response rate was 82%. We identified characteristics of users associated with the initial decision to use the CDR. Compatibility with an individual's skills and work style was associated strongly with satisfaction and continued use. Secondly, the importance of organizational culture and the need for data was illuminated by management interviews.

CONCLUSIONS

We have shown that diffusion of innovation theory can be used to help understand factors contributing to the success of a data warehouse in a healthcare setting. Our results suggest areas for future research and inquiry as the CDR evolves.

Comparing user acceptance of a computer system in two pediatric offices: a qualitative study

The purpose of this qualitative study was to examine user acceptance of a clinical computer system in two pediatric practices in the southeast. Data were gathered through interviews with practice and IS staff, observations in the clinical area, and review of system implementation records. Five months after implementation, Practice A continued to use the system but Practice B had quit using it because it was unacceptable to the users. The results are presented here, in relation to a conceptual framework, which was originally developed to describe the process of successful implementation of research findings into practice. Five main themes were identified relative to the differences in user acceptance at the two practices: 1) Benefits versus expense of system use varied, 2) Organizational cultures differed, 3) IS staff's relationship with practices differed, 4) Post-implementation experiences differed, and 5) Transfer of technology from the academic center to private practice proved challenging in Practice B. The findings indicate a need for the development and validation of tools to measure healthcare organizational climate and readiness for change.

Managing change: an overview

As increasingly powerful informatics systems are designed, developed, and implemented, they inevitably affect larger, more heterogeneous groups of people and more organizational areas. In turn, the major challenges to system success are often more behavioral than technical. Successfully introducing such systems into complex health care organizations requires an effective blend of good technical and good organizational skills. People who have low psychological ownership in a system and who vigorously resist its implementation can bring a "technically best" system to its knees. However, effective leadership can sharply reduce the behavioral resistance to change-including to new technologies-to achieve a more rapid and productive introduction of informatics technology. This paper looks at four major areas-why information system failures occur, the core theories supporting change management, the practical applications of change management, and the change management efforts in informatics.

Use of the equity implementation model to review clinical system implementation efforts: a case report

This paper presents the equity implementation model (EIM) in the context of a case that describes the implementation of a medical scheduling system. The model is based on equity theory, a well-established theory in the social sciences that has been tested in hundreds of experimental and field studies. The predictions of equity theory have been supported in organizational, societal, family, and other social settings. Thus, the EIM helps provide a theory-based understanding for collecting and reviewing users' reactions to, and acceptance or rejection of, a new technology or system. The case study (implementation of a patient scheduling and appointment setting system in a large health maintenance organization) illustrates how the EIM can be used to examine users' reactions to the implementation of a new system.

Medical informatics education: the University of Utah experience

The University of Utah has been educating health professionals in medical informatics since 1964. Over the 35 years since the program's inception, 272 graduate students have studied in the department. Most students have been male (80 percent) and have come from the United States (75 percent). Students entering the program have had diverse educational backgrounds, most commonly in medicine, engineering, computer science, or biology (59 percent of all informatics students). A total of 209 graduate degrees have been awarded, with an overall graduation rate of 87 percent since the program's start. Alumni are located in the United States (91 percent) and abroad (9 percent); half (51 percent) have remained in Utah. Former students are employed in a wide variety of jobs, primarily concerned with the application of medical informatics in sizable health care delivery organizations. Trends toward increasing managerial responsibility for medical informatics graduates and the emergence of the chief information officer role are noted.

Workflow analysis and evidence-based medicine: towards integration of knowledge-based functions in hospital information systems

The large extent and complexity of scientific evidence described in the concept of evidence-based medicine often overwhelms clinicians who want to apply best external evidence. Hospital Information Systems usually do not provide knowledge-based functions to support context-sensitive linking to external information sources. Knowledge-based components need specific data, which must be entered manually and should be well adapted to clinical environment to be accepted by clinicians. This paper describes a workflow-based approach to understand and visualize clinical reality as a preliminary to designing software applications, and possible starting points for further software development.

Housestaff attitudes toward computer-based clinical decision support

OBJECTIVE

To measure housestaff attitudes towards computer-based decision support and their threshold for having CDSS messages displayed.

DESIGN

770 self-administered surveys were distributed to housestaff physicians.

RESULTS

209 surveys were returned. 63% of respondents agreed or strongly agreed that CDSS would improve quality of care, while 52% agreed or strongly agreed that it would decrease adverse drug events. Respondents were neutral regarding the impact of CDSS on productivity and on their autonomy. Sixty percent approved of a reminder to consider surgical consultation in a patient with abdominal pain, while 88% approved of alerts about hypokalemia. Respondents felt both reminders should be triggered when their PPV exceeded 67%. Attitudes toward POE correlated positively with attitudes toward CDSS (Pearson's rho 0.56; p < 0.0001). Respondents who were dissatisfied with POE had a higher threshold PPV for seeing reminders.

CONCLUSION

The majority of housestaff favor the implementation of a CDSS. Housestaff with favorable POE experiences were more likely to endorse CDSS, and those with negative POE experience were more likely to oppose it. The results suggest that a carefully designed CDSS with rules constructed to exceed a threshold PPV would be accepted by housestaff.

Computers in the consulting room: a case study of clinician and patient perspectives

Few clinicians in the United States use computers during patient encounters and many still worry that computers will depersonalize their interactions with patients. This case study describes patient and clinician reactions to a computer-based health appraisal system. Findings showed no difference in any aspect of patient satisfaction between computer and non-computer groups. Use of a computer in the consulting room neither depersonalized nor enhanced patient satisfaction. Clinicians (in this case, nurse practitioners and physician assistants) were willing to use the system, which they perceived as having benefits for patient care, but were concerned about the increased time required for exams, effort required to learn the system while still interacting appropriately with the patient, increased monitoring of their performance, and other organizational issues. Clinicians who used the system showed a higher tolerance for uncertainty and communicated more frequently with each other and with others throughout the department. Implementation was slowed by the need to demonstrate the monetary value of the system.

Q-methodology: definition and application in health care informatics

OBJECTIVE

To introduce the Q-methodology research technique to the field of health informatics. Q-methodology--the systematic study of subjectivity--was used to identify and categorize the opinions of primary care physicians and medical students that contributed to our understanding of their reasons for acceptance of and/or resistance to adapting information technologies in the health care workplace.

DESIGN

Thirty-four physicians and 25 medical students from the Chicago area were surveyed and asked to rank-order 30 opinion statements about information technologies within the health care workplace. The Q-methodology research technique was employed to structure an opinion typology from their rank-ordered statements. (The rank-ordered sorts were subjected to correlation and by-person factor analysis to obtain groupings of participants who sorted the opinion statements into similar arrangements.)

RESULTS

The typology for this study revealed groupings of similar opinion-types associated with the likelihood of physicians and medical students to adapt information technology into their health care workplace. A typology of six opinions was identified in the following groups: (1) Full-Range Adopters; (2) Skills-Concerned Adopters; (3) Technology-Critical Adopters; (4) Independently-Minded and Concerned; (5) Inexperienced and Worried; and (6) Business-Minded and Adaptive. It is imperative to understand that in the application of Q-methodology, the domain is subjectivity and research is performed on small samples. The methodology is a combination of qualitative and quantitative research techniques that reveals dimensions of subjective phenomena from a perspective intrinsic to the individual to determine what is statistically different about the dimensions and to identify characteristics of individuals who share common viewpoints. Low response rates do not bias Q-methodology because the primary purpose is to identify a typology, not to test the typology's proportional distribution within the larger population.

CONCLUSION

Q-methodology can allow for the simultaneous study of objective and subjective issues to determine an individual's opinion and forecast their likeliness to adapt information technologies in the health care workplace. This study suggests that an organization's system implementers could employ Q-methodology to individualize and customize their approach to understanding the personality complexities of physicians in their organization and their willingness to adapt and utilize information technologies within the workplace.

Medical narratives in electronic medical records

In this article, we describe the state of the art and directions of current development and research with respect to the inclusion of medical narratives in electronic medical-record systems. We used information about 20 electronic medical-record systems as presented in the literature. We divided these systems into 'classical' systems that matured before 1990 and are now used in a broad range of medical domains, and 'experimental' systems, more recently developed and, in general, more innovative. In the literature, three major challenges were addressed: facilitation of direct data entry, achieving unambiguous understandability of data, and improvement of data presentation. Promising approaches to tackle the first and second challenge are the use of dynamic data-entry forms that anticipate sensible input, and free-text data entry followed by natural-language interpretation. Both these approaches require a highly expressive medical terminology. How to facilitate the access to medical narratives has not been studied much. We found facilitating examples of presenting this information as fluent prose, of optimising the screen design with fixed position cues, and of imposing medical narratives with a structure of indexable paragraphs that can be used in flowsheets. We conclude that further study is needed to develop an optimal searching structure for medical narratives.

An information system for metabolic intensive care unit

The metabolic intensive care unit (MICU) is a special department where patients requiring long-term metabolic and nutritional support are treated. Parenteral nutrition is of primary importance in the MICU. The optimal organisation of many different data within the MICU, requires an information system which is computerised. However, the problem of paramount importance is data collection. To prevent data losses, we have constructed our computerised information system for MICU as the only information channel for mutual communication of physicians, nursing staff, laboratories, etc. The following modules are typical and necessary for work in the MICU: (1) module for parenteral nutrition; (2) module for enteral nutrition; (3) biochemical module; (4) drug therapy administration module; (5) clinical microbiology database; (6) nursing therapeutic module. At present we have 6-years experience of full computerisation of the MICU. According to this experience the assumption that the logical central point of medical information system must be close to the physician, nurse and patient proved to be correct.

Clinical Information Systems for Intensive Care, Pediatric Critical Care, and Neonatology

Computer information systems are expected to soon take the place of current paper charting practices, and they offer great promise to assist management of the considerable amounts of data encountered in the information-rich environment of intensive care units (ICUs). Efforts to create an electronic medical record (EMR) have been underway for more than two decades, and major national organizations, such as the Institute of Medicine, have issued recommendations on standards. Benefits of an EMR include a legible patient record, enhanced communication, provision of timely reminders and alerts to clinicians, reduction of calculation errors, access to data bases for quality assurance and research, reduced healthcare costs, and improved patient outcomes. Despite these benefits, successful EMR implementations have been confined to a few committed institutions, and expensive failures have occurred. Practitioners of neonatology and pediatric intensive care are likely to have substantial difficulty implementing an EMR to fit their specialized needs because most experience in this area has been gained through care of adult patients, and systems being developed are oriented toward nonpediatric patients. It is therefore important to examine experience thus far with the functional components of an EMR so practitioners will be able to evaluate systems better as they become available. System components discussed include nursing charting facilities, lab reporting, physician order entry, physician progress notes, structured reports, decision support systems, and problem list management. Other concerns discussed include research and quality assurance functions, data access and confidentiality issues, and electronic mail. Maximizing the “structured data” content, as opposed to narrative content of an EMR, is an important priority, and progress on developing a uniform medical language is discussed. An approach to evaluating clinical information systems for use in the ICU is presented; it should assist practitioners of pediatric critical care and neonatology in identifying computer-based charting solutions that are optimal for infants and children, while cooperating with medical center-wide needs for compatibility and a common data base.

Implementation of physician order entry: user satisfaction and self-reported usage patterns

OBJECTIVES

To evaluate user satisfaction, correlates of satisfaction, and self-reported usage patterns regarding physician order entry (POE) in one hospital.

DESIGN

Surveys were sent to physician and nurse POE users from medical and surgical services.

RESULTS

The users were generally satisfied with POE (mean = 5.07 on a 1 to 7 scale). The physicians were more satisfied than the nurses, and the medical staff were more satisfied than the surgical staff; satisfaction levels were acceptable (more than 3.50) even in the less satisfied groups. Satisfaction was highly correlated with perceptions about POE's effects on productivity, ease of use, and speed. POE features directed at improving the quality of care were less strongly correlated with satisfaction. The physicians valued POE's off-floor accessibility most, and the nurses valued legibility and accuracy of POE orders most. Some features, such as off-floor ordering, were perceived to be highly useful and reported to be frequently used by the physicians; while other features, such as "quick mode'' ordering and personal order sets, received little self-reported use.

CONCLUSIONS

Survey of POE users showed that satisfaction with POE was good. Satisfaction was more correlated with perceptions about POE's effect on productivity than with POE's effect on quality of care. Physicians and nurses constitute two very different types of users, underscoring the importance of involving both physicians and nonphysicians in POE development. The results suggest that development efforts should focus on improving system speed, adding on-line help, and emphasizing quality benefits of POE.