Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting

A Survey of the Literature on Unintended Consequences Associated with Health Information Technology: 2014-2015

OBJECTIVE

To summarize recent research on unintended consequences associated with implementation and use of health information technology (health IT). Included in the review are original empirical investigations published in English between 2014 and 2015 that reported unintended effects introduced by adoption of digital interventions. Our analysis focuses on the trends of this steam of research, areas in which unintended consequences have continued to be reported, and common themes that emerge from the findings of these studies.

METHOD

Most of the papers reviewed were retrieved by searching three literature databases: MEDLINE, Embase, and CINAHL. Two rounds of searches were performed: the first round used more restrictive search terms specific to unintended consequences; the second round lifted the restrictions to include more generic health IT evaluation studies. Each paper was independently screened by at least two authors; differences were resolved through consensus development.

RESULTS

The literature search identified 1,538 papers that were potentially relevant; 34 were deemed meeting our inclusion criteria after screening. Studies described in these 34 papers took place in a wide variety of care areas from emergency departments to ophthalmology clinics. Some papers reflected several previously unreported unintended consequences, such as staff attrition and patients' withholding of information due to privacy and security concerns. A majority of these studies (71%) were quantitative investigations based on analysis of objectively recorded data. Several of them employed longitudinal or time series designs to distinguish between unintended consequences that had only transient impact, versus those that had persisting impact. Most of these unintended consequences resulted in adverse outcomes, even though instances of beneficial impact were also noted. While care areas covered were heterogeneous, over half of the studies were conducted at academic medical centers or teaching hospitals.

CONCLUSION

Recent studies published in the past two years represent significant advancement of unintended consequences research by seeking to include more types of health IT applications and to quantify the impact using objectively recorded data and longitudinal or time series designs. However, more mixed-methods studies are needed to develop deeper insights into the observed unintended adverse outcomes, including their root causes and remedies. We also encourage future research to go beyond the paradigm of simply describing unintended consequences, and to develop and test solutions that can prevent or minimize their impact.

Unintended Consequences: New Problems, New Solutions. Contributions From 2015

OBJECTIVE

To select the best of the 2015 published papers on unintended consequences of healthcare information technology (HIT).

METHOD

Literature searches in several areas of scholarship, including IT, human factors, evaluation studies, medical errors, medical informatics, and implementation science. Also, because the specific terms "unintended consequences" were not often included in abstracts and titles, a more nuanced search algorithm was developed.

RESULTS

We identified 754 papers that had some empirical research on unintended consequences of HIT. An initial screen of titles and abstracts reduced this to 171 papers of potential interest. We then further filtered out papers that did not meet the following criteria: 1) the paper had to report an original empirical investigation, and 2) the impact reported had to be not negligible, i.e., in quantitative studies, the results related to unintended consequences were statistically significant; and in qualitative studies the relevant themes emerged were prominent. This resulted in 33 papers of which 15 were selected as best paper candidates. Each of these 15 papers was then separately evaluated by four reviewers. The final selection of four papers was made jointly by the external reviewers and the two section editors.

CONCLUSIONS

There is a growing awareness of the importance of HIT's unintended consequences-be they generated by the HIT vendors, the implementation process, the consultants, the users, or most probably, some combination of the above. There has also been greater creativity in use of data sources, including secondary data (e.g., medical malpractice cases and surveys) and a wider acceptance of mixed methods to identify unintended consequences. Unfortunately, the complexity of causes mitigates the value of recommendations to avoid unwanted outcomes. Suggestions are often contentious rather than obvious, setting-specific, and not universally applicable. "Lessons learned" often take on generalized-and perhaps platitudinous-forms, such as: "plan extra time," "involve all of the stakeholders," "recognize the different needs of different units or disciplines." The greater awareness of these problems, and the increased desire to identify and eliminate them is clearly reflected in the area's growing literature. We are hopeful the topic will receive additional attention and the discipline will improve its ability to identify and address these unexpected and usually adverse outcomes.

Methods for Addressing Technology-induced Errors: The Current State

OBJECTIVES

The objectives of this paper are to review and discuss the methods that are being used internationally to report on, mitigate, and eliminate technology-induced errors.

METHODS

The IMIA Working Group for Health Informatics for Patient Safety worked together to review and synthesize some of the main methods and approaches associated with technology- induced error reporting, reduction, and mitigation. The work involved a review of the evidence-based literature as well as guideline publications specific to health informatics.

RESULTS

The paper presents a rich overview of current approaches, issues, and methods associated with: (1) safe HIT design, (2) safe HIT implementation, (3) reporting on technology-induced errors, (4) technology-induced error analysis, and (5) health information technology (HIT) risk management. The work is based on research from around the world.

CONCLUSIONS

Internationally, researchers have been developing methods that can be used to identify, report on, mitigate, and eliminate technology-induced errors. Although there remain issues and challenges associated with the methodologies, they have been shown to improve the quality and safety of HIT. Since the first publications documenting technology-induced errors in healthcare in 2005, we have seen in a short 10 years researchers develop ways of identifying and addressing these types of errors. We have also seen organizations begin to use these approaches. Knowledge has been translated into practice in a short ten years whereas the norm for other research areas is of 20 years.

Understanding Unintended Consequences and Health Information Technology:. Contribution from the IMIA Organizational and Social Issues Working Group

OBJECTIVE

No framework exists to identify and study unintended consequences (UICs) with a focus on organizational and social issues (OSIs). To address this shortcoming, we conducted a literature review to develop a framework for considering UICs and health information technology (HIT) from the perspective of OSIs.

METHODS

A literature review was conducted for the period 2000- 2015 using the search terms "unintended consequences" and "health information technology". 67 papers were screened, of which 18 met inclusion criteria. Data extraction was focused on the types of technologies studied, types of UICs identified, and methods of data collection and analysis used. A thematic analysis was used to identify themes related to UICs.

RESULTS

We identified two overarching themes. One was the definition and terminology of how people classify and discuss UICs. Second was OSIs and UICs. For the OSI theme, we also identified four sub-themes: process change and evolution, individual-collaborative interchange, context of use, and approaches to model, study, and understand UICs.

CONCLUSIONS

While there is a wide body of research on UICs, there is a lack of overall consensus on how they should be classified and reported, limiting our ability to understand the implications of UICs and how to manage them. More mixed-methods research and better proactive identification of UICs remain priorities. Our findings and framework of OSI considerations for studying UICs and HIT extend existing work on HIT and UICs by focusing on organizational and social issues.

Improving Evaluation to Address the Unintended Consequences of Health Information Technology:. a Position Paper from the Working Group on Technology Assessment & Quality Development

BACKGROUND AND OBJECTIVES

With growing use of IT by healthcare professionals and patients, the opportunity for any unintended effects of technology to disrupt care health processes and outcomes is intensified. The objectives of this position paper by the IMIA Working Group (WG) on Technology Assessment and Quality Development are to highlight how our ongoing initiatives to enhance evaluation are also addressing the unintended consequences of health IT.

METHODS

Review of WG initiatives Results: We argue that an evidence-based approach underpinned by rigorous evaluation is fundamental to the safe and effective use of IT, and for detecting and addressing its unintended consequences in a timely manner. We provide an overview of our ongoing initiatives to strengthen study design, execution and reporting by using evaluation frameworks and guidelines which can enable better characterization and monitoring of unintended consequences, including the Good Evaluation Practice Guideline in Health Informatics (GEP-HI) and the Statement on Reporting of Evaluation Studies in Health Informatics (STARE-HI). Indicators to benchmark the adoption and impact of IT can similarly be used to monitor unintended effects on healthcare structures, processes and outcome. We have also developed EvalDB, a web-based database of evaluation studies to promulgate evidence about unintended effects and are developing the content for courses to improve training in health IT evaluation.

CONCLUSION

Evaluation is an essential ingredient for the effective use of IT to improve healthcare quality and patient safety. WG resources and skills development initiatives can facilitate a proactive and evidence-based approach to detecting and addressing the unintended effects of health IT.

Development of a cognitive framework of patient record summary review in the formative phase of user-centered design

Excellent usability characteristics allow electronic health record (EHR) systems to more effectively support clinicians providing care and contribute to better quality and safety. The Office of the National Coordinator for Health IT (ONC) therefore requires all vendors to follow a User-Centered Design (UCD) process to increase the usability of their products in order to meet certification criteria for the Safety-Enhanced Design part of the Meaningful Use (stage 2) EHR incentive program. This report describes the initial stage of a UCD process in which foundational design concepts were formulated. We designed a functional prototype of an EHR module intended to help clinicians to efficiently complete a summary review of an electronic patient record before an ambulatory visit. Cognitively-based studies were performed and the results used to develop a cognitive framework that subsequently guided design of a prototype. Results showed that clinicians categorized and reasoned with patient data in distinct patterns; they preferred to review relevant history in the assessment and plan section of the most recent note, to search for changes in health and for new episodes of care since the last visit and to look up current-day data such as vital signs. These basic concepts were represented in the design, for instance, by screen division into vertical thirds that had historical content to the left and most recent data to the right. Other characteristics such as visual association of contextual information or direct, one-click access to the assessment and plan section of visit notes were directly informed by our findings and refined in a series of UCD-specific iterative testing. Understanding of tasks and cognitive demands early in the UCD process was critically important for developing a tool optimized for reasoning and workflow preferences of clinicians.

A literature review for large-scale health information system project planning, implementation and evaluation

Information technology is perceived as a potential panacea for healthcare organisations to manage pressure to improve services in the face of increased demand. However, the implementation and evaluation of health information systems (HIS) is plagued with problems and implementation shortcomings and failures are rife. HIS implementation is complex and relies on organisational, structural, technological, and human factors to be successful. It also requires reflective, nuanced, multidimensional evaluation to provide ongoing feedback to ensure success. This article provides a comprehensive review of the literature about evaluating and implementing HIS, detailing the challenges and recommendations for both evaluators and healthcare organisations. The factors that inhibit or promote successful HIS implementation are identified and effective evaluation strategies are described with the goal of informing teams evaluating complex HIS.

The painful truth: The documentation burden of a trauma surgeon

BACKGROUND

Implementation of the electronic medical record (EMR) has introduced several unintended consequences, including increased documentation demands. The purpose of this study was to define the EMR documentation burden and its economic impact at a busy regional Level I trauma center, comparing attending trauma surgeons (TSs) with orthopedic surgeons (OSs), and neurosurgeons (NSs).

METHODS

The EMR was queried to determine the number of attending documentation entries during 2014 for TS, OS, and NS. The eight TSs were then surveyed to estimate the time it took to write each note type, and this was used to calculate the total time needed for documentation. The hospital financial database was queried for 2014 hospital charges and work relative value units (WRVUs) for TSs, OSs, and NSs to generate a comparison. The charges and WRVUs were broken down into those generated from nonprocedural documentation and procedures.

RESULTS

During 2014, there were 5,864 trauma activations with 3,111 patient admissions. The attending TSs wrote a total of 26,455 documentation entries. Of these notes, 92% were from inpatients, and 74% were progress notes. Documentation time estimates for TSs demonstrated that it took 1,760.5 hours or 73.3 twenty-four-hour days to complete these 26,455 notes. Financial data revealed that 44% of the TS charges were directly related to nonprocedural documentation, compared with 14% for OSs and 7% for NSs. Evaluation of WRVUs demonstrated that 55% of the TS WRVUs were directly related to nonprocedural documentation, compared with 28% for OSs and 19% for NSs.

CONCLUSION

The EMR has introduced a significant documentation burden to the busy TSs. This documentation burden is critical for defining hospital charges and WRVUs, and it differs from that of OSs and NSs. Workflow changes, such as the introduction of scribes, may lessen the documentation burden and improve hospital charges and WRVUs of the TSs.

Information security risk management for computerized health information systems in hospitals: a case study of Iran

Background

In recent years, hospitals in Iran – similar to those in other countries – have experienced growing use of computerized health information systems (CHISs), which play a significant role in the operations of hospitals. But, the major challenge of CHIS use is information security. This study attempts to evaluate CHIS information security risk management at hospitals of Iran.

Materials and methods

This applied study is a descriptive and cross-sectional research that has been conducted in 2015. The data were collected from 551 hospitals of Iran. Based on literature review, experts’ opinion, and observations at five hospitals, our intensive questionnaire was designed to assess security risk management for CHISs at the concerned hospitals, which was then sent to all hospitals in Iran by the Ministry of Health.

Results

Sixty-nine percent of the studied hospitals pursue information security policies and procedures in conformity with Iran Hospitals Accreditation Standards. At some hospitals, risk identification, risk evaluation, and risk estimation, as well as risk treatment, are unstructured without any specified approach or methodology. There is no significant structured approach to risk management at the studied hospitals.

Conclusion

Information security risk management is not followed by Iran’s hospitals and their information security policies. This problem can cause a large number of challenges for their CHIS security in future. Therefore, Iran’s Ministry of Health should develop practical policies to improve information security risk management in the hospitals of Iran.

Decision-making in healthcare as a complex adaptive system

Healthcare transformation requires a change in how the business of healthcare is done. Traditional decision-making approaches based on stable and predictable systems are inappropriate in healthcare because of the complex nature of healthcare delivery. This article reviews challenges to using traditional decision-making approaches in healthcare and how insight from Complex Adaptive Systems (CAS) could support healthcare management. The article also provides a system model to guide decision-making in healthcare as a CAS.

Trends in biomedical informatics: automated topic analysis of JAMIA articles

Biomedical Informatics is a growing interdisciplinary field in which research topics and citation trends have been evolving rapidly in recent years. To analyze these data in a fast, reproducible manner, automation of certain processes is needed. JAMIA is a "generalist" journal for biomedical informatics. Its articles reflect the wide range of topics in informatics. In this study, we retrieved Medical Subject Headings (MeSH) terms and citations of JAMIA articles published between 2009 and 2014. We use tensors (i.e., multidimensional arrays) to represent the interaction among topics, time and citations, and applied tensor decomposition to automate the analysis. The trends represented by tensors were then carefully interpreted and the results were compared with previous findings based on manual topic analysis. A list of most cited JAMIA articles, their topics, and publication trends over recent years is presented. The analyses confirmed previous studies and showed that, from 2012 to 2014, the number of articles related to MeSH terms Methods, Organization & Administration, and Algorithms increased significantly both in number of publications and citations. Citation trends varied widely by topic, with Natural Language Processing having a large number of citations in particular years, and Medical Record Systems, Computerized remaining a very popular topic in all years.

Physician Perceptions and Beliefs about Generating and Providing a Clinical Summary of the Office Visit

BACKGROUND

A core measure of the meaningful use of EHR incentive program is the generation and provision of the clinical summary of the office visit, or the after visit summary (AVS), to patients. However, little research has been conducted on physician perceptions and beliefs about the AVS.

OBJECTIVES

Evaluate physician perceptions and beliefs about the AVS and the effect of the AVS on workload, patient outcomes, and the care the physician delivers.

METHODS

A cross-sectional online survey of physicians at two academic medical centers (AMCs) in the northeast who are participating in the meaningful use EHR incentive program.

RESULTS

Of the 1 795 physicians at both AMCs participating in the incentive program, 853 completed the survey for a response rate of 47.5%. Eighty percent of the respondents reported that the AVS was easy (very easy or quite easy or somewhat easy) to generate and provide to patients. Nonetheless, more than three-fourths of the respondents reported a negative effect of generating and providing the AVS on workload of office staff (78%) and workload of physicians (76%). Primary care physicians had more positive beliefs about the effect of the AVS on patient outcomes than specialists (p<0.001) and also had more positive beliefs about the effect of the AVS on the care they delivered than specialists (p<0.001).

CONCLUSIONS

Achieving the core meaningful use measure of generating and providing the AVS was easy for physicians but it did not necessarily translate into positive beliefs about the effect of the AVS on patient outcomes or the care the physician delivered. Physicians also had negative beliefs about the effect of the AVS on workload. To promote positive beliefs among physicians around the AVS, organizations should obtain physician input into the design and implementation of the AVS and develop strategies to mitigate its negative impacts on workload.

Review of Social and Organizational Issues in Health Information Technology

Objectives

This paper reviews organizational and social issues (OSIs) in health information technology (HIT).

Methods

A review and synthesis of the literature on OSIs in HIT was conducted.

Results

Five overarching themes with respect to OSIs in HIT were identified and discussed: scope and frameworks for defining OSIs in HIT, context matters, process immaturity and complexity, trade-offs will happen and need to be discussed openly, and means of studying OSIs in HIT.

Conclusions

There is a wide body of literature that provides insight into OSIs in HIT, even if many of the studies are not explicitly labelled as such. The two biggest research needs are more explicit and theoretical studies of OSI in HITs and more research on integrating micro and macro perspectives of HIT use in organizations.

Causes and consequences of e-prescribing errors in community pharmacies

Major national policy forces are promoting the adoption and use of health information technology (health IT) to improve the quality, safety, and efficiency of health care delivery. One such health IT is electronic prescribing (e-prescribing), which is the direct transmission of prescription information from a provider to a pharmacy. Given research showing that handwritten prescriptions are unsafe and associated errors can lead to tremendous inefficiency for patients and pharmacists, e-prescribing has many potential benefits. However, as with the introduction of any new technology, unintended, adverse consequences may result. The purpose of this review is to explore the causes and consequences of e-prescribing errors in community pharmacies, which are pharmacies not affiliated with a hospital or clinic. Many new types of errors - including provider order entry errors, transcription errors, and dispensing errors - appear to result from e-prescribing. These lead to important consequences for pharmacies, including safety threats to patients, reduced efficiency for pharmacists, processing delays, and increased pharmacy cost. Increased attention to system design and pharmacist training, as well as additional research in this area, will be critical to realize the full benefits of e-prescribing.

A smartphone-enabled communication system to improve hospital communication: usage and perceptions of medical trainees and nurses on general internal medicine wards

BACKGROUND

There is increasing interest in the use of information and communication technologies to improve how clinicians communicate in hospital settings.

METHODS

We implemented a communication system with support for physician handover and secure messaging on 2 general internal medicine wards. We measured usage and surveyed physicians and nurses on perceptions of the system's effects on communication.

RESULTS

Between May 2011 and August 2012, a clinical teaching team received, on average, 14.8 messages per day through the system. Messages were typically sent as urgent (69.1%) and requested a text reply (76.5%). For messages requesting a text reply, 8.6% did not receive a reply. For those messages that did receive a reply, the median response time was 2.3 minutes, and 84.5% of messages received a reply within 15 minutes. Of those who completed the survey, 95.3% were medical residents (82 of 86) and 81.7% were nurses (83 of 116). Medical trainees (82.8%) and nursing staff (78.3%) agreed or strongly agreed that the system helped to speed up their daily work tasks. Overall, 67.1% of the trainees and 73.2% of nurses agreed or strongly agreed that the system made them more accountable in their clinical roles. Only 35.8% of physicians and 26.3% of nurses agreed or strongly agreed that the system was useful for communicating complex issues.

CONCLUSIONS

In summary, with a system designed to improve communication, we found that there was high uptake and that users perceived that the system improved efficiency and accountability but was not appropriate for communicating complex issues.

Dashboards for improving patient care: review of the literature

AIM

This review aimed to provide a comprehensive overview of the current state of evidence for the use of clinical and quality dashboards in health care environments.

METHODS

A literature search was performed for the dates 1996-2012 on CINAHL, Medline, Embase, Cochrane Library, PsychInfo, Science Direct and ACM Digital Library. A citation search and a hand search of relevant papers were also conducted.

RESULTS

One hundred and twenty two full text papers were retrieved of which 11 were included in the review. There was considerable heterogeneity in implementation setting, dashboard users and indicators used. There was evidence that in contexts where dashboards were easily accessible to clinicians (such as in the form of a screen saver) their use was associated with improved care processes and patient outcomes.

CONCLUSION

There is some evidence that implementing clinical and/or quality dashboards that provide immediate access to information for clinicians can improve adherence to quality guidelines and may help improve patient outcomes. However, further high quality detailed research studies need to be conducted to obtain evidence of their efficacy and establish guidelines for their design.

Can utilizing a computerized provider order entry (CPOE) system prevent hospital medical errors and adverse drug events?

Computerized provider order entry (CPOE) systems allow physicians to prescribe patient services electronically. In hospitals, CPOE essentially eliminates the need for handwritten paper orders and achieves cost savings through increased efficiency. The purpose of this research study was to examine the benefits of and barriers to CPOE adoption in hospitals to determine the effects on medical errors and adverse drug events (ADEs) and examine cost and savings associated with the implementation of this newly mandated technology. This study followed a methodology using the basic principles of a systematic review and referenced 50 sources. CPOE systems in hospitals were found to be capable of reducing medical errors and ADEs, especially when CPOE systems are bundled with clinical decision support systems designed to alert physicians and other healthcare providers of pending lab or medical errors. However, CPOE systems face major barriers associated with adoption in a hospital system, mainly high implementation costs and physicians' resistance to change.

Big Data in Healthcare - Defining the Digital Persona through User Contexts from the Micro to the Macro. Contribution of the IMIA Organizational and Social Issues WG

OBJECTIVES

While big data offers enormous potential for improving healthcare delivery, many of the existing claims concerning big data in healthcare are based on anecdotal reports and theoretical vision papers, rather than scientific evidence based on empirical research. Historically, the implementation of health information technology has resulted in unintended consequences at the individual, organizational and social levels, but these unintended consequences of collecting data have remained unaddressed in the literature on big data. The objective of this paper is to provide insights into big data from the perspective of people, social and organizational considerations.

METHOD

We draw upon the concept of persona to define the digital persona as the intersection of data, tasks and context for different user groups. We then describe how the digital persona can serve as a framework to understanding sociotechnical considerations of big data implementation. We then discuss the digital persona in the context of micro, meso and macro user groups across the 3 Vs of big data.

RESULTS

We provide insights into the potential benefits and challenges of applying big data approaches to healthcare as well as how to position these approaches to achieve health system objectives such as patient safety or patient-engaged care delivery. We also provide a framework for defining the digital persona at a micro, meso and macro level to help understand the user contexts of big data solutions.

CONCLUSION

While big data provides great potential for improving healthcare delivery, it is essential that we consider the individual, social and organizational contexts of data use when implementing big data solutions.

Nurses' use of an integrated electronic health record: results of a case site analysis

PURPOSE

To explore how nurses use an integrated Electronic Health Record (EHR) in practice.

METHODS

A multi-site case study across two hospitals in Kaiser Permanente Northern California. Non-participant observation was used to explore nurses' use of the EHR, while semi-structured interviews with nurses and managers explored their perceptions of the EHR and how it affected their practice. Data were analyzed thematically using codes derived deductively from the literature and inductively from the data.

RESULTS

Key themes arising from the analysis suggest that the EHR changed various elements of the way nurses practiced. Introducing the EHR was thought to have improved communication, ease of access to information and the safety of medication administration processes. At an organizational level, there was variability in how the EHR was used to support care documentation and initiatives to improve the quality of care provided by nurses.

CONCLUSION

The EHR was perceived to improve efficiency, safety and communication by the majority of nurses who were interviewed. However, it is likely that a number of other factors such as individual nurse's characteristics and organizational culture influence how an EHR can be used effectively to improve outcomes for patients.

Expert panel evaluation of health information technology effects on adverse events

RATIONALE, AIMS AND OBJECTIVES

Adverse events (AEs) among hospitalized patients occur frequently and result in significant sequelae. Federal policy is incentivizing health information technology (HIT) use, although research demonstrating safety benefits from HIT is mixed. Our objective was to evaluate the potential effects of HIT on reducing 21 different inpatient AEs. Identifying AEs most likely to be reduced by HIT can inform the design of future studies evaluating its effectiveness.

METHODS

We conducted a modified Delphi panel of national experts in HIT and safety. We conducted a focused literature review to inform the experts. Using a novel framework, experts rated each AE as 'definitely reduced by health IT,' 'possibly reduced by health IT' and 'not likely to be reduced by health IT'.

RESULTS

From our panel discussion, experts identified six AEs as 'definitely reduced by health IT': (1) adverse drug events (ADEs) associated with digoxin; (2) ADE associated with IV heparin; (3) ADE associated with hypoglycaemic agents; (4) ADE associated with low molecular weight heparin and factor Xa inhibitor; (5) contrast nephropathy associated with catheter angiography; and (6) ADE hospital-acquired antibiotic-associated Clostridium difficile.

CONCLUSIONS

Understanding the effects of HIT on patient outcomes will be essential to ensuring that the significant federal investment results in anticipated improvements. This study serves as an important early step in helping with the design of future work evaluating level of HIT infrastructure and rates of inpatient AEs.

Short message service or disService: issues with text messaging in a complex medical environment

BACKGROUND

Hospitals today are experiencing major changes in their clinical communication workflows as conventional numeric paging and face-to-face verbal conversations are being replaced by computer mediated communication systems. In this paper, we highlight the importance of understanding this transition and discuss some of the impacts that may emerge when verbal clinical conversations are replaced by short text messages.

METHODS

In-depth interviews (n=108) and non-participatory observation sessions (n=260h) were conducted on the General Internal Medicine wards at five academic teaching hospitals in Toronto, Canada.

RESULTS

From our analysis of the qualitative data, we identified two major themes. De-contextualization of complex issues led to an increase in misinterpretation and an increase in back and forth messaging for clarification. Depersonalization of communication was due to less verbal conversations and face-to-face interactions and led to a negative impact on work relationships.

CONCLUSIONS

Text-based communication in hospital settings led to the oversimplification of messages and the depersonalization of communication. It is important to recognize and understand these unintended consequences of new technology to avoid the negative impacts to patient care and work relationships.

Clinical decision support alert appropriateness: a review and proposal for improvement

BACKGROUND

Many healthcare providers are adopting clinical decision support (CDS) systems to improve patient safety and meet meaningful use requirements. Computerized alerts that prompt clinicians about drug-allergy, drug-drug, and drug-disease warnings or provide dosing guidance are most commonly implemented. Alert overrides, which occur when clinicians do not follow the guidance presented by the alert, can hinder improved patient outcomes.

METHODS

We present a review of CDS alerts and describe a proposal to develop novel methods for evaluating and improving CDS alerts that builds upon traditional informatics approaches. Our proposal incorporates previously described models for predicting alert overrides that utilize retrospective chart review to determine which alerts are clinically relevant and which overrides are justifiable.

RESULTS

Despite increasing implementations of CDS alerts, detailed evaluations rarely occur because of the extensive labor involved in manual chart reviews to determine alert and response appropriateness. Further, most studies have solely evaluated alert overrides that are appropriate or justifiable. Our proposal expands the use of web-based monitoring tools with an interactive dashboard for evaluating CDS alert and response appropriateness that incorporates the predictive models. The dashboard provides 2 views, an alert detail view and a patient detail view, to provide a full history of alerts and help put the patient's events in context.

CONCLUSION

The proposed research introduces several innovations to address the challenges and gaps in alert evaluations. This research can transform alert evaluation processes across healthcare settings, leading to improved CDS, reduced alert fatigue, and increased patient safety.

Health information technology and hospital patient safety: a conceptual model to guide research

BACKGROUND

The literature indicates that health information technology (IT) use may lead to some gains in the quality and safety of care in some situations but provides little insight into this variability in the results that has been found. The inconsistent findings point to the need for a conceptual model that will guide research in sorting out the complex relationships between health IT and the quality and safety of care.

METHODS

A conceptual model was developed that describes how specific health IT functions could affect different types of inpatient safety errors and that include contextual factors that influence successful health IT implementation. The model was applied to a readily available patient safety measure and nationwide data (2009 AHA Annual Survey Information Technology Supplement and 2009 Healthcare Cost and Utilization Project State Inpatient Databases).

FINDINGS

The model was difficult to operationalize because (1) available health IT adoption data did not characterize health IT features and extent of usage, and (2) patient safety measures did not elucidate the process failures leading to safety-related outcomes. The sample patient safety measure--Postoperative Physiologic and Metabolic Derangement Rate--was not significantly related to self-reported health IT capabilities when adjusted for hospital structural characteristics.

CONCLUSION

These findings illustrate the critical need for collecting data that are germane to health IT and the possible mechanisms by which health IT may affect inpatient safety. Well-defined and sufficiently granular measures of provider's correct use of health IT functions, the contextual factors surrounding health IT use, and patient safety errors leading to health care-associated conditions are needed to illuminate the impact of health IT on patient safety.

Recent trends in biomedical informatics: a study based on JAMIA articles

In a growing interdisciplinary field like biomedical informatics, information dissemination and citation trends are changing rapidly due to many factors. To understand these factors better, we analyzed the evolution of the number of articles per major biomedical informatics topic, download/online view frequencies, and citation patterns (using Web of Science) for articles published from 2009 to 2012 in JAMIA. The number of articles published in JAMIA increased significantly from 2009 to 2012, and there were some topic differences in the last 4 years. Medical Record Systems, Algorithms, and Methods are topic categories that are growing fast in several publications. We observed a significant correlation between download frequencies and the number of citations per month since publication for a given article. Earlier free availability of articles to non-subscribers was associated with a higher number of downloads and showed a trend towards a higher number of citations. This trend will need to be verified as more data accumulate in coming years.

Diagnostic Reasoning and Decision Making in the Context of Health Information Technology

Diagnostic reasoning and medical decision making have been focal areas of research in the fields of medical education, cognition, and artificial intelligence in medicine. Drawing on several decades worth of research, we propose an integrated summary of prior research on diagnostic reasoning and decision making—in terms of both historical development and theoretical shifts. We also characterize the changes in research and theory resulting from the incorporation and adoption of health information technology in the clinical work place. In this paper, we differentiate between the various forms of diagnostic reasoning and trace the evolution of the various models of reasoning, including knowledge-based, exemplar-based, and visual strategies. We also discuss the effect of clinical expertise on reasoning processes. Within the medical decision-making research, we delineate the various approaches highlighting decision-making errors that arise due to the nature of heuristics and biases and other factors. Although there has been significant progress in our understanding, there is still a need for greater theoretical integration of disparate empirical phenomena. Specifically, there is a need to reconcile the various characterizations of reasoning and to evaluate the similarity and differences in the context of current health care practice. Finally, we discuss the role of human factors research in the study of clinical environments and also in relation to devising approaches and methodologies for understanding, evaluating, and supporting the diagnostic reasoning and decision processes.

Using statistical text classification to identify health information technology incidents

OBJECTIVE

To examine the feasibility of using statistical text classification to automatically identify health information technology (HIT) incidents in the USA Food and Drug Administration (FDA) Manufacturer and User Facility Device Experience (MAUDE) database.

DESIGN

We used a subset of 570 272 incidents including 1534 HIT incidents reported to MAUDE between 1 January 2008 and 1 July 2010. Text classifiers using regularized logistic regression were evaluated with both 'balanced' (50% HIT) and 'stratified' (0.297% HIT) datasets for training, validation, and testing. Dataset preparation, feature extraction, feature selection, cross-validation, classification, performance evaluation, and error analysis were performed iteratively to further improve the classifiers. Feature-selection techniques such as removing short words and stop words, stemming, lemmatization, and principal component analysis were examined.

MEASUREMENTS

κ statistic, F1 score, precision and recall.

RESULTS

Classification performance was similar on both the stratified (0.954 F1 score) and balanced (0.995 F1 score) datasets. Stemming was the most effective technique, reducing the feature set size to 79% while maintaining comparable performance. Training with balanced datasets improved recall (0.989) but reduced precision (0.165).

CONCLUSIONS

Statistical text classification appears to be a feasible method for identifying HIT reports within large databases of incidents. Automated identification should enable more HIT problems to be detected, analyzed, and addressed in a timely manner. Semi-supervised learning may be necessary when applying machine learning to big data analysis of patient safety incidents and requires further investigation.

STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics: explanation and elaboration

BACKGROUND

Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI - Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report.

OBJECTIVES

To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting.

METHODS

A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature.

RESULTS

The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations.

CONCLUSION

The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein.

LIMITATIONS

This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice.

Educational impact of using smartphones for clinical communication on general medicine: more global, less local

BACKGROUND

Medical trainees increasingly use smartphones in their clinical work. Similar to other information technology implementations, smartphone use can result in unintended consequences. This study aimed to examine the impact of smartphone use for clinical communication on medical trainees' educational experiences.

DESIGN

Qualitative research methodology using interview data, ethnographic data, and analysis of e-mail messages.

ANALYSIS

We analyzed the interview transcripts, ethnographic data, and e-mails by applying a conceptual framework consisting of 5 educational domains.

RESULTS

Smartphone use increased connectedness and resulted in a high level of interruptions. These 2 factors impacted 3 discrete educational domains: supervision, teaching, and professionalism. Smartphone use increased connectedness to supervisors and may improve supervision, making it easier for supervisors to take over but can limit autonomy by reducing learner decision making. Teaching activities may be easier to coordinate, but smartphone use interrupted learners and reduced teaching effectiveness during these sessions. Finally, there may be professionalism issues in relation to how residents use smartphones during encounters with patients and health professionals and in teaching sessions.

CONCLUSIONS

We summarized the impact of a rapidly emerging information technology-smartphones-on the educational experience of medical trainees. Smartphone use increase connectedness and allow trainees to be more globally available for patient care but creates interruptions that cause trainees to be less present in their local interactions with staff during teaching sessions. Educators should be aware of these findings and need to develop curriculum to address the negative impacts of smartphone use in the clinical training environment.

The hazard of software updates to clinical workstations: a natural experiment

Emergency department (ED) electronic tracking boards provide a snapshot view of patient status and a quick link to other clinical applications, such as a web-based image viewer client to view current and previous radiology images from the picture archiving and communication systems (PACS). We describe a case where an update to Microsoft Internet Explorer severed the link between the ED tracking board and web-based image viewer. The loss of this link resulted in decreased web-based image viewer access rates for ED patients during the 10 days of the incident (2.8 views/study) compared with image review rates for a similar 10-day period preceding this event (3.8 views/study, p<0.001). Single-click user interfaces that transfer user and patient contexts are efficient mechanisms to link disparate clinical systems. Maintaining hazard analyses and rigorously testing all software updates to clinical workstations, including seemingly minor web-browser updates, are important to minimize the risk of unintended consequences.

Paper- and computer-based workarounds to electronic health record use at three benchmark institutions

BACKGROUND

Healthcare professionals develop workarounds rather than using electronic health record (EHR) systems. Understanding the reasons for workarounds is important to facilitate user-centered design and alignment between work context and available health information technology tools.

OBJECTIVE

To examine both paper- and computer-based workarounds to the use of EHR systems in three benchmark institutions.

METHODS

Qualitative data were collected in 11 primary care outpatient clinics across three healthcare institutions. Data collection methods included direct observation and opportunistic questions. In total, 120 clinic staff and providers and 118 patients were observed. All data were analyzed using previously developed workaround categories and examined for potential new categories. Additionally, workarounds were coded as either paper- or computer-based.

RESULTS

Findings corresponded to 10 of 11 workaround categories identified in previous research. All 10 of these categories applied to paper-based workarounds; five categories also applied to computer-based workarounds. One new category, no correct path (eg, a desired option did not exist in the computer interface, precipitating a workaround), was identified for computer-based workarounds. The most consistent reasons for workarounds across the three institutions were efficiency, memory, and awareness.

CONCLUSIONS

Consistent workarounds across institutions suggest common challenges in outpatient clinical settings and failures to accommodate these challenges in EHR design. An examination of workarounds provides insight into how providers adapt to limiting EHR systems. Part of the design process for computer interfaces should include user-centered methods particular to providers and healthcare settings to ensure uptake and usability.

Publication bias in clinical trials of electronic health records

OBJECTIVE

To measure the rate of non-publication and assess possible publication bias in clinical trials of electronic health records.

METHODS

We searched ClinicalTrials.gov to identify registered clinical trials of electronic health records and searched the biomedical literature and contacted trial investigators to determine whether the results of the trials were published. Publications were judged as positive, negative, or neutral according to the primary outcome.

RESULTS

Seventy-six percent of trials had publications describing trial results; of these, 74% were positive, 21% were neutral, and 4% were negative (harmful). Of unpublished studies for which the investigator responded, 43% were positive, 57% were neutral, and none were negative; the lower rate of positive results was significant (p<0.001).

CONCLUSION

The rate of non-publication in electronic health record studies is similar to that in other biomedical studies. There appears to be a bias toward publication of positive trials in this domain.

Use of simulation to assess electronic health record safety in the intensive care unit: a pilot study

OBJECTIVE

To establish the role of high-fidelity simulation training to test the efficacy and safety of the electronic health record (EHR)-user interface within the intensive care unit (ICU) environment.

DESIGN

Prospective pilot study.

SETTING

Medical ICU in an academic medical centre.

PARTICIPANTS

Postgraduate medical trainees.

INTERVENTIONS

A 5-day-simulated ICU patient was developed in the EHR including labs, hourly vitals, medication administration, ventilator settings, nursing and notes. Fourteen medical issues requiring recognition and subsequent changes in management were included. Issues were chosen based on their frequency of occurrence within the ICU and their ability to test different aspects of the EHR-user interface. ICU residents, blinded to the presence of medical errors within the case, were provided a sign-out and given 10 min to review the case in the EHR. They then presented the case with their management suggestions to an attending physician. Participants were graded on the number of issues identified. All participants were provided with immediate feedback upon completion of the simulation.

PRIMARY AND SECONDARY OUTCOMES

To determine the frequency of error recognition in an EHR simulation. To determine factors associated with improved performance in the simulation.

RESULTS

38 participants including 9 interns, 10 residents and 19 fellows were tested. The average error recognition rate was 41% (range 6-73%), which increased slightly with the level of training (35%, 41% and 50% for interns, residents, and fellows, respectively). Over-sedation was the least-recognised error (16%); poor glycemic control was most often recognised (68%). Only 32% of the participants recognised inappropriate antibiotic dosing. Performance correlated with the total number of screens used (p=0.03).

CONCLUSIONS

Despite development of comprehensive EHRs, there remain significant gaps in identifying dangerous medical management issues. This gap remains despite high levels of medical training, suggesting that EHR-specific training may be beneficial. Simulation provides a novel tool in order to both identify these gaps as well as foster EHR-specific training.

An application for monitoring order set usage in a commercial electronic health record

Organizations that use electronic health records (EHRs) often maintain a considerable amount of clinical content in the form of order sets, documentation templates, and decision support rules. EHR vendors seldom provide analytic tools for customers to maintain such content and monitor its usage. We developed an application for tracking order sets, documentation templates and clinical alerts in a commercial electronic health record. Using the application, we compared trends in order set creation and usage at two academic medical centers over a three-year period. In January 2012, one medical center had 873 order sets available to clinicians; the other had 787. Approximately 50-75 new order sets were added each year at each medical center. We found that 46% of order sets at the first medical center and 39% at the second medical center were unused over the three-year period.

The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting

Much of what is currently documented in the electronic health record is in response toincreasingly complex and prescriptive medicolegal, reimbursement, and regulatory requirements. These requirements often result in redundant data capture and cumbersome documentation processes. AMIA's 2011 Health Policy Meeting examined key issues in this arena and envisioned changes to help move toward an ideal future state of clinical data capture and documentation. The consensus of the meeting was that, in the move to a technology-enabled healthcare environment, the main purpose of documentation should be to support patient care and improved outcomes for individuals and populations and that documentation for other purposes should be generated as a byproduct of care delivery. This paper summarizes meeting deliberations, and highlights policy recommendations and research priorities. The authors recommend development of a national strategy to review and amend public policies to better support technology-enabled data capture and documentation practices.

Modeling nurses' acceptance of bar coded medication administration technology at a pediatric hospital

OBJECTIVE

To identify predictors of nurses' acceptance of bar coded medication administration (BCMA).

DESIGN

Cross-sectional survey of registered nurses (N=83) at an academic pediatric hospital that recently implemented BCMA.

METHODS

Surveys assessed seven BCMA-related perceptions: ease of use; usefulness for the job; social influence from non-specific others to use BCMA; training; technical support; usefulness for patient care; and social influence from patients/families. An all possible subset regression procedure with five goodness-of-fit indicators was used to identify which set of perceptions best predicted BCMA acceptance (intention to use, satisfaction).

RESULTS

Nurses reported a moderate perceived ease of use and low perceived usefulness of BCMA. Nurses perceived moderate-or-higher social influence to use BCMA and had moderately positive perceptions of BCMA-related training and technical support. Behavioral intention to use BCMA was high, but satisfaction was low. Behavioral intention to use was best predicted by perceived ease of use, perceived social influence from non-specific others, and perceived usefulness for patient care (56% of variance explained). Satisfaction was best predicted by perceived ease of use, perceived usefulness for patient care, and perceived social influence from patients/families (76% of variance explained).

DISCUSSION

Variation in and low scores on ease of use and usefulness are concerning, especially as these variables often correlate with acceptance, as found in this study. Predicting acceptance benefited from using a broad set of perceptions and adapting variables to the healthcare context.

CONCLUSION

Success with BCMA and other technologies can benefit from assessing end-user acceptance and elucidating the factors promoting acceptance and use.

Biomedical informatics--a confluence of disciplines?

BACKGROUND

Biomedical informatics is a broad discipline that borrows many methods and techniques from other disciplines.

OBJECTIVE

To reflect a) on the character of biomedical informatics and to determine whether it is multi-disciplinary or inter-disciplinary; b) on the question whether biomedical informatics is more than the sum of its supporting disciplines and c) on the position of biomedical informatics with respect to related disciplines.

METHOD

Inviting an international group of experts in biomedical informatics and related disciplines on the occasion of the 50th anniversary of Methods of Information in Medicine to present their viewpoints.

RESULTS AND CONCLUSIONS

This paper contains the reflections of a number of the invited experts on the character of biomedical informatics. Most of the authors agree that biomedical informatics is an interdisciplinary field of study where researchers with different scientific backgrounds alone or in combination carry out research. Biomedical informatics is a very broad scientific field and still expanding, yet comprised of a constructive aspect (designing and building systems). One author expressed that the essence of biomedical informatics, as opposed to related disciplines, lies in the modelling of the biomedical content. Interdisciplinarity also has consequences for education. Maintaining rigid disciplinary structures does not allow for sufficient adaptability to capitalize on important trends nor to leverage the influences these trends may have on biomedical informatics. It is therefore important for students to become aware of research findings in related disciplines. In this respect, it was also noted that the fact that many scientific fields use different languages and that the research findings are stored in separate bibliographic databases makes it possible that potentially connected findings will never be linked, despite the fact that these findings were published. Bridges between the sciences are needed for the success of biomedical informatics.

New paradigms for measuring clinical performance using electronic health records

Measures of provider success are the centerpiece of quality improvement and pay-for-performance programs around the globe. In most nations, these measures are derived from administrative records, paper charts and consumer surveys; increasingly, electronic patient record systems are also being used. We use the term 'e-QMs' to describe quality measures that are based on data found within electronic health records and other related health information technology (HIT). We offer a framework or typology for e-QMs and describe opportunities and impediments associated with the transition from old to new

DATA SOURCES

If public and private systems of care are to effectively use HIT to support and evaluate health-care system quality and safety, the quality measurement field must embrace new paradigms and strategically address a series of technical, conceptual and practical challenges.

Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure

OBJECTIVES

Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics.

MATERIALS AND METHODS

We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements.

RESULTS

System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%).

DISCUSSION

An EF value of <40% can be accurately identified in VA echocardiogram reports.

CONCLUSIONS

An automated information extraction system can be used to accurately extract EF for quality measurement.

Impact of electronic health record transition on behavioral health screening in a large pediatric practice

OBJECTIVE

The objective was to determine whether transitioning from paper to electronic health records affected behavioral health screening rates in a large Northeastern pediatric practice.

METHODS

The study setting was a pediatric practice with seven pediatricians, serving about 6,000 patients. The patient population was diverse (54% nonwhite, 40% publicly insured or self-paying, and 31% non-English speakers). An interrupted times series design was used to evaluate the impact of electronic record implementation on behavioral health screening rates. The main outcome measure was the rate of such screening 18 months before and 36 months after implementation.

RESULTS

The rate of behavioral health screening increased from 70% to 91% during the baseline period. The training period-six months before electronic record implementation-was associated with a 28% decline in adjusted screening rates (from 83.3% to 55.5%). Only 50% of eligible youths were screened in the first month after implementation. The screening rate took more than three years to recover to baseline levels, climbing to 82% by April 2008.

CONCLUSIONS

Practice changes resulting from electronic record adoption were highly disruptive of care, and disruptions took several years to resolve completely. When medical assistants rather than physicians were tasked with transferring data from paper screening forms to the electronic record, reporting compliance improved. Compliance with Healthcare Effectiveness Data and Information Set standards and Medicaid performance measures will likely be similarly affected as electronic records are implemented nationwide. Although implementing a fully automated medical record has some benefits, the unintended effects on care after implementation must be acknowledged.

The dangerous decade

Over the next 10 years, more information and communication technology (ICT) will be deployed in the health system than in its entire previous history. Systems will be larger in scope, more complex, and move from regional to national and supranational scale. Yet we are at roughly the same place the aviation industry was in the 1950s with respect to system safety. Even if ICT harm rates do not increase, increased ICT use will increase the absolute number of ICT related harms. Factors that could diminish ICT harm include adoption of common standards, technology maturity, better system development, testing, implementation and end user training. Factors that will increase harm rates include complexity and heterogeneity of systems and their interfaces, rapid implementation and poor training of users. Mitigating these harms will not be easy, as organizational inertia is likely to generate a hysteresis-like lag, where the paths to increase and decrease harm are not identical.

The future of health IT innovation and informatics: a report from AMIA's 2010 policy meeting

While much attention has been paid to the short-term impact that widespread adoption of health information technology (health IT) will have on the healthcare system, there is a corresponding need to look at the long-term effects that extant policies may have on health IT system resilience, innovation, and related ethical, social/legal issues. The American Medical Informatics Association's 2010 Health Policy Conference was convened to further the national discourse on the issues surrounding these longer-term considerations. Conference participants self-selected into three broad categories: resilience in healthcare and health IT; ethical, legal, and social challenges; and innovation, adoption, and sustainability. The discussions about problem areas lead to findings focusing on the lack of encouragement for long-term IT innovation that may result from current health IT policies; the potential impact of uneven adoption of health IT based on the exclusions of the current financial incentives; the weaknesses of contingency and risk mitigation planning that threaten system resilience; and evolving standards developed in response to challenges relating to the security, integrity, and availability of electronic health information. This paper discusses these findings and also offers recommendations that address the interwoven topics of innovation, resilience, and adoption. The goal of this paper is to encourage public and private sector organizations that have a role in shaping health information policy to increase attention to developing a national strategy that assures that health IT innovation and resilience are not impeded by shorter-term efforts to implement current approaches emphasizing adoption and meaningful use of electronic health records.

Using FDA reports to inform a classification for health information technology safety problems

OBJECTIVE

To expand an emerging classification for problems with health information technology (HIT) using reports submitted to the US Food and Drug Administration Manufacturer and User Facility Device Experience (MAUDE) database.

DESIGN

HIT events submitted to MAUDE were retrieved using a standardized search strategy. Using an emerging classification with 32 categories of HIT problems, a subset of relevant events were iteratively analyzed to identify new categories. Two coders then independently classified the remaining events into one or more categories. Free-text descriptions were analyzed to identify the consequences of events.

MEASUREMENTS

Descriptive statistics by number of reported problems per category and by consequence; inter-rater reliability analysis using the κ statistic for the major categories and consequences.

RESULTS

A search of 899 768 reports from January 2008 to July 2010 yielded 1100 reports about HIT. After removing duplicate and unrelated reports, 678 reports describing 436 events remained. The authors identified four new categories to describe problems with software functionality, system configuration, interface with devices, and network configuration; the authors' classification with 32 categories of HIT problems was expanded by the addition of these four categories. Examination of the 436 events revealed 712 problems, 96% were machine-related, and 4% were problems at the human-computer interface. Almost half (46%) of the events related to hazardous circumstances. Of the 46 events (11%) associated with patient harm, four deaths were linked to HIT problems (0.9% of 436 events).

CONCLUSIONS

Only 0.1% of the MAUDE reports searched were related to HIT. Nevertheless, Food and Drug Administration reports did prove to be a useful new source of information about the nature of software problems and their safety implications with potential to inform strategies for safe design and implementation.