Classification of Electronic Health Record-Related Patient Safety Incidents: Development and Validation Study

Taxonomy development methods regarding patient safety in health sciences - A systematic review

BACKGROUND

Taxonomies are needed for automated analysis of clinical data in healthcare. Few reviews of the taxonomy development methods used in health sciences are found. This systematic review aimed to describe the scope of the available taxonomies relative to patient safety, the methods used for taxonomy development, and the strengths and limitations of the methods. The purpose of this systematic review is to guide future taxonomy development projects.

METHODS

The CINAHL, PubMed, Scopus, and Web of Science databases were searched for studies from January 2012 to April 25, 2023. Two authors selected the studies using inclusion and exclusion criteria and critical appraisal checklists. The data were analysed inductively, and the results were reported narratively.

RESULTS

The studies (n = 13) across healthcare concerned mainly taxonomies of adverse events and medication safety but little for specialised fields and information technology. Critical appraisal indicated inadequate reporting of the used taxonomy development methods. Ten phases of taxonomy development were identified: (1) defining purpose and (2) the theory base for development, (3) relevant data sources' identification, (4) main terms' identification and definitions, (5) items' coding and pooling, (6) reliability and validity evaluation of coding and/or codes, (7) development of a hierarchical structure, (8) testing the structure, (9) piloting the taxonomy and (10) reporting application and validation of the final taxonomy. Seventeen statistical tests and seven software systems were utilised, but automated data extraction methods were used rarely. Multimethod and multi-stakeholder approach, code- and hierarchy testing and piloting were strengths and time consumption and small samples in testing limitations.

CONCLUSION

New taxonomies are needed on diverse specialities and information technology related to patient safety. Structured method is needed for taxonomy development, reporting and appraisal to strengthen taxonomies' quality. A new guide was proposed for taxonomy development, for which testing is required. Prospero registration number CRD42023411022.

Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis

BACKGROUND

Burnout among health care professionals is a significant concern, with detrimental effects on health care service quality and patient outcomes. The use of the electronic health record (EHR) system has been identified as a significant contributor to burnout among health care professionals.

OBJECTIVE

This systematic review and meta-analysis aims to assess the prevalence of burnout among health care professionals associated with the use of the EHR system, thereby providing evidence to improve health information systems and develop strategies to measure and mitigate burnout.

METHODS

We conducted a comprehensive search of the PubMed, Embase, and Web of Science databases for English-language peer-reviewed articles published between January 1, 2009, and December 31, 2022. Two independent reviewers applied inclusion and exclusion criteria, and study quality was assessed using the Joanna Briggs Institute checklist and the Newcastle-Ottawa Scale. Meta-analyses were performed using R (version 4.1.3; R Foundation for Statistical Computing), with EndNote X7 (Clarivate) for reference management.

RESULTS

The review included 32 cross-sectional studies and 5 case-control studies with a total of 66,556 participants, mainly physicians and registered nurses. The pooled prevalence of burnout among health care professionals in cross-sectional studies was 40.4% (95% CI 37.5%-43.2%). Case-control studies indicated a higher likelihood of burnout among health care professionals who spent more time on EHR-related tasks outside work (odds ratio 2.43, 95% CI 2.31-2.57).

CONCLUSIONS

The findings highlight the association between the increased use of the EHR system and burnout among health care professionals. Potential solutions include optimizing EHR systems, implementing automated dictation or note-taking, employing scribes to reduce documentation burden, and leveraging artificial intelligence to enhance EHR system efficiency and reduce the risk of burnout.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42021281173; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021281173.

An Analysis of Incident Reports Related to Electronic Medication Management: How They Change Over Time

OBJECTIVE

Electronic medication management (EMM) systems have been shown to introduce new patient safety risks that were not possible, or unlikely to occur, with the use of paper charts. Our aim was to examine the factors that contribute to EMM-related incidents and how these incidents change over time with ongoing EMM use.

METHODS

Incidents reported at 3 hospitals between January 1, 2010, and December 31, 2019, were extracted using a keyword search and then screened to identify EMM-related reports. Data contained in EMM-related incident reports were then classified as unsafe acts made by users and the latent conditions contributing to each incident.

RESULTS

In our sample, 444 incident reports were determined to be EMM related. Commission errors were the most frequent unsafe act reported by users (n = 298), whereas workarounds were reported in only 13 reports. User latent conditions (n = 207) were described in the highest number of incident reports, followed by conditions related to the organization (n = 200) and EMM design (n = 184). Over time, user unfamiliarity with the system remained a key contributor to reported incidents. Although fewer articles to electronic transfer errors were reported over time, incident reports related to the transfer of information between different computerized systems increased as hospitals adopted more clinical information systems.

CONCLUSIONS

Electronic medication management-related incidents continue to occur years after EMM implementation and are driven by design, user, and organizational conditions. Although factors contribute to reported incidents in varying degrees over time, some factors are persistent and highlight the importance of continuously improving the EMM system and its use.

Clinical Informatics Team Members' Perspectives on Health Information Technology Safety After Experiential Learning and Safety Process Development: Qualitative Descriptive Study

BACKGROUND

Although intended to support improvement, the rapid adoption and evolution of technologies in health care can also bring about unintended consequences related to safety. In this project, an embedded researcher with expertise in patient safety and clinical education worked with a clinical informatics team to examine safety and harm related to health information technologies (HITs) in primary and community care settings. The clinical informatics team participated in learning activities around relevant topics (eg, human factors, high reliability organizations, and sociotechnical systems) and cocreated a process to address safety events related to technology (ie, safety huddles and sociotechnical analysis of safety events).

OBJECTIVE

This study aimed to explore clinical informaticians' experiences of incorporating safety practices into their work.

METHODS

We used a qualitative descriptive design and conducted web-based focus groups with clinical informaticians. Thematic analysis was used to analyze the data.

RESULTS

A total of 10 informants participated. Barriers to addressing safety and harm in their context included limited prior knowledge of HIT safety, previous assumptions and perspectives, competing priorities and organizational barriers, difficulty with the reporting system and processes, and a limited number of reports for learning. Enablers to promoting safety and mitigating harm included participating in learning sessions, gaining experience analyzing reported events, participating in safety huddles, and role modeling and leadership from the embedded researcher. Individual outcomes included increased ownership and interest in HIT safety, the development of a sociotechnical systems perspective, thinking differently about safety, and increased consideration for user perspectives. Team outcomes included enhanced communication within the team, using safety events to inform future work and strategic planning, and an overall promotion of a culture of safety.

CONCLUSIONS

As HITs are integrated into care delivery, it is important for clinical informaticians to recognize the risks related to safety. Experiential learning activities, including reviewing safety event reports and participating in safety huddles, were identified as particularly impactful. An HIT safety learning initiative is a feasible approach for clinical informaticians to become more knowledgeable and engaged in HIT safety issues in their work.

A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study

BACKGROUND

Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration.

OBJECTIVE

This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.

METHODS

This study used a data set of 861 PSE reports from a large academic hospital's maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier's predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems.

RESULTS

The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95/126) compared to an accuracy of 66.7% (84/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight.

CONCLUSIONS

This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.

Development and Validation of SafeHIT: An Instrument to Assess the Self-Reported Safe Use of Health Information Technology

BACKGROUND

Implementing health information technology (HIT) may cause unintended consequences and safety risks when incorrectly designed and used. Yet, the tools to assess self-reported safe use of HIT are not well established.

OBJECTIVE

This study aims to develop and validate SafeHIT, an instrument to assess self-reported safe use of HIT among health care practitioners.

METHODS

Systematic literature review and a semistructured interview with 31 experts were adopted to generate SafeHIT instrument items. In total, 450 physicians from various departments at three Malaysian public hospitals participated in the questionnaire survey to validate SafeHIT. Exploratory factor analysis and confirmatory factor analysis (CFA) were undertaken to explore the items that best represent a specific construct and to confirm the reliability and validity of the SafeHIT, respectively.

RESULTS

The final SafeHIT consisted of 14 constructs and 58 items in total. The result of the CFA confirmed that all constructs demonstrated adequate convergent and discriminant validity.

CONCLUSION

A reliable and valid theoretically underpinned measure of determinants of safe HIT use behavior has been developed. Understanding external factors that influence safe HIT use is useful for developing targeted interventions that favor the quality and safety of health care.

Interorganizational health information exchange-related patient safety incidents: A descriptive register-based qualitative study

PURPOSE

The current literature related to patient safety of interorganizational health information is fragmented. This study aims to identify interorganizational health information exchange-related patient safety incidents occurring in the emergency department, emergency medical services, and home care. The research also aimed to describe the causes and consequences of these incidents.

METHODS

A total of sixty (n = 60) interorganizational health information exchange-related patient safety incident free text reports were analyzed. The reports were reported in the emergency department, emergency medical services, or home care between January 2016 and December 2019 in one hospital district in Finland.

RESULTS

The identified interorganizational health information exchange-related incidents were grouped under two main categories: "Inadequate documentation"; and "Inadequate use of information". The causes of these incidents were grouped under the two main categories "Factors related to the healthcare professional " and "Organizational factors", while the consequences of these incidents fell under the two main categories "Adverse events" and "Additional actions to prevent, avoid, and correct adverse events".

CONCLUSION

This study shows that the inadequate documentation and use of information is mainly caused by factors related to the healthcare professional and organization, including technical problems. These incidents cause adverse events and additional actions to prevent, avoid, and correct the events. The sociotechnical perspective, including factors related to health care professionals, organization, and technology, should be emphasized in patient safety development of inter-organizational health information exchange and it will be the focus of our future research. Continuous research and development work is needed because the processes and information systems used in health care are constantly evolving.

Preliminary Exploration of Main Elements for Systematic Classification Development: Case Study of Patient Safety Incidents (Preprint)

Background

Currently, there is no holistic theoretical approach available for guiding classification development. On the basis of our recent classification development research in the area of patient safety in health information technology, this focus area would benefit from a more systematic approach. Although some valuable theoretical and methodological approaches have been presented, classification development literature typically is limited to methodological development in a specific domain or is practically oriented.

Objective

The main purposes of this study are to fill the methodological gap in classification development research by exploring possible elements of systematic development based on previous literature and to promote sustainable and well-grounded classification outcomes by identifying a set of recommended elements. Specifically, the aim is to answer the following question: what are the main elements for systematic classification development based on research evidence and our use case?

Methods

This study applied a qualitative research approach. On the basis of previous literature, preliminary elements for classification development were specified, as follows: defining a concept model, documenting the development process, incorporating multidisciplinary expertise, validating results, and maintaining the classification. The elements were compiled as guiding principles for the research process and tested in the case of patient safety incidents (n=501).

Results

The results illustrate classification development based on the chosen elements, with 4 examples of technology-induced errors. Examples from the use case regard usability, system downtime, clinical workflow, and medication section problems. The study results confirm and thus suggest that a more comprehensive and theory-based systematic approach promotes well-grounded classification work by enhancing transparency and possibilities for assessing the development process.

Conclusions

We recommend further testing the preliminary main elements presented in this study. The research presented herein could serve as a basis for future work. Our recently developed classification and the use case presented here serve as examples. Data retrieved from, for example, other type of electronic health records and use contexts could refine and validate the suggested methodological approach.