Identifying barriers and benefits of patient safety event reporting toward user-centered design

A large dataset of annotated incident reports on medication errors

Incident reports of medication errors are valuable learning resources for improving patient safety. However, pertinent information is often contained within unstructured free text, which prevents automated analysis and limits the usefulness of these data. Natural language processing can structure this free text automatically and retrieve relevant past incidents and learning materials, but to be able to do so requires a large, fully annotated and validated corpus of incident reports. We present a corpus of 58,658 machine-annotated incident reports of medication errors that can be used to advance the development of information extraction models and subsequent incident learning. We report the best F1-scores for the annotated dataset: 0.97 and 0.76 for named entity recognition and intention/factuality analysis, respectively, for the cross-validation exercise. Our dataset contains 478,175 named entities and differentiates between incident types by recognising discrepancies between what was intended and what actually occurred. We explain our annotation workflow and technical validation and provide access to the validation datasets and machine annotator for labelling future incident reports of medication errors.

A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study

BACKGROUND

Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration.

OBJECTIVE

This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.

METHODS

This study used a data set of 861 PSE reports from a large academic hospital's maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier's predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems.

RESULTS

The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95/126) compared to an accuracy of 66.7% (84/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight.

CONCLUSIONS

This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.

Investigating Racial and Ethnic Disparities in Maternal Care at the System Level Using Patient Safety Incident Reports

BACKGROUND

Maternal mortality in the United States is high, and women and birthing people of color experience higher rates of mortality and severe maternal morbidity (SMM). More than half of maternal deaths and cases of SMM are considered preventable. The research presented here investigated systems issues contributing to adverse outcomes and racial/ethnic disparities in maternal care using patient safety incident reports.

METHODS

The authors reviewed incidents reported in the labor and delivery unit (L&D) and the antepartum and postpartum unit (A&P) of a large academic hospital in 2019 and 2020. Deliveries associated with a reported incident were described by race/ethnicity, age group, method of delivery, and several other process variables. Differences across racial/ethnic group were statistically evaluated.

RESULTS

Almost two thirds (64.8%) of the 528 reports analyzed were reported in L&D, and 35.2% were reported in A&P. Non-Hispanic white (NHW) patients accounted for 43.9% of reported incidents, non-Hispanic Black (NHB) patients accounted for 43.2%, Hispanic patients accounted for 8.9%, and patients categorized as "other" accounted for 4.0%. NHB patients were disproportionally represented in the incident reports, as they accounted for only 36.5% of the underlying birthing population. The odds ratio (OR) demonstrated a higher risk of a reported adverse incident for NHB patients; however, adjustment for cesarean section attenuated the association (OR 1.25, 95% confidence interval 1.01-1.54).

CONCLUSION

Greater integration of patient safety and health equity efforts in hospitals are needed to promptly identify and alleviate racial and ethnic disparities in maternal health outcomes. Although additional systems analysis is necessary, the authors offer recommendations to support safer, more equitable maternal care.

Accident & Emergency Informatics and One Digital Health

OBJECTIVES

Climate changes are the major challenge in public and individual health, as they modify the ecosystem and yield contagious diseases from animal to human. Furthermore, we notice the rapid development of elderly, changing the population demographic. These critical measures have imposed economical costs, require trained personnel, and reduce the healthcare systems' performances.

METHODS

COVID-19 pandemic showed that digital health paradigms such as m-health, telemedicine, and Internet of medical things (IoMT) should be further developed for such disasters. Quarantine was experienced frequently at different levels, which indicates the urgent need to develop smart medical homes for continuous monitoring of the patients. Human health, environment, and animals are the three interwoven aspects of public health that should be formulated under a conceptual and unified framework. Accident and Emergency Informatics (A&EI) considers the prediction and prevention of an individual's health in the long term and detects instant accidents and emergencies for further processes linking to hospital and rescue services for lowering the impact. One Digital Health (ODH) considers the health of the human, the animal, and the environment as a whole.

RESULTS & CONCLUSION

In this position paper, we discuss the mutual benefits of A&EI and ODH in disaster management. We outline the mission, current status of A&EI in healthcare, and summarize the most important development of A&EI-related scope in the other fields of science. We discuss developing smart environments to monitor environmental and animal aspects. Then we examine the use of the ODH framework for enhancing the A&EI capacities to deal with complex disasters. Moreover, we discuss the further development of the international standard accident number (ISAN) to include and link environmental and animal event related data. Besides, ODH will cope with the A&EI protocols and technical specifications to be part of A&EI in the application layer.

How do healthcare practitioners use incident data to improve patient safety in Japan? A qualitative study

BACKGROUND

Patient incident reporting systems have been widely used for ensuring safety and improving quality in care settings in many countries. However, little is known about the way in which incident data are used by frontline clinical staff. Furthermore, while the use of a systems perspective has been reported as an effective way of learning from incident data in a multidisciplinary team, the level of adaptability of this perspective to a different cultural context has not been widely explored. The primary aim of the study, therefore, was to investigate how healthcare practitioners in Japan perceive the reporting systems and utilize a systems perspective in learning from incident data in acute care and mental health settings.

METHODS

A non-experimental, descriptive and exploratory research design was adopted with the following two data-collection methods: 1) Sixty-one semi-structured interviews with frontline staff in two hospitals; and 2) Non-participatory observations of thirty-seven regular incident review meetings. The two hospitals in the Greater Tokyo area which were invited to take part were: 1) a not-for-profit, privately-run, acute care hospital with approximately 500 beds; and 2) a publicly-run mental health hospital with 200 beds.

RESULTS

While the majority of staff acknowledge the positive impacts of the reporting systems on safety, the observation data found that little consideration was given to systems aspects during formal meetings. The meetings were primarily a place for the exchange of practical information, as opposed to in-depth discussions regarding causes of incidents and corrective measures. Learning from incident data was influenced by four factors: professional boundaries; dealing with a psychological burden; leadership and educational approach; and compatibility of patient safety with patient-centered care.

CONCLUSIONS

Healthcare organizations are highly complex, comprising of many professional boundaries and risk perceptions, and various communication styles. In order to establish an optimum method of individual and organizational learning and effective safety management, a fine balance has to be struck between respect for professional expertise in a local team and centralized safety oversight with a strong focus on systems. Further research needs to examine culturally-sensitive organizational and professional dynamics, including leader-follower relationships and the impact of resource constraints.

Addressing Mistreatment of Providers by Patients and Family Members as a Patient Safety Event

OBJECTIVE

Mistreatment of health care providers (HCPs) is associated with burnout and lower-quality patient care, but mistreatment by patients and family members is underreported. We hypothesized that an organizational strategy that includes training, safety incident reporting, and a response protocol would increase HCP knowledge, self-efficacy, and reporting of mistreatment.

METHODS

In this single-center, serial, cross-sectional study, we sent an anonymous survey to HCPs before and after the intervention at a 213-bed tertiary care university children's hospital between 2018 and 2019. We used multivariable logistic regression to examine the effect of training on the outcomes of interest and whether this association was moderated by staff role.

RESULTS

We received 309 baseline surveys from 72 faculty, 191 nurses, and 46 residents, representing 39.1%, 27.1%, and 59.7%, respectively, of eligible HCPs. Verbal threats from patients or family members were reported by 214 (69.5%) HCPs. Offensive behavior was most commonly based on provider age (85, 28.5%), gender (85, 28.5%), ethnicity or race (55, 18.5%), and appearance (43, 14.6%) but varied by role. HCPs who received training had a higher odds of reporting knowledge, self-efficacy, and experiencing offensive behavior. Incident reporting of mistreatment increased threefold after the intervention.

CONCLUSIONS

We report an effective organizational approach to address mistreatment of HCPs by patients and family members. Our approach capitalizes on existing patient safety culture and systems that can be adopted by other institutions to address all forms of mistreatment, including those committed by other HCPs.

A Machine Learning Approach to Reclassifying Miscellaneous Patient Safety Event Reports

BACKGROUND AND OBJECTIVES

Medical errors are a leading cause of death in the United States. Despite widespread adoption of patient safety reporting systems to address medical errors, making sense of the reports collected in these systems is challenging in practice. Event classification taxonomies used in many reporting systems can be complex and difficult to understand by frontline reporters, leading reporters to classify reports as "miscellaneous" as opposed to assigning a specific event-type category, which may facilitate analysis.

METHODS

To assist patient safety analysts in their analysis of "miscellaneous" reports, we developed an ensemble machine learning natural language processing model to reclassify these reports. We integrated the model into a clinical workflow dashboard, evaluated user feedback, and compared differences in user thresholds for model performance.

RESULTS AND CONCLUSIONS

Integrating an ensemble model to classify "miscellaneous" event reports with an interactive visualization was helpful to patient safety analysts review "miscellaneous" reports. However, patient safety analysts have different thresholds for model reclassification depending on their role and experience with "miscellaneous" event reports.

Development, Validation, and Implementation of a Guideline to Improve Clinical Event Debriefing at a Level-I Adult and Level-II Pediatric Trauma Center

OBJECTIVE

Clinical event debriefing is recommended by the American Heart Association and the European Resuscitation Council, because debriefings improve team performance. The purpose here was to develop and validate tools needed to overcome barriers to debriefing in the emergency department.

METHOD

This quality improvement project was conducted in 4 phases. Phase 1: Current evidence related to debriefing in the emergency department was reviewed and synthesized to inform an iterative process for drafting the debriefing guideline and instrument for documentation. Phase 2: Content Validity Index of the tools was evaluated by obtaining ratings of items' clarity and relevance from 5 national experts in 2 rounds of review. On the basis of experts' feedback, tools were revised, and a Facilitators' Guide was created. Phase 3: The validated debriefing tools were implemented. Phase 4: Debriefing facilitators completed a survey about their experience with using the new tools.

RESULTS

The Content Validity Index of 71 debriefing tool items (guideline, instrument, Facilitators' Guide) was 0.93 and 0.96 for clarity and relevance, respectively. Of the 32 debriefings conducted during the first 8 weeks of implementation, 53% described patient safety concerns, and 97% described recommendations to improve performance. Most (94%) facilitators agreed that the guideline clarified debriefing requirements.

CONCLUSION

The use of debriefing tools validated by computation of the Content Validity Index led to the identification of safety threats and recommendations to improve care processes. These tools can be used in ED settings to promote team learning and aid in identifying and resolving safety concerns.

Assessment of patient safety challenges and electronic occurrence variance reporting (e-OVR) barriers facing physicians and nurses in the emergency department: a cross sectional study

BACKGROUND

The purpose of patient safety is to prevent harm occurring in the healthcare system. Patient safety is improved by the use of a reporting system in which healthcare workers can document and learn from incidents, and thus prevent potential medical errors. The present study aimed to determine patient safety challenges facing clinicians (physicians and nurses) in emergency medicine and to assess barriers to using e-OVR (electronic occurrence variance reporting).

METHODS

This cross-sectional study involved physicians and nurses in the emergency department (ED) at King Khalid University Hospital (KKUH) in Riyadh, Saudi Arabia. Using convenience sampling, a self-administered questionnaire was distributed to 294 clinicians working in the ED. The questionnaire consisted of items pertaining to patient safety and e-OVR usability. Data were analyzed using frequencies, means, and percentages, and the chi-square test was used for comparison.

RESULTS

A total of 197 participants completed the questionnaire (67% response rate) of which 48 were physicians (24%) and 149 nurses (76%). Only 39% of participants thought that there was enough staff to handle work in the ED. Roughly half (48%) of participants spoke up when something negatively affected patient safety, and 61% admitted that they sometimes missed important patient care information during shift changes. Two-thirds (66%) of the participants reported experiencing violence. Regarding e-OVR, 31% of participants found reporting to be time consuming. Most (85%) participants agreed that e-OVR training regarding knowledge and skills was sufficient. Physicians reported lower knowledge levels regarding how to access (46%) and how to use (44%) e-OVR compared to nurses (98 and 95%, respectively; p < 0.01). Less than a quarter of the staff did not receive timely feedback after reporting. Regarding overall satisfaction with e-OVR, only 25% of physicians were generally satisfied compared to nearly half (52%) of nurses.

CONCLUSION

Although patient safety is well emphasized in clinical practice, especially in the ED, many factors hinder patient safety. More awareness is needed to eliminate violence and to emphasize the needs of additional staff in the ED. Electronic reporting and documentation of incidents should be well supported by continuous staff training, help, and feedback.

Factors affecting nursing error communication in intensive care units: A qualitative study

BACKGROUND

Error communication includes both reporting errors to superiors and disclosing their consequences to patients and their families. It significantly contributes to error prevention and safety improvement. Yet, some errors in intensive care units are not communicated.

OBJECTIVES

The aim of the present study was to explore factors affecting error communication in intensive care units.

DESIGN AND PARTICIPANTS

This qualitative study was conducted in 2019. Participants were 17 critical care nurses purposively recruited from the intensive care units of 2 public hospitals affiliated to Iran University of Medical Sciences, Tehran, Iran. Data were collected through in-depth semi-structured interviews and were analyzed through the conventional content analysis method proposed by Graneheim and Lundman.

ETHICAL CONSIDERATIONS

The Ethics Committee of Iran University of Medical Sciences, Tehran, Iran approved the study (code: IR.IUMS. REC.1397.792). Participants were informed about the study aim and methods and were ensured of data confidentiality. They were free to withdraw from the study at will. Written informed consent was obtained from all of them.

FINDINGS

Factors affecting error communication in intensive care units fell into four main categories, namely the culture of error communication (subcategories were error communication organizational atmosphere, clarity of processes and guidelines, managerial support for nurses, and learning organization), the consequences of errors for nurses and nursing (subcategories were fear over being stigmatized as incompetent, fear over punishment, and fear over negative judgments about nursing), the consequences of errors for patients (subcategories were monitoring the effects of errors on patients and predicting the effects of errors on patients), and ethical and professional characteristics (subcategories were ethical characteristics and inter-professional relationships).

DISCUSSION

The results of this study show many factors affect error communication, some facilitate and some prohibit it. Organizational factors such as the culture of error communication and the consequences of error communication for the nurse and the patient, as well as individual and professional characteristics, including ethical characteristics and interprofessional relationship, influence this process.

CONCLUSION

Errors confront nurses with ethical challenges and make them assess error consequences and then, communicate or hide them based on the results of their assessments. Health authorities can promote nurses' error communication through creating a supportive environment for them, developing clear error communication processes and guidelines, and providing them with education about the principles of ethical practice.

Practical and Cultural Barriers to Reporting Incidents Among Health Workers in Indonesian Public Hospitals

PURPOSE

This study investigated the practical and cultural barriers of reporting patient safety incidents in three accredited public hospitals in East Java, Indonesia.

METHODS

This study employed a mixed methods approach using a convergent parallel design. We surveyed 1121 health workers and interviewed 27 managerial staff members from the sampled hospitals. A chi-square analysis was performed to evaluate differences in demographic factors, barriers to reporting, and practices of reporting between those who had reported an incident and those who had witnessed an incident but had not reported it. NVivo 11 software was used to perform the qualitative data analysis.

RESULTS

This study had a 76.53% response rate. The quantitative evaluation identified significant differences in professions and work units and in participation in quality and safety training between the reporting group and the non-reporting group. The analysis of practical barriers displayed significant differences between the groups with the following responses: "did not know how to report," "did not know where to report," and "lack of feedback". For cultural barriers, a significant difference was shown only for the response "did not want conflict." In the qualitative assessment, most of the interview participants reported lack of knowledge and lack of socialization or training as practical barriers in reporting incidents. Furthermore, reluctance and fear to report were mentioned as cultural barriers by most of the interviewees.

CONCLUSION

Because there were conflicting findings in the barriers of reporting incidents, these barriers must be identified, discussed, and resolved by health workers and their managers or supervisors to improve incident reporting. Managers must foster open communication and build positive connections with health workers. Further research is necessary to focus on possible ways of addressing the barriers to reporting.

Analysis of Patient Safety Incident reporting system as an indicator of quality nursing in critical care units in KwaZulu-Natal, South Africa

BACKGROUND

Patient Safety Incidents occur frequently in critical care units, contribute to patient harm, compromise quality of patient care and increase healthcare costs. It is essential that Patient Safety Incidents in critical care units are continually measured to plan for quality improvement interventions.

AIM

To analyse Patient Safety Incident reporting system, including the evidence of types, frequencies, and patient outcomes of reported incidents in critical care units.

SETTING

The study was conducted in the critical care units of ten hospitals of eThekwini district, in KwaZulu-Natal, South Africa.

METHODS

A quantitative approach using a descriptive cross sectional survey was adopted to collect data from the registered nurses working in critical care units of randomly selected hospitals. Self-administered questionnaires were distributed to 270 registered nurses of which 224 (83%) returned completed questionnaires. A descriptive statistical analysis was initially conducted, then the Pearson Chi-square test was performed between the participating hospitals.

FINDINGS

One thousand and seventeen (n = 1017) incidents in ten hospitals were self-reported. Of these incidents, 18% (n = 70) were insignificant, 35% (n = 90) minor, 25% (n = 75) moderate, 12% (n = 32) major and 10% (n = 26) catastrophic. Patient Safety Incidents were classified into six categories: (a) Hospital-related incidents (42% [n = 416]); (b) Patient care-related incidents (30% [n = 310]); (c) (Death 12% [n = 124]); (d) Medication-related incidents, (7% [n = 75]); (e) Blood product-related incidents (5% [n = 51]) and (f) Procedure-related incidents (4% [n = 41]).

CONCLUSION

This study's findings indicating 1017 Patient Safety Incidents of predominantly serious nature, (47% considering moderate, major and catastrophic) are a cause for concern.

Closing the Loop with Ambulatory Staff on Safety Reports

BACKGROUND

A commonly cited reason among nurses and physicians for not reporting safety events is a perceived lack of feedback from management on filed safety reports. This suggests that the value of a safety reporting system could be improved with a closed-loop feedback system between management and frontline staff on filed safety reports in which feedback was requested.

METHODS

Ambulatory staff were surveyed on barriers to reporting to assess this challenge at an academic medical center. In response, system changes were implemented to the electronic safety reporting system, gained leadership buy-in, incorporated managers into a work group tasked with enhancing feedback to staff, established project management support, and developed a safety star manager recognition program. Ultimately, a process was developed to measure and ensure that feedback was provided to staff who requested it through a series of Plan-Do-Study-Act cycles termed the Feedback to Reporter program.

RESULTS

At baseline in 2013, the team found that staff who indicated they wanted feedback on safety reports received it less than 50% of the time. By the end of fiscal year 2018, the monthly feedback to reporter rate was consistently 90% or higher. The percentage of safety reports in which feedback was requested ranged from 35.0% to 49.7% of all safety reports submitted.

CONCLUSION

Ultimately, a multidimensional approach improved closed-loop communication from local managers to frontline staff and between managers of different departments on ambulatory safety reports when feedback was requested. Improvements were sustained for more than one year.

A Model for Sustaining Health at the Primary Care Level

As the healthcare industry in USA is changing from a fee-for-service to a value-based system, the need for a shift in how patients are treated is evident. Healthcare organizations are reimbursed based on value and quality of service. The system shift recognizes that each patient possesses differing medical needs moving care from generalized medical treatments to individualistic care. To deal with this change and attempt to increase quality and value, many healthcare organizations are adopting a team care approach through the development of Patient-Centered Medical Homes (PCMH). In many examples of the team approach, the Primary Care Practitioner (PCP) is viewed as the main coordinator of care. Having this responsibility can create added stress for practitioners, which can lead to a decrease in the quality of care. The proposed model, in this article, outlines an example of how individualistic care can be achieved and assembled in the PCMH with the PCP as the main coordinator of care to sustain patient health.

Survey to identify depth of penetration of critical incident reporting systems in Austrian healthcare facilities

Incident reporting systems or so-called critical incident reporting systems (CIRS) were first recommended for use in health care more than 15 years ago. The uses of these CIRS are highly variable among countries, ranging from being used to report critical incidents, falls, or sentinel events resulting in death. In Austria, CIRS have only been introduced to the health care sector relatively recently. The goal of this work, therefore, was to determine whether and specifically how CIRS are used in Austria. A working group from the Austrian Society for Quality and Safety in Healthcare (ASQS) developed a survey on the topic of CIRS to collect information on penetration of CIRS in general and on how CIRS reports are used to increase patient safety. Three hundred seventy-one health care professionals from 274 health care facilities were contacted via e-mail. Seventy-eight respondents (21.0%) completed the online survey, thereof 66 from hospitals and 12 from other facilities (outpatient clinics, nursing homes). In all, 64.1% of the respondents indicated that CIRS were used in the entire health care facility; 20.6% had not yet introduced CIRS and 15.4% used CIRS only in particular areas. Most often, critical incidents without any harm to patients were reported (76.9%); however, some health care facilities also use their CIRS to report patient falls (16.7%), needle stick injuries (17.9%), technical problems (51.3%), or critical incidents involving health care professionals. CIRS are not yet extensively or homogeneously used in Austria. Inconsistencies exist with respect to which events are reported as well as how they are followed up and reported to health care professionals. Further recommendations for general use are needed to support the dissemination in Austrian health care environments.