A comparative assessment of adverse event classification in the out-of-hospital setting

Patient safety classifications, taxonomies and ontologies, part 2: A systematic review on content coverage

BACKGROUND

Content coverage of patient safety ontology and classification systems should be evaluated to provide a guide for users to select appropriate ones for specific applications. In this review, we identified and compare content coverage of patient safety classifications and ontologies.

METHODS

We searched different databases and ontology/classification repositories to identify these classifications and ontologies. We included patient safety-related taxonomies, ontologies, classifications, and terminologies. We identified and extracted different concepts covered by these systems and mapped these concepts to international classification for patient safety (ICPS) and finally compared the content of these systems.

RESULTS

Finally, 89 papers (77 classifications or ontologies) were analyzed. Thirteen classifications have been developed to cover all medical domains. Among specific domain systems, most systems cover medication (16), surgery (8), medical devices (3), general practice (3), and primary care (3). The most common patient safety-related concepts covered in these systems include incident types (41), contributing factors/hazards (31), patient outcomes (29), degree of harm (25), and action (18). However, stage/phase (6), incident characteristics (5), detection (5), people involved (5), organizational outcomes (4), error type (4), and care setting (3) are some of the less covered concepts in these classifications/ontologies.

CONCLUSION

Among general systems, ICPS, World Health Organization's Adverse Reaction Terminology (WHO-ART), and Ontology of Adverse Events (OAE) cover most patient safety concepts and can be used as a gold standard for all medical domains. As a result, reporting systems could make use of these broad classifications, but the majority of their covered concepts are related to patient outcomes, with the exception of ICPS, which covers other patient safety concepts. However, the ICPS does not cover specialized domain concepts. For specific medical domains, MedDRA, NCC MERP, OPAE, ADRO, PPST, OCCME, TRTE, TSAHI, and PSIC-PC provide the broadest coverage of concepts. Many of the patient safety classifications and ontologies are not formally registered or available as formal classification/ontology in ontology repositories such as BioPortal. This study may be used as a guide for choosing appropriate classifications for various applications or expanding less developed patient safety classifications/ontologies. Furthermore, the same concepts are not represented by the same terms; therefore, the current study could be used to guide a harmonization process for existing or future patient safety classifications/ontologies.

Building Consensus for a Shared Definition of Adverse Events: A Case Study in the Profession of Dentistry

BACKGROUND

To achieve high-quality health care, adverse events (AEs) must be proactively recognized and mitigated. However, there is often ambiguity in applying guidelines and definitions. We describe the iterative calibration process needed to achieve a shared definition of AEs in dentistry. Our alignment process includes both independent and consensus building approaches.

OBJECTIVE

We explore the process of defining dental AEs and the steps necessary to achieve alignment across different care providers.

METHODS

Teams from 4 dental institutions across the United States iteratively reviewed patient records after identification of charts using an automated trigger tool. Calibration across teams was supported through negotiated definition of AEs and standardization of evidence provided in review. Interrater reliability was assessed using descriptive and κ statistics.

RESULTS

After 5 iterative cycles of calibration, the teams (n = 8 raters) identified 118 cases. The average percent agreement for AE determination was 82.2%. Furthermore, the average, pairwise prevalence and bias-adjusted κ (PABAK) was 57.5% (κ = 0.575) for determining AE presence. The average percent agreement for categorization of the AE type was 78.5%, whereas the PABAK was 48.8%. Lastly, the average percent agreement for categorization of AE severity was 82.2% and the corresponding PABAK was 71.7%.

CONCLUSIONS

Successful calibration across reviewers is possible after consensus building procedures. Higher levels of agreement were found when categorizing severity (of identified events) rather than the events themselves. Our results demonstrate the need for collaborative procedures as well as training for the identification and severity rating of AEs.

Patient safety classification, taxonomy and ontology systems: A systematic review on development and evaluation methodologies

INTRODUCTION

Patient safety classifications/ontologies enable patient safety information systems to receive and analyze patient safety data to improve patient safety. Patient safety classifications/ontologies have been developed and evaluated using a variety of methods. The purpose of this review was to discuss and analyze the methodologies for developing and evaluating patient safety classifications/ontologies.

METHODS

Studies that developed or evaluated patient safety classifications, terminologies, taxonomies, or ontologies were searched through Google Scholar, Google search engines, National Center for Biomedical Ontology (NCBO) BioPortal, Open Biological and Biomedical Ontology (OBO) Foundry and World Health Organization (WHO) websites and Scopus, Web of Science, PubMed, and Science Direct. We updated our search on 30 February 2021 and included all studies published until the end of 2020. Studies that developed or evaluated classifications only for patient safety and provided information on how they were developed or evaluated were included. Systems with covered patient safety terms (such as ICD-10) but are not specifically developed for patient safety were excluded. The quality and the risk of bias of studies were not assessed because all methodologies and criteria were intended to be covered. In addition, we analyzed the data through descriptive narrative synthesis and compared and classified the development and evaluation methods and evaluation criteria according to available development and evaluation approaches for biomedical ontologies.

RESULTS

We identified 84 articles that met all of the inclusion criteria, resulting in 70 classifications/ontologies, nine of which were for the general medical domain. The most papers were published in 2010 and 2011, with 8 and 7 papers, respectively. The United States (50) and Australia (23) have the most studies. The most commonly used methods for developing classifications/ontologies included the use of existing systems (for expanding or mapping) (44) and qualitative analysis of event reports (39). The most common evaluation methods were coding or classifying some safety report samples (25), quantitative analysis of incidents based on the developed classification (24), and consensus among physicians (16). The most commonly applied evaluation criteria were reliability (27), content and face validity (9), comprehensiveness (6), usability (5), linguistic clarity (5), and impact (4), respectively.

CONCLUSIONS

Because of the weaknesses and strengths of the development/evaluation methods, it is advised that more than one method for development or evaluation, as well as evaluation criteria, should be used. To organize the processes of developing classification/ontologies, well-established approaches such as Methontology are recommended. The most prevalent evaluation methods applied in this domain are well fitted to the biomedical ontology evaluation methods, but it is also advised to apply some evaluation approaches such as logic, rules, and Natural language processing (NLP) based in combination with other evaluation approaches. This research can assist domain researchers in developing or evaluating domain ontologies using more complete methodologies. There is also a lack of reporting consistency in the literature and same methods or criteria were reported with different terminologies.

Measurement and monitoring patient safety in prehospital care: a systematic review

BACKGROUND

Prehospital care is potentially hazardous with the possibility for patients to experience an adverse event. However, as compared to secondary care, little is known about how patient safety is managed in prehospital care settings.

OBJECTIVES

The objectives of this systematic review were to identify and classify the methods of measuring and monitoring patient safety that have been used in prehospital care using the five dimensions of the Measuring and Monitoring Safety (MMS) framework and use this classification to identify where there are safety 'blind spots' and make recommendations for how these deficits could be addressed.

METHODS

Searches were conducted in January 2020, with no limit on publication year, using Medline, PsycInfo, CINAHL, Web of Science and Academic Search. Reference lists of included studies and existing related reviews were also screened. English-language, peer-reviewed studies concerned with measuring and monitoring safety in prehospital care were included. Two researchers independently extracted data from studies and applied a quality appraisal tool (the Quality Assessment Tool for Studies with Diverse Designs).

RESULTS

A total of 5301 studies were screened, with 52 included in the review. A total of 73% (38/52) of the studies assessed past harm, 25% (13/52) the reliability of safety critical processes, 1.9% (1/52) sensitivity to operations, 38.5% (20/52) anticipation and preparedness and 5.8% (3/52) integration and learning. A total of 67 methods for measuring and monitoring safety were used across the included studies. Of these methods, 38.8% (26/67) were surveys, 29.9% (20/67) were patient records reviews, 14.9% (10/67) were incident reporting systems, 11.9% (8/67) were interviews or focus groups and 4.5% (3/67) were checklists.

CONCLUSIONS

There is no single method of measuring and monitoring safety in prehospital care. Arguably, most safety monitoring systems have evolved, rather than been designed. This leads to safety blind spots in which information is lacking, as well as to redundancy and duplication of effort. It is suggested that the findings from this systematic review, informed by the MMS framework, can provide a structure for critically thinking about how safety is being measured and monitored in prehospital care. This will support the design of a safety surveillance system that provides a comprehensive understanding of what is being done well, where improvements should be made and whether safety interventions have had the desired effect.

Medication safety in emergency medical services: approaching an evidence-based method of verification to reduce errors

Lack of verification is often cited as a root cause of medication errors; however, medication errors occur in spite of conventional verification practices and it appears that human factors engineering (HFE) can inform the design of a more effective method. To this end, an HFE-driven process was designed and implemented in an urban, Midwestern emergency medical service agency. Medication error data were collected over a 54-month period, 27 months before and after implementation. A decrease in the average monthly error rate was realized for all medications administered (49.0%) during the post-intervention time period. The average monthly error rate for fentanyl, a commonly administered analgesic, demonstrated a 71.1% error rate decrease. This study is the first to evaluate the effectiveness of a team-based cross-check process for medication verification to prevent errors in the prehospital setting.

Adverse events in prehospital emergency care: a trigger tool study

Background

Prehospital emergency care has developed rapidly during the past decades. The care is given in a complex context which makes prehospital care a potential high-risk activity when it comes to patient safety. Patient safety in the prehospital setting has been only sparsely investigated. The aims of the present study were 1) To investigate the incidence of adverse events (AEs) in prehospital care and 2) To investigate the factors contributing to AEs in prehospital care.

Methods

We used a retrospective study design where 30 randomly selected prehospital medical records were screened for AEs each month in three prehospital organizations in Sweden during a period of one year. A total of 1080 prehospital medical records were included. The record review was based on the use of 11 screening criteria.

Results

The reviewers identified 46 AEs in 46 of 1080 (4.3%) prehospital medical records. Of the 46 AEs, 43 were classified as potential for harm (AE1) (4.0, 95% CI = 2.9–5.4) and three as harm identified (AE2) (0.3, 95% CI = 0.1–0.9). However, among patients with a life-threatening condition (priority 1), the risk of AE was higher (16.5%). The most common factors contributing to AEs were deviations from standard of care and missing, incomplete, or unclear documentation. The most common cause of AEs was the result of action(s) or inaction(s) by the emergency medical service (EMS) crew.

Conclusions

There were 4.3 AEs per 100 ambulance missions in Swedish prehospital care. The majority of AEs originated from deviations from standard of care and incomplete documentation. There was an increase in the risk of AE among patients who the EMS team assessed as having a life-threatening condition. Most AEs were possible to avoid.

Electronic supplementary material

The online version of this article (10.1186/s12873-019-0228-3) contains supplementary material, which is available to authorized users.