Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference with the NSQIP-Pediatric System

An Electronic Health Record-integrated Web Application Augments a QI-directed Morbidity & Mortality Conference and Improves Quality of Care

BACKGROUND

In 2014, we developed a QI-directed Morbidity and Mortality (M&M) Conference, prioritizing discussion of individual and system failures, as well as development of action items to prevent failure recurrence. However, due to a reliance on individual electronic documents to store M&M data, our ability to assess trends in failures and action item implementation was hindered. To address this issue, in 2019, we created a secure electronic health record (EHR)-integrated web application (web app) to store M&M data.

STUDY DESIGN

In this study, we assessed the impact of our web app on efficient review and tracking of M&M data, including system failure occurrence and closure of action items. Additionally, in 2021, it was discovered that a backlog of action items existed. To address this issue, we implemented a QI initiative to reduce the backlog, and used the web app to compare action item closure over time.

RESULTS

Use of the web app dramatically improved review of M&M data. During the study period, there was a 67.0% reduction in the occurrence of the most common system failures. Additionally, our QI initiative resulted in a 97.7% reduction in the duration of time to complete a single action item and a 61.1% increase in the on-time closure rate for action items.

CONCLUSIONS

Integration of a web app into a QI-directed M&M Conference enhanced our ability to track system level failures and action item closure over time. Using this web app, we demonstrated that our M&M Conference achieved its intended goal of improving the quality of patient care.

LEVEL OF EVIDENCE

IV.

Practice-Based Learning and Improvement: Improving Morbidity and Mortality Review Using Natural Language Processing

INTRODUCTION

Practice-Based Learning and Improvement, a core competency identified by the Accreditation Council for Graduate Medical Education, carries importance throughout a physician's career. Practice-Based Learning and Improvement is cultivated by a critical review of complications, yet methods to accurately identify complications are inadequate. Machine-learning algorithms show promise in improving identification of complications. We compare a manual-supplemented natural language processing (ms-NLP) methodology against a validated electronic morbidity and mortality (MM) database, the Morbidity and Mortality Adverse Event Reporting System (MARS) to understand the utility of NLP in MM review.

METHODS

The number and severity of complications were compared between MARS and ms-NLP of surgical hospitalization discharge summaries among three academic medical centers. Clavien-Dindo (CD) scores were assigned to cases with identified complications and classified into minor (CD I-II) or major (CD III-IV) harm.

RESULTS

Of 7774 admissions, 987 cases were identified to have 1659 complications by MARS and 1296 by ms-NLP. MARS identified 611 (62%) cases, whereas ms-NLP identified 670 (68%) cases. Less than one-third of cases (299, 30.3%) were detected by both methods. MARS identified a greater number of complications with major harm (457, 46.30%) than did ms-NLP (P < 0.0001).

CONCLUSIONS

Both a prospectively maintained MM database and ms-NLP review of discharge summaries fail to identify a significant proportion of postoperative complications and overlap 1/3 of the time. ms-NLP more frequently identifies cases with minor complications, whereas prospective voluntary reporting more frequently identifies major complications. The educational benefit of reporting and analysis of complication data may be supplemented by ms-NLP but not replaced by it at this time.

A Survey-Based Assessment of the Practices Governing Morbidity and Mortality Conferences and the Effects of the COVID-19 Pandemic

BACKGROUND

Morbidity and mortality (M&M) conferences are essential components for resident education and provide a valuable tool to improve patient safety and quality of care. M&M conferences help identify important gaps in safety and reduce avoidable events in future patient care. Active methods to improve the utilization of M&M conferences have been shown to enhance their educational value for residents, faculty and multidisciplinary teams in healthcare institutions.

OBJECTIVE

The purpose of this study was to use a survey-based methodology to assess how morbidity and mortality conferences are conducted in residency programs, including characteristics such as frequency, involvement of personnel and the effects of COVID-19.

METHODS

From February to October 2021, a validated 19 question survey was electronically distributed to residency program directors in anesthesiology, emergency medicine and general surgery, after a search for email addresses in the ACGME database. The survey was created and hosted on Google Forms.

RESULTS

A total of 125 of 713 program directors (17.5%) responded to the survey. Eighty-three percent of respondent programs reported mandatory participation for residents, with residents providing most of the presentations. Case presentations utilized various formats including SBAR, adverse event analysis and root cause analysis as the most common modalities. Though most programs reported no change in frequency of M&M conferences due to COVID-19, most respondents reported a shift to a virtual or hybrid platform.

CONCLUSION

M&M conferences are an important educational and quality improvement modality, and many residency directors changed practice to incorporate virtual platforms due to the COVID-19 pandemic to maintain uninterrupted educational sessions. Nonetheless, significant variation still exists in how these conferences are conducted between different institutions.

A Sisyphean Task for Residents: Preparing Literature Reviews About Adverse Events Presented at Morbidity and Mortality Conferences

OBJECTIVE

Surgery Morbidity and Mortality (M&M) presentations include a thorough literature review. This requires a significant amount of time expenditure frequently incompatible with the current surgical resident work hours. Additionally, literature reviews can be redundant for commonly encountered adverse events. The goal of this study was to explore (a) how surgery residents perform literature reviews, and (b) how repetitive presented adverse events are.

DESIGN

A survey was sent out during the academic year 2019-2020. The Morbidity and Mortality repository for that academic year was indexed, and the proportion of adverse events having occurred more than once calculated. The amount of time spent on literature reviews, proportion of repetitive adverse events as well as degree of thoroughness of reviews was evaluated on a 1 to 5 Likert scale.

SETTING

Tulane University General Surgery program, New Orleans, LA, USA.

PARTICIPANTS

All clinically active residents.

RESULTS

All residents, filled out the survey. Seventeen out of 29 (58.6%) residents reported dedicating approximately one hour performing literature reviews. Median studying time was 1 hour (interquartile range: 1-1.5 hours). Seventeen out of 29 (58.6%) residents employed 2 resources. The most common combination of resources was PubMed and Google (11/29, 37.9%). Most residents (21/29, 72.4%) believed that their thoroughness was at most average (≤3/5 on a Likert scale) and 27/29 (93.1%) believed that their literature review could have been more thorough. More than half of the adverse events presented were found to be redundant during that academic year.

CONCLUSIONS

Time spent reviewing the literature does not allow for a thorough review, and a significant portion of adverse events presented are redundant. A central repository for literature reviews of adverse events would improve the quality of reviews and avoid duplicating efforts.

The Morbidity & Mortality Assessment Tool (MMAT): Design and Proof of Concept

BACKGROUND

Morbidity and Mortality Conference (MMC) is a traditional quality assessment tool among surgical residencies to evaluate complications within their care and to integrate and educate best practices. Unfortunately, it is difficult to validate and measure loop closure as a result of MMC. This may result in repeating past mistakes or worse, becoming a meeting of anecdotal experiences. The goal of this study is to present results from a morbidity and mortality assessment tool (MMAT) which provides a means of measuring and tracking factors related to those surgical complications discussed within the MMC.

MATERIALS AND METHODS

Three years of MMC presentations were organized into a database and further divided and organized into variables which included case by the class year of the presenting resident, service line, month the case was presented, and potential contributing factors. Contributing factors considered for each case were categorized as: Systems-Based, Direct Patient Care, and Interpersonal Communication. Contributing factors were assigned to cases by a review committee consisting of residents and faculty members.

RESULTS

The lack of knowledge, technical error, lack of experience, lack of supervision, failure to communicate with team members were present in greater than 10% of the presented cases. There was a "July Effect" in the Trauma service, where there was a statistically significant difference between the percentage of cases presented that involved Failure to Communicate errors when compared with the remainder of the year.

CONCLUSIONS

MMAT allows longitudinal collection of data from each MMC to recognize patterns that facilitate improvements in systems of care and institutional memory.

Evidence Review for the American College of Surgeons Quality Verification Part II: Processes for Reliable Quality Improvement

After decades of experience supporting surgical quality and safety by the American College of Surgeons, the American College of Surgeons Quality Verification Program was developed to help hospitals improve surgical quality, safety, and reliability. This review is the second of a 3-part review aiming to synthesize the evidence supporting the main principles of the American College of Surgeons Quality Verification Program. Evidence was systematically reviewed for 5 principles: case review, peer review, credentialing and privileging, data for surveillance, and continuous quality improvement using data. MEDLINE was searched for articles published from inception to January 2019 and 2 reviewers independently screened studies for inclusion in a hierarchical fashion, extracted data, and summarized results in a narrative fashion. A total of 9,098 studies across the 5 principles were identified. After exclusion criteria, a total of 184 studies in systematic reviews and primary studies were included for assessment. The identified literature supports the importance of standardized processes and systems to identify problems and improve quality of care.

NSQIP versus institutional morbidity and mortality conference: complementary complication reporting in pediatric spine fusion

PURPOSE

Other fields of medicine have demonstrated underreported surgical complication rates by institutional M&M compared to NSQIP. However, a study comparing surgical complication rates in the pediatric spine population, using an identical set of patients rather than nationally extrapolated, has not been performed.

METHODS

A single institution's ASC-NSQIP Pediatric spine fusion cases and its departmental team-reported M&M database for the same were reviewed for January 1, 2012 to December 31, 2018. Differences in surgical complication reporting between the two databases for the identical patient cohort were recorded.

RESULTS

NSQIP identified 50 pediatric spine fusion patients with complications out of 386 NSQIP-algorithm-sampled cases (13%). Of these complications, 23 were not reported in the M&M conference database (6% of NSQIP-sampled cases, 2.5% of all M&M cases). The most common under-reported complication categories include pneumonia (100% under-reported), clostridium difficile (100%), urinary tract infection (83%), and superficial wound disruption (67%). During the same 7 years, M&M covered 924 spine fusions and identified 162 complications. Of these 162 patients, 22 were included in the NSQIP sampling and were not reported as complications (6% of NSQIP sampled patients).

CONCLUSION

Recognizing complication rates is central to implement strategies for delivering better quality care. NSQIP data may serve as an important quality check for pediatric spine institutional M&M data, but both may not include all complications even within its sampled patients. In general, NSQIP's protocols identified more medical complications, while M&M has a surgical focus, benefits from the limitless follow-up, and involves timely departmental awareness of complications.

Pediatric Surgeon Perceptions of Participation in External Patient Safety Programs: impact on Patient Safety

Introduction:

Surgeons play a crucial role in preventing harm and contributing to the safety culture of their institutions. External safety data programs are designed to review adverse events and provide performance benchmarks to ameliorate future adverse events. The extent to which pediatric surgeons are aware of these programs, utilize data from these programs, and believe that they improve patient safety, is unknown.

Methods:

A cross-sectional survey of the American Pediatric Surgical Association membership was conducted to evaluate participation in and attitudes toward national safety benchmark programs (eg, National Surgical Quality Improvement Program). Surgeons’ perceptions of these activities, including barriers to participation and utilization of safety data, were measured. We performed standard frequency analyses and tests of associations between surgeon characteristics and the likelihood of utilizing safety data.

Results:

The response rate was 38% (353/928). Seventy-two percentage of respondents reported institutional participation in external safety benchmark programs. Of those, only 68% utilized data to improve or monitor patient safety. Surgeon-reported barriers to this process included lack of knowledge, time, and institutional resources.

Conclusions:

Many pediatric surgeons are aware of institutional participation in external safety data programs, but only a portion are involved in the utilization of these data. We have identified several barriers to participation and data utilization. These findings can help direct educational efforts to optimize our ability to learn from adverse event benchmarking and improve pediatric surgical care.

Room for "quality" improvement? Validating National Surgical Quality Improvement Program-Pediatric (NSQIP-P) appendectomy data

INTRODUCTION

Accurate data are essential for the validity of clinical registries. This study aimed to validate NSQIP-P data, assess representativeness, and evaluate risk-adjusted predictive ability at a single institution.

METHODS

A prospective appendectomy-specific pediatric surgery research database (RD) maintained by clinical researchers was compared to the NSQIP-P data for appendectomies performed in 2016 at a tertiary children's hospital. NSQIP-P sampled data collected by trained surgical clinical reviewers (SCRs) were compared to matched RD patients. Both datasets used NSQIP-P definitions. Using χ², datasets were compared by patient demographics, disease severity (simple vs. complicated), and outcomes.

RESULTS

458 appendectomies for acute appendicitis were performed in 2016, of which 250 (55%) were abstracted by SCRs and matched to RD patients. Patient demographics were similar between datasets. Disease severity (NSQIP-P:50% complicated vs RD:31% complicated) and composite morbidity (NSQIP-P:6.0% vs RD:14.4%) were significantly different (both p < 0.01). Demographics and outcomes were similar between matched (n = 250) and unsampled patients in the RD (n = 208). NSQIP-P's risk-adjusted predicted morbidity was significantly lower than morbidity observed in all (n = 458) RD patients (NSQIP-P:9.9% vs RD:14.2%, p < 0.01).

CONCLUSIONS

Though constituting a representative sample, NSQIP-P appendectomy data were inconsistent with department data. Discrepancies appear to be the result of underreporting of outcome variables and disease misclassification.

TYPE OF STUDY

Retrospective comparative review.

LEVEL OF EVIDENCE

Level III.

Electronic Voting to Improve Morbidity and Mortality Conferences

BACKGROUND

It is of major importance in clinical surgery to identify potential patterns and specific causes of complications. Therefore, morbidity and mortality meetings (M&M) are widely used to discuss and evaluate deviations from expected outcomes in order to improve surgical practice. Moreover, M&M represent an important tool for continuous medical education. In this study, we introduced an electronic voting system to assess whether anonymity during M&M could limit potential biases due to hierarchical structures or opinion leaders.

METHODS

This study was conducted in the surgical department of a European tertiary care center. During the study period, electronic voting was applied in 412 M&M cases and compared with a baseline of 330 conventional M&M entries. In this interrupted time series, the educational quality and participant satisfaction of the M&M were assessed using surveys before and after the introduction of electronic voting. The surveys were refined using principle component analysis. In addition, the classification of the cause of the complication was recorded.

RESULTS

The introduction of electronic voting led to a significant increase in perceived educational quality from 2.63 to 3.36 (p < 0.01), and the overall participant satisfaction increased from 2.6 ± 0.9 to 3.7 ± 1.2 (p < 0.01) on a five-point Likert scale. The frequency of voting shifted from "patient's disease" (before 42.9, after 27.6%, p = 0.04) to "misadventure" (before 1.1, after 16.0%, p < 0.01). The voting frequencies for the causes attributed to "management" and "technical" remained constant.

CONCLUSIONS

An electronic voting system in M&M meetings increases perceived educational quality and participant satisfaction.

Advances in perioperative quality and safety

For decades, safe surgery focused on intraoperative technique and decision-making. The traditional hierarchy placed the surgeon as the leader with ultimate authority and responsibility. Despite the advances in surgical technique and equipment, too many patients have suffered unnecessary complications and suboptimal care. Today, we understand that the conduct of safe and effective surgery requires evidence-based decision-making, multifaceted treatment approaches to prevent complications, and effective communication in and out of the operating room. In this manuscript, we describe three significant advances in quality and safety that have changed the approach to surgical care: the National Surgical Quality Improvement Program, evidence-based bundled prevention of surgical site infections, and the Surgical Safety Checklist.

Performance Improvement in Head and Neck Cancer

Performance improvement requires establishing a platform to set benchmarks and monitor the quality of care provided through quality indicators and metrics. This has long been recognized as critical to overall quality improvement and more recently, has become federally mandated. Here, we review recent studies evaluating performance in head and neck cancer care, from those spanning all phases of head and neck cancer care to others focused on head and neck surgical performance, including both national and departmental/institutional efforts.