1
|
Warren BE, Bilbily A, Gichoya JW, Chartier LB, Fawzy A, Barragán C, Jaberi A, Mafeld S. An Introductory Guide to Artificial Intelligence in Interventional Radiology: Part 2: Implementation Considerations and Harms. Can Assoc Radiol J 2024; 75:568-574. [PMID: 38445517 DOI: 10.1177/08465371241236377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2024] Open
Abstract
The introduction of artificial intelligence (AI) in interventional radiology (IR) will bring about new challenges and opportunities for patients and clinicians. AI may comprise software as a medical device or AI-integrated hardware and will require a rigorous evaluation that should be guided based on the level of risk of the implementation. A hierarchy of risk of harm and possible harms are described herein. A checklist to guide deployment of an AI in a clinical IR environment is provided. As AI continues to evolve, regulation and evaluation of the AI medical devices will need to continue to evolve to keep pace and ensure patient safety.
Collapse
Affiliation(s)
- Blair Edward Warren
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Alexander Bilbily
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- 16 Bit Inc., Toronto, ON, Canada
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | | | - Lucas B Chartier
- Division of Emergency Medicine, Department of Medicine, University of Toronto, Toronto, ON, Canada
- Department of Emergency Medicine, University Health Network, Toronto, ON, Canada
| | - Aly Fawzy
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Camilo Barragán
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Arash Jaberi
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Sebastian Mafeld
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| |
Collapse
|
2
|
Ahrari A, Healy GM, Min A, Alkhalifah F, Oreopoulos G, Teng Tan K, Jaberi A, Rajan DK, Mafeld S. Real-World Experience With the Angio-Seal Closure Device: Insights From Manufacturer and User Facility Device Experience Database. J Endovasc Ther 2023:15266028231219226. [PMID: 38110358 DOI: 10.1177/15266028231219226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2023]
Abstract
PURPOSE Angio-Seal (Terumo Medical Corporations, Somerset, New Jersey) device is indicated for femoral arteriotomy closure. Real-world published data on complications are limited. We present 1 year of safety events involving Angio-Seal from the US Food and Drug Administration's post-market surveillance database of Manufacturer and User Facility Device Experience (MAUDE). Steps for managing frequent device-related problems are discussed. MATERIALS AND METHODS Angio-Seal MAUDE data from November 2019 to December 2020 was classified according to (1) mode of device failure, (2) complication, (3) treatment, and (4) Cardiovascular and Interventional Radiological Society of Europe (CIRSE) adverse event classification system. RESULTS There were 715 safety events, involving Angio-Seal VIP (93.1%), Evolution (5.7%), STS Plus (1.1%), and sizes 6F (62.5%) and 8F (37.5%). Failure mode involved unrecognized use of a damaged device (43.4%), failed deployment (20.1%), failed arterial advancement (6.3%), detachment of device component (4.9%), failed retraction (3.6%), operator error (1.1%), and indeterminate (20.6%). Of total, 44.8% of events were associated with patient harm. Complications involved minor blood loss (34.1%), hematoma (5.6%), significant blood loss (1.4%), and pseudoaneurysm (1.4%). Of total, 43.3% of cases required manual compression (MC), whereas 8.8% required more advanced intervention. Interventions included surgical repair (49.2%), thrombin injection (9.5%), balloon tamponade (6.3%), covered stent (4.8%), and unspecified (30.2%). Majority of safety events were CIRSE grade 1 (92.0%), followed by grades 2 (3.1%), 3 (4.6%), and 6 (deaths, 0.3%). Minority of devices were returned for manufacturer analysis (27.8%). CONCLUSIONS The majority of safety events were associated with minor blood loss or local hematoma and could be addressed with MC alone. Most events were attributed to damaged device; however, very few devices were returned to manufacturer for analysis. This should be encouraged to allow for root cause analysis in order to improve safety profile of devices. System-level strategies for addressing barriers to under-reporting of safety events may also be considered. CLINICAL IMPACT Our study highlights important safety events encountered in real-world practice with Angio-Seal closure device. The MAUDE database captures real-world device malfunctions not typically appreciated in conventional clinical trials. Our study provides valuable insight for clinician-users on anticipating and managing the most common device malfunctions. Additionally, our data provide feedback for manufactures to optimize product design and direct manufacturer user training to improve safety. Finally, we hope that the study promotes system-level strategies that foster reporting of safety events and undertaking of root cause analysis.
Collapse
Affiliation(s)
- Aida Ahrari
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - Gerard M Healy
- Department of Radiology, St Vincent's University Hospital, Dublin, Ireland
| | - Adam Min
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - Fahd Alkhalifah
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - George Oreopoulos
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
- Division of Vascular Surgery, University Health Network, Toronto, ON, Canada
| | - Kong Teng Tan
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - Arash Jaberi
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - Dheeraj K Rajan
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| | - Sebastian Mafeld
- Department of Radiology, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network and Sinai Health System, Toronto, ON, Canada
| |
Collapse
|
3
|
Healy GM, Ahrari A, Alkhalifah F, Oreopoulos G, Tan KT, Jaberi A, Mafeld S. Typology, Severity, and Outcomes of Adverse Events Related to Angiographic Equipment-A Ten-Year Analysis of the FDA MAUDE Database. Can Assoc Radiol J 2023; 74:737-744. [PMID: 37023704 DOI: 10.1177/08465371231167990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/08/2023] Open
Abstract
Purpose: Angiographic equipment is a key component of healthcare infrastructure, used for endovascular procedures throughout the body. The literature on adverse events related to this technology is limited. The purpose of this study was to analyze adverse events related to angiographic devices from the US Food and Drug Administration's Manufacturer and User Facility Device Experience (MAUDE) database. Methods: MAUDE data on angiographic imaging equipment from July 2011 to July 2021 were extracted. Qualitative content analysis was performed, a typology of adverse events was derived, and this was used to classify the data. Outcomes were assessed using the Healthcare Performance Improvement (HPI) and Society of Interventional Radiology (SIR) adverse event classifications. Results: There were 651 adverse events reported. Most were near misses (67%), followed by precursor safety events (20.5%), serious safety events (11.2%), and unclassifiable (1.2%). Events impacted patients (42.1%), staff (3.2%), both (1.2%), or neither (53.5%). The most common events associated with patient harm were intra-procedure system shut down, foot pedal malfunction, table movement malfunction, image quality deterioration, patient falls, and fluid damage to system. Overall, 34 (5.2%) events were associated with patient death; 18 during the procedure and 5 during patient transport to another angiographic suite/hospital due to critical failure of equipment. Conclusion: Adverse events related to angiographic equipment are rare; however, serious adverse events and deaths have been reported. This study has defined a typology of the most common adverse events associated with patient and staff harm. Increased understanding of these failures may lead to improved product design, user training, and departmental contingency planning.
Collapse
Affiliation(s)
- Gerard M Healy
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
- Department of Radiology, St Vincent's University Hospital, Dublin, Ireland
- School of Medicine, University College Dublin, Ireland
| | - Aida Ahrari
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
| | - Fahd Alkhalifah
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
| | - George Oreopoulos
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
- Division of Vascular Surgery, University Health Network, University of Toronto, Toronto, ON, CA
| | - Kong Teng Tan
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
| | - Arash Jaberi
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
| | - Sebastian Mafeld
- Division of Vascular and Interventional Radiology, Joint Department of Medical Imaging, University Health Networkand Sinai Health System, Toronto, ON, CA
- Department of Medical Imaging, University of Toronto, Toronto, ON, CA
| |
Collapse
|
4
|
Soyer P, Patlas MN. Adverse Events Self-Reporting in Radiology:A New Avenue for Excellence. Can Assoc Radiol J 2023; 74:612-613. [PMID: 37058002 DOI: 10.1177/08465371231171597] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/15/2023] Open
Affiliation(s)
- Philippe Soyer
- Department of Radiology, Hopital Cochin, APHP, 751014 Paris, France
- Université Paris Cité, Faculté de Médecine, 75006 Paris, France
| | - Michael N Patlas
- Department of Radiology, Hamilton General Hospital, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
5
|
Conway A, Goudarzi Rad M, Zhou W, Parotto M, Jungquist C. Deep learning classification of capnography waveforms: secondary analysis of the PRODIGY study. J Clin Monit Comput 2023; 37:1327-1339. [PMID: 37178234 DOI: 10.1007/s10877-023-01028-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 04/30/2023] [Indexed: 05/15/2023]
Abstract
Capnography monitors trigger high priority 'no breath' alarms when CO2 measurements do not exceed a given threshold over a specified time-period. False alarms occur when the underlying breathing pattern is stable, but the alarm is triggered when the CO2 value reduces even slightly below the threshold. True 'no breath' events can be falsely classified as breathing if waveform artifact causes an aberrant spike in CO2 values above the threshold. The aim of this study was to determine the accuracy of a deep learning approach to classifying segments of capnography waveforms as either 'breath' or 'no breath'. A post hoc secondary analysis of data from 9 North American sites included in the PRediction of Opioid-induced Respiratory Depression In Patients Monitored by capnoGraphY (PRODIGY) study was conducted. We used a convolutional neural network to classify 15 s capnography waveform segments drawn from a random sample of 400 participants. Loss was calculated over batches of 32 using the binary cross-entropy loss function with weights updated using the Adam optimizer. Internal-external validation was performed by iteratively fitting the model using data from all but one hospital and then assessing its performance in the remaining hospital. The labelled dataset consisted of 10,391 capnography waveform segments. The neural network's accuracy was 0.97, precision was 0.97 and recall was 0.96. Performance was consistent across hospitals in internal-external validation. The neural network could reduce false capnography alarms. Further research is needed to compare the frequency of alarms derived from the neural network with the standard approach.
Collapse
Affiliation(s)
- Aaron Conway
- Peter Munk Cardiac Centre, University Health Network, Toronto, Canada.
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada.
| | | | - Wentao Zhou
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
| | - Matteo Parotto
- Department of Anesthesia and Pain Management, Toronto General Hospital, UHN, Toronto, Canada
- Department of Anesthesiology and Pain Medicine and Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, Canada
| | | |
Collapse
|
6
|
Kumari D, Ahmed O, Jilani S, Funaki E, Funaki B. A Review of Professional Liability in IR: Sweeping the Mines. J Vasc Interv Radiol 2023; 34:157-163. [PMID: 36241149 DOI: 10.1016/j.jvir.2022.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 09/30/2022] [Accepted: 10/04/2022] [Indexed: 11/06/2022] Open
Abstract
Medical professional liability (MPL) is becoming a substantial issue in interventional radiology (IR), with both impact on health care costs and negative psychological effects on physicians. MPL presents special challenges within IR because of the field's complex and innovative therapies that are provided to a diverse group of patients and complicated by the off-label use of devices and drugs that is pervasive in the field. This review discusses the principles and practices to avoid and manage MPLs that are specific to the field of IR.
Collapse
Affiliation(s)
- Divya Kumari
- Department of Radiology, University of Chicago Medicine, Chicago, Illinois.
| | - Osman Ahmed
- Department of Radiology, University of Chicago Medicine, Chicago, Illinois
| | | | | | - Brian Funaki
- Department of Radiology, University of Chicago Medicine, Chicago, Illinois
| |
Collapse
|
7
|
Wattamwar K, Garg T, Gabr A, Hirschl D, Hammer MM. Intraprocedural Errors in Interventional Radiology: A Perspective for Trainees and Training Programs. Radiographics 2022; 42:E162-E164. [DOI: 10.1148/rg.220156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
8
|
AIM in Interventional Radiology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
9
|
Conway A, Jungquist CR, Chang K, Kamboj N, Sutherland J, Mafeld S, Parotto M. Predicting Prolonged Apnea During Nurse-Administered Procedural Sedation: Machine Learning Study. JMIR Perioper Med 2021; 4:e29200. [PMID: 34609322 PMCID: PMC8527383 DOI: 10.2196/29200] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Revised: 05/26/2021] [Accepted: 08/23/2021] [Indexed: 01/17/2023] Open
Abstract
BACKGROUND Capnography is commonly used for nurse-administered procedural sedation. Distinguishing between capnography waveform abnormalities that signal the need for clinical intervention for an event and those that do not indicate the need for intervention is essential for the successful implementation of this technology into practice. It is possible that capnography alarm management may be improved by using machine learning to create a "smart alarm" that can alert clinicians to apneic events that are predicted to be prolonged. OBJECTIVE To determine the accuracy of machine learning models for predicting at the 15-second time point if apnea will be prolonged (ie, apnea that persists for >30 seconds). METHODS A secondary analysis of an observational study was conducted. We selected several candidate models to evaluate, including a random forest model, generalized linear model (logistic regression), least absolute shrinkage and selection operator regression, ridge regression, and the XGBoost model. Out-of-sample accuracy of the models was calculated using 10-fold cross-validation. The net benefit decision analytic measure was used to assist with deciding whether using the models in practice would lead to better outcomes on average than using the current default capnography alarm management strategies. The default strategies are the aggressive approach, in which an alarm is triggered after brief periods of apnea (typically 15 seconds) and the conservative approach, in which an alarm is triggered for only prolonged periods of apnea (typically >30 seconds). RESULTS A total of 384 apneic events longer than 15 seconds were observed in 61 of the 102 patients (59.8%) who participated in the observational study. Nearly half of the apneic events (180/384, 46.9%) were prolonged. The random forest model performed the best in terms of discrimination (area under the receiver operating characteristic curve 0.66) and calibration. The net benefit associated with the random forest model exceeded that associated with the aggressive strategy but was lower than that associated with the conservative strategy. CONCLUSIONS Decision curve analysis indicated that using a random forest model would lead to a better outcome for capnography alarm management than using an aggressive strategy in which alarms are triggered after 15 seconds of apnea. The model would not be superior to the conservative strategy in which alarms are only triggered after 30 seconds.
Collapse
Affiliation(s)
- Aaron Conway
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada.,Peter Munk Cardiac Centre, Toronto General Hospital, Toronto, ON, Canada.,School of Nursing, Queensland University of Technology, Brisbane, Australia
| | - Carla R Jungquist
- School of Nursing, The University at Buffalo, Buffalo, NY, United States
| | - Kristina Chang
- Peter Munk Cardiac Centre, Toronto General Hospital, Toronto, ON, Canada
| | - Navpreet Kamboj
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada
| | - Joanna Sutherland
- Rural Clinical School, University of New South Wales, Coffs Harbour, Australia
| | - Sebastian Mafeld
- Joint Department of Medical Imaging, Toronto General Hospital, Toronto, ON, Canada
| | - Matteo Parotto
- Department of Anesthesia and Pain Management, Toronto General Hospital, Toronto, ON, Canada.,Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada.,Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
10
|
Pinto A, Giurazza F, Califano T, Rea G, Valente T, Niola R, Caranci F. Interventional radiology in gynecology and obstetric practice: Safety issues. Semin Ultrasound CT MR 2021; 42:104-112. [PMID: 33541584 PMCID: PMC7525270 DOI: 10.1053/j.sult.2020.09.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Interventional radiology is continuing to reshape current practice in many specialties of clinical care and the fields of gynecology and obstetrics are no exception. Imaging skills, clinical knowledge as well as vascular and non-vascular interventional technical ability, are essential to practice interventional radiology effectively. Patient safety is of paramount importance in interventional radiology as in all branches of medicine. Potential failures occur throughout successful procedures and are attributed to a spectrum of errors, including equipment unavailability, planning errors, and communication errors. These are mainly preventable by improved preprocedural planning and teamwork. Of all the targeted and effective actions that can be undertaken to reduce adverse events, the use of safety checklists might have a prominent role. The advantage of a safety checklist for interventional radiology is that it guarantees that human error in terms of forgetting key steps in patient preparation, intraprocedural care, and postoperative care are not forgotten.
Collapse
Affiliation(s)
- Antonio Pinto
- Department of Radiology, CTO Hospital, Azienda Ospedaliera dei Colli, Naples, Italy.
| | - Francesco Giurazza
- Vascular and Interventional Radiology Department, Cardarelli Hospital, Naples, Italy
| | - Teresa Califano
- Department of Radiology, CTO Hospital, Azienda Ospedaliera dei Colli, Naples, Italy
| | - Gaetano Rea
- Department of Radiology, Monaldi Hospital, Azienda Ospedaliera dei Colli, Naples, Italy
| | - Tullio Valente
- Department of Radiology, Monaldi Hospital, Azienda Ospedaliera dei Colli, Naples, Italy
| | - Raffaella Niola
- Vascular and Interventional Radiology Department, Cardarelli Hospital, Naples, Italy
| | - Ferdinando Caranci
- Department of Precision Medicine, School of Medicine, "Luigi Vanvitelli" University of Campania, Naples, Italy
| |
Collapse
|
11
|
Datta S. AIM in Interventional Radiology. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_283-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|