1
|
Hoppe BF, Rueckel J, Dikhtyar Y, Heimer M, Fink N, Sabel BO, Ricke J, Rudolph J, Cyran CC. Implementing Artificial Intelligence for Emergency Radiology Impacts Physicians' Knowledge and Perception: A Prospective Pre- and Post-Analysis. Invest Radiol 2024; 59:404-412. [PMID: 37843828 DOI: 10.1097/rli.0000000000001034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2023]
Abstract
PURPOSE The aim of this study was to evaluate the impact of implementing an artificial intelligence (AI) solution for emergency radiology into clinical routine on physicians' perception and knowledge. MATERIALS AND METHODS A prospective interventional survey was performed pre-implementation and 3 months post-implementation of an AI algorithm for fracture detection on radiographs in late 2022. Radiologists and traumatologists were asked about their knowledge and perception of AI on a 7-point Likert scale (-3, "strongly disagree"; +3, "strongly agree"). Self-generated identification codes allowed matching the same individuals pre-intervention and post-intervention, and using Wilcoxon signed rank test for paired data. RESULTS A total of 47/71 matched participants completed both surveys (66% follow-up rate) and were eligible for analysis (34 radiologists [72%], 13 traumatologists [28%], 15 women [32%]; mean age, 34.8 ± 7.8 years). Postintervention, there was an increase that AI "reduced missed findings" (1.28 [pre] vs 1.94 [post], P = 0.003) and made readers "safer" (1.21 vs 1.64, P = 0.048), but not "faster" (0.98 vs 1.21, P = 0.261). There was a rising disagreement that AI could "replace the radiological report" (-2.04 vs -2.34, P = 0.038), as well as an increase in self-reported knowledge about "clinical AI," its "chances," and its "risks" (0.40 vs 1.00, 1.21 vs 1.70, and 0.96 vs 1.34; all P 's ≤ 0.028). Radiologists used AI results more frequently than traumatologists ( P < 0.001) and rated benefits higher (all P 's ≤ 0.038), whereas senior physicians were less likely to use AI or endorse its benefits (negative correlation with age, -0.35 to 0.30; all P 's ≤ 0.046). CONCLUSIONS Implementing AI for emergency radiology into clinical routine has an educative aspect and underlines the concept of AI as a "second reader," to support and not replace physicians.
Collapse
Affiliation(s)
- Boj Friedrich Hoppe
- From the Department of Radiology, University Hospital, LMU Munich, Munich, Germany (B.F.J., J.Rueckel, Y.D., M.H., N.F., B.O.S., J.Ricke, J.Rudolph, C.C.C.); and Institute of Neuroradiology, University Hospital, LMU Munich, Munich, Germany (J.R.)
| | | | | | | | | | | | | | | | | |
Collapse
|
2
|
Calatrava M, de Irala J, Osorio A, Benítez E, Lopez-del Burgo C. Matched and Fully Private? A New Self-Generated Identification Code for School-Based Cohort Studies to Increase Perceived Anonymity. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 2022; 82:465-481. [PMID: 35444340 PMCID: PMC9014735 DOI: 10.1177/00131644211035436] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Anonymous questionnaires are frequently used in research with adolescents in order to obtain sincere answers about sensitive topics. Most longitudinal studies include self-generated identification codes (SGICs) to match information. Typical elements include a combination of letters and digits from personal data. However, these data may make the participant feel that their answers are not truly anonymous, and some studies using these types of SGICs have been perceived as not entirely anonymous by some participants. Furthermore, data protection laws could place limits on research carried out with these codes. The objective of our article is to test an SGIC with a higher degree of anonymity. We conducted two studies. In Study 1, we tested the perceived anonymity of this new SGIC code. Adolescents aged 12 to 18 years (N = 601) completed an anonymous questionnaire about lifestyles and risk behaviors, which also included the SGIC. Adolescents with and without risk behaviors were compared regarding whether or not they answered to the SGIC questions. We did not find any differences to suggest that participants felt identifiable. In Study 2, we assessed the efficiency of the new SGIC. At baseline, 123 students from two high schools (eighth grade) filled in questionnaires consisting of the new SGIC and their full names. Two years later, these same students (then in the 10th grade) were invited to fill in the same information again (116 students responded to this second call). A total of 97 students were present in both waves. The SGIC showed a moderate performance, with good enough indices of recall and precision. Evidence suggests that the new SGIC is a suitable tool for the anonymous matching of adolescents in follow-ups of school cohorts.
Collapse
Affiliation(s)
- Maria Calatrava
- Institute for Culture and Society. Universidad de Navarra, Pamplona, Navarra, Spain
- Navarra Institute for Health Research, Pamplona, Navarra, Spain
| | - Jokin de Irala
- Institute for Culture and Society. Universidad de Navarra, Pamplona, Navarra, Spain
- Navarra Institute for Health Research, Pamplona, Navarra, Spain
| | - Alfonso Osorio
- Institute for Culture and Society. Universidad de Navarra, Pamplona, Navarra, Spain
- Navarra Institute for Health Research, Pamplona, Navarra, Spain
- School of Education and Psychology. Universidad de Navarra, Pamplona, Navarra, Spain
| | - Edgar Benítez
- Institute for Culture and Society. Universidad de Navarra, Pamplona, Navarra, Spain
- Instituto de Ciencia de los Datos e Inteligencia Artificial. Universidad de Navarra, Pamplona, Navarra, Spain
| | - Cristina Lopez-del Burgo
- Institute for Culture and Society. Universidad de Navarra, Pamplona, Navarra, Spain
- Navarra Institute for Health Research, Pamplona, Navarra, Spain
| |
Collapse
|
3
|
Agley J, Tidd D, Jun M, Eldridge L, Xiao Y, Sussman S, Jayawardene W, Agley D, Gassman R, Dickinson SL. Developing and Validating a Novel Anonymous Method for Matching Longitudinal School-Based Data. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 2021; 81:90-109. [PMID: 33456063 PMCID: PMC7797962 DOI: 10.1177/0013164420938457] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
Prospective longitudinal data collection is an important way for researchers and evaluators to assess change. In school-based settings, for low-risk and/or likely-beneficial interventions or surveys, data quality and ethical standards are both arguably stronger when using a waiver of parental consent-but doing so often requires the use of anonymous data collection methods. The standard solution to this problem has been the use of a self-generated identification code. However, such codes often incorporate personalized elements (e.g., birth month, middle initial) that, even when meeting the technical standard for anonymity, may raise concerns among both youth participants and their parents, potentially altering willingness to participate, response quality, or generating outrage. There may be value, therefore, in developing a self-generated identification code and matching approach that not only is technically anonymous but also appears anonymous to a research-naive individual. This article provides a proof of concept for a novel matching approach for school-based longitudinal data collection that potentially accomplishes this goal.
Collapse
Affiliation(s)
- Jon Agley
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - David Tidd
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Mikyoung Jun
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Lori Eldridge
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Yunyu Xiao
- Silver School of Social Work, New York University, New York, NY, USA
| | - Steve Sussman
- Departments of Preventive Medicine and Psychology, and School of Social Work, University of Southern California, Los Angeles, CA, USA
| | - Wasantha Jayawardene
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Daniel Agley
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Ruth Gassman
- Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| | - Stephanie L. Dickinson
- Biostatistics Consulting Center, School of Public Health Bloomington, Indiana University Bloomington, Bloomington, IN, USA
| |
Collapse
|