Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors

Artificial Intelligence in Radiology: What Is Its True Role at Present, and Where Is the Evidence?

The integration of artificial intelligence (AI) in radiology has brought about substantial advancements and transformative potential in diagnostic imaging practices. This study presents an overview of the current research on the application of AI in radiology, highlighting key insights from recent studies and surveys. These recent studies have explored the expected impact of AI, encompassing machine learning and deep learning, on the work volume of diagnostic radiologists. The present and future role of AI in radiology holds great promise for enhancing diagnostic capabilities, improving workflow efficiency, and ultimately, advancing patient care.

Current Radiology workforce perspective on the integration of artificial intelligence in clinical practice: A systematic review

INTRODUCTION

Artificial Intelligence (AI) represents the application of computer systems to tasks traditionally performed by humans. The medical imaging profession has experienced a transformative shift through the integration of AI. While there have been several independent primary studies describing various aspects of AI, the current review employs a systematic approach towards describing the perspectives of radiologists and radiographers about the integration of AI in clinical practice. This review provides a holistic view from a professional standpoint towards understanding how the broad spectrum of AI tools are perceived as a unit in medical imaging practice.

METHODS

The study utilised a systematic review approach to collect data from quantitative, qualitative, and mixed-methods studies. Inclusion criteria encompassed articles concentrating on the viewpoints of either radiographers or radiologists regarding the incorporation of AI in medical imaging practice. A stepwise approach was employed in the systematic search across various databases. The included studies underwent quality assessment using the Quality Assessment Tool for Studies with Diverse Designs (QATSSD) checklist. A parallel-result convergent synthesis approach was employed to independently synthesise qualitative and quantitative evidence and to integrate the findings during the discussion phase.

RESULTS

Forty-one articles were included, all of which employed a cross-sectional study design. The main findings were themed around considerations and perspectives relating to AI education, impact on image quality and radiation dose, ethical and medico-legal implications for the use of AI, patient considerations and their perceived significance of AI for their care, and factors that influence development, implementation and job security. Despite varying emphasis, these themes collectively provide a global perspective on AI in medical imaging practice.

CONCLUSION

While expertise levels are varied and different, both radiographers and radiologists were generally optimistic about incorporation of AI in medical imaging practice. However, low levels of AI education and knowledge remain a critical barrier. Furthermore, equipment errors, cost, data security and operational difficulties, ethical constraints, job displacement concerns and insufficient implementation efforts are integration challenges that should merit the attention of stakeholders.

Evaluation of the Impact of Artificial Intelligence on Clinical Practice of Radiology in Saudi Arabia

Background

Artificial Intelligence (AI) is becoming integral to the health sector, particularly radiology, because it enhances diagnostic accuracy and optimizes patient care. This study aims to assess the awareness and acceptance of AI among radiology professionals in Saudi Arabia, identifying the educational and training needs to bridge knowledge gaps and enhance AI-related competencies.

Methods

This cross-sectional observational study surveyed radiology professionals across various hospitals in Saudi Arabia. Participants were recruited through multiple channels, including direct invitations, emails, social media, and professional societies. The survey comprised four sections: demographic details, perceptions of AI, knowledge about AI, and willingness to adopt AI in clinical practice.

Results

Out of 374 radiology professionals surveyed, 45.2% acknowledged AI's significant impact on their field. Approximately 44% showed enthusiasm for AI adoption. However, 58.6% reported limited AI knowledge and inadequate training, with 43.6% identifying skill development and the complexity of AI educational programs as major barriers to implementation.

Conclusion

While radiology professionals in Saudi Arabia are generally positive about integrating AI into clinical practice, significant gaps in knowledge and training need to be addressed. Tailored educational programs are essential to fully leverage AI's potential in improving medical imaging practices and patient care outcomes.

Enhancing a deep learning model for pulmonary nodule malignancy risk estimation in chest CT with uncertainty estimation

OBJECTIVE

To investigate the effect of uncertainty estimation on the performance of a Deep Learning (DL) algorithm for estimating malignancy risk of pulmonary nodules.

METHODS AND MATERIALS

In this retrospective study, we integrated an uncertainty estimation method into a previously developed DL algorithm for nodule malignancy risk estimation. Uncertainty thresholds were developed using CT data from the Danish Lung Cancer Screening Trial (DLCST), containing 883 nodules (65 malignant) collected between 2004 and 2010. We used thresholds on the 90th and 95th percentiles of the uncertainty score distribution to categorize nodules into certain and uncertain groups. External validation was performed on clinical CT data from a tertiary academic center containing 374 nodules (207 malignant) collected between 2004 and 2012. DL performance was measured using area under the ROC curve (AUC) for the full set of nodules, for the certain cases and for the uncertain cases. Additionally, nodule characteristics were compared to identify trends for inducing uncertainty.

RESULTS

The DL algorithm performed significantly worse in the uncertain group compared to the certain group of DLCST (AUC 0.62 (95% CI: 0.49, 0.76) vs 0.93 (95% CI: 0.88, 0.97); p < .001) and the clinical dataset (AUC 0.62 (95% CI: 0.50, 0.73) vs 0.90 (95% CI: 0.86, 0.94); p < .001). The uncertain group included larger benign nodules as well as more part-solid and non-solid nodules than the certain group.

CONCLUSION

The integrated uncertainty estimation showed excellent performance for identifying uncertain cases in which the DL-based nodule malignancy risk estimation algorithm had significantly worse performance.

CLINICAL RELEVANCE STATEMENT

Deep Learning algorithms often lack the ability to gauge and communicate uncertainty. For safe clinical implementation, uncertainty estimation is of pivotal importance to identify cases where the deep learning algorithm harbors doubt in its prediction.

KEY POINTS

• Deep learning (DL) algorithms often lack uncertainty estimation, which potentially reduce the risk of errors and improve safety during clinical adoption of the DL algorithm. • Uncertainty estimation identifies pulmonary nodules in which the discriminative performance of the DL algorithm is significantly worse. • Uncertainty estimation can further enhance the benefits of the DL algorithm and improve its safety and trustworthiness.

Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated Consolidated Framework for Implementation Research

OBJECTIVES

To identify the barriers and facilitators to the successful implementation of imaging-based diagnostic artificial intelligence (AI)-assisted decision-making software in China, using the updated Consolidated Framework for Implementation Research (CFIR) as a theoretical basis to develop strategies that promote effective implementation.

DESIGN

This qualitative study involved semistructured interviews with key stakeholders from both clinical settings and industry. Interview guide development, coding, analysis and reporting of findings were thoroughly informed by the updated CFIR.

SETTING

Four healthcare institutions in Beijing and Shanghai and two vendors of AI-assisted decision-making software for lung nodules detection and diabetic retinopathy screening were selected based on purposive sampling.

PARTICIPANTS

A total of 23 healthcare practitioners, 6 hospital informatics specialists, 4 hospital administrators and 7 vendors of the selected AI-assisted decision-making software were included in the study.

RESULTS

Within the 5 CFIR domains, 10 constructs were identified as barriers, 8 as facilitators and 3 as both barriers and facilitators. Major barriers included unsatisfactory clinical performance (Innovation); lack of collaborative network between primary and tertiary hospitals, lack of information security measures and certification (outer setting); suboptimal data quality, misalignment between software functions and goals of healthcare institutions (inner setting); unmet clinical needs (individuals). Key facilitators were strong empirical evidence of effectiveness, improved clinical efficiency (innovation); national guidelines related to AI, deployment of AI software in peer hospitals (outer setting); integration of AI software into existing hospital systems (inner setting) and involvement of clinicians (implementation process).

CONCLUSIONS

The study findings contributed to the ongoing exploration of AI integration in healthcare from the perspective of China, emphasising the need for a comprehensive approach considering both innovation-specific factors and the broader organisational and contextual dynamics. As China and other developing countries continue to advance in adopting AI technologies, the derived insights could further inform healthcare practitioners, industry stakeholders and policy-makers, guiding policies and practices that promote the successful implementation of imaging-based diagnostic AI-assisted decision-making software in healthcare for optimal patient care.

Artificial intelligence and machine learning applications for the imaging of bone and soft tissue tumors

Recent advancements in artificial intelligence (AI) and machine learning offer numerous opportunities in musculoskeletal radiology to potentially bolster diagnostic accuracy, workflow efficiency, and predictive modeling. AI tools have the capability to assist radiologists in many tasks ranging from image segmentation, lesion detection, and more. In bone and soft tissue tumor imaging, radiomics and deep learning show promise for malignancy stratification, grading, prognostication, and treatment planning. However, challenges such as standardization, data integration, and ethical concerns regarding patient data need to be addressed ahead of clinical translation. In the realm of musculoskeletal oncology, AI also faces obstacles in robust algorithm development due to limited disease incidence. While many initiatives aim to develop multitasking AI systems, multidisciplinary collaboration is crucial for successful AI integration into clinical practice. Robust approaches addressing challenges and embodying ethical practices are warranted to fully realize AI's potential for enhancing diagnostic accuracy and advancing patient care.

Barriers to and Facilitators of Artificial Intelligence Adoption in Health Care: Scoping Review

BACKGROUND

Artificial intelligence (AI) use cases in health care are on the rise, with the potential to improve operational efficiency and care outcomes. However, the translation of AI into practical, everyday use has been limited, as its effectiveness relies on successful implementation and adoption by clinicians, patients, and other health care stakeholders.

OBJECTIVE

As adoption is a key factor in the successful proliferation of an innovation, this scoping review aimed at presenting an overview of the barriers to and facilitators of AI adoption in health care.

METHODS

A scoping review was conducted using the guidance provided by the Joanna Briggs Institute and the framework proposed by Arksey and O'Malley. MEDLINE, IEEE Xplore, and ScienceDirect databases were searched to identify publications in English that reported on the barriers to or facilitators of AI adoption in health care. This review focused on articles published between January 2011 and December 2023. The review did not have any limitations regarding the health care setting (hospital or community) or the population (patients, clinicians, physicians, or health care administrators). A thematic analysis was conducted on the selected articles to map factors associated with the barriers to and facilitators of AI adoption in health care.

RESULTS

A total of 2514 articles were identified in the initial search. After title and abstract reviews, 50 (1.99%) articles were included in the final analysis. These articles were reviewed for the barriers to and facilitators of AI adoption in health care. Most articles were empirical studies, literature reviews, reports, and thought articles. Approximately 18 categories of barriers and facilitators were identified. These were organized sequentially to provide considerations for AI development, implementation, and the overall structure needed to facilitate adoption.

CONCLUSIONS

The literature review revealed that trust is a significant catalyst of adoption, and it was found to be impacted by several barriers identified in this review. A governance structure can be a key facilitator, among others, in ensuring all the elements identified as barriers are addressed appropriately. The findings demonstrate that the implementation of AI in health care is still, in many ways, dependent on the establishment of regulatory and legal frameworks. Further research into a combination of governance and implementation frameworks, models, or theories to enhance trust that would specifically enable adoption is needed to provide the necessary guidance to those translating AI research into practice. Future research could also be expanded to include attempts at understanding patients' perspectives on complex, high-risk AI use cases and how the use of AI applications affects clinical practice and patient care, including sociotechnical considerations, as more algorithms are implemented in actual clinical environments.

A comprehensive overview of barriers and strategies for AI implementation in healthcare: Mixed-method design

Implementation of artificial intelligence systems for healthcare is challenging. Understanding the barriers and implementation strategies can impact their adoption and allows for better anticipation and planning. This study's objective was to create a detailed inventory of barriers to and strategies for AI implementation in healthcare to support advancements in methods and implementation processes in healthcare. A sequential explanatory mixed method design was used. Firstly, scoping reviews and systematic literature reviews were identified using PubMed. Selected studies included empirical cases of AI implementation and use in clinical practice. As the reviews were deemed insufficient to fulfil the aim of the study, data collection shifted to the primary studies included in those reviews. The primary studies were screened by title and abstract, and thereafter read in full text. Then, data on barriers to and strategies for AI implementation were extracted from the included articles, thematically coded by inductive analysis, and summarized. Subsequently, a direct qualitative content analysis of 69 interviews with healthcare leaders and healthcare professionals confirmed and added results from the literature review. Thirty-eight empirical cases from the six identified scoping and literature reviews met the inclusion and exclusion criteria. Barriers to and strategies for AI implementation were grouped under three phases of implementation (planning, implementing, and sustaining the use) and were categorized into eleven concepts; Leadership, Buy-in, Change management, Engagement, Workflow, Finance and human resources, Legal, Training, Data, Evaluation and monitoring, Maintenance. Ethics emerged as a twelfth concept through qualitative analysis of the interviews. This study illustrates the inherent challenges and useful strategies in implementing AI in healthcare practice. Future research should explore various aspects of leadership, collaboration and contracts among key stakeholders, legal strategies surrounding clinicians' liability, solutions to ethical dilemmas, infrastructure for efficient integration of AI in workflows, and define decision points in the implementation process.

Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers

BACKGROUND

Efforts are underway to capitalize on the computational power of the data collected in electronic medical records (EMRs) to achieve a learning health system (LHS). Artificial intelligence (AI) in health care has promised to improve clinical outcomes, and many researchers are developing AI algorithms on retrospective data sets. Integrating these algorithms with real-time EMR data is rare. There is a poor understanding of the current enablers and barriers to empower this shift from data set-based use to real-time implementation of AI in health systems. Exploring these factors holds promise for uncovering actionable insights toward the successful integration of AI into clinical workflows.

OBJECTIVE

The first objective was to conduct a systematic literature review to identify the evidence of enablers and barriers regarding the real-world implementation of AI in hospital settings. The second objective was to map the identified enablers and barriers to a 3-horizon framework to enable the successful digital health transformation of hospitals to achieve an LHS.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were adhered to. PubMed, Scopus, Web of Science, and IEEE Xplore were searched for studies published between January 2010 and January 2022. Articles with case studies and guidelines on the implementation of AI analytics in hospital settings using EMR data were included. We excluded studies conducted in primary and community care settings. Quality assessment of the identified papers was conducted using the Mixed Methods Appraisal Tool and ADAPTE frameworks. We coded evidence from the included studies that related to enablers of and barriers to AI implementation. The findings were mapped to the 3-horizon framework to provide a road map for hospitals to integrate AI analytics.

RESULTS

Of the 1247 studies screened, 26 (2.09%) met the inclusion criteria. In total, 65% (17/26) of the studies implemented AI analytics for enhancing the care of hospitalized patients, whereas the remaining 35% (9/26) provided implementation guidelines. Of the final 26 papers, the quality of 21 (81%) was assessed as poor. A total of 28 enablers was identified; 8 (29%) were new in this study. A total of 18 barriers was identified; 5 (28%) were newly found. Most of these newly identified factors were related to information and technology. Actionable recommendations for the implementation of AI toward achieving an LHS were provided by mapping the findings to a 3-horizon framework.

CONCLUSIONS

Significant issues exist in implementing AI in health care. Shifting from validating data sets to working with live data is challenging. This review incorporated the identified enablers and barriers into a 3-horizon framework, offering actionable recommendations for implementing AI analytics to achieve an LHS. The findings of this study can assist hospitals in steering their strategic planning toward successful adoption of AI.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA

Artificial intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. KEY POINTS.

Errors in Radiology: A Standard Review

Radiological interpretations, while essential, are not infallible and are best understood as expert opinions formed through the evaluation of available evidence. Acknowledging the inherent possibility of error is crucial, as it frames the discussion on improving diagnostic accuracy and patient care. A comprehensive review of error classifications highlights the complexity of diagnostic errors, drawing on recent frameworks to categorize them into perceptual and cognitive errors, among others. This classification underpins an analysis of specific error types, their prevalence, and implications for clinical practice. Additionally, we address the psychological impact of radiological practice, including the effects of mental health and burnout on diagnostic accuracy. The potential of artificial intelligence (AI) in mitigating errors is discussed, alongside ethical and regulatory considerations in its application. This research contributes to the body of knowledge on radiological errors, offering insights into preventive strategies and the integration of AI to enhance diagnostic practices. It underscores the importance of a nuanced understanding of errors in radiology, aiming to foster improvements in patient care and radiological accuracy.

Nuclear medicine technologists practice impacted by AI denoising applications in PET/CT images

PURPOSE

Artificial intelligence (AI) in positron emission tomography/computed tomography (PET/CT) can be used to improve image quality when it is useful to reduce the injected activity or the acquisition time. Particular attention must be paid to ensure that users adopt this technological innovation when outcomes can be improved by its use. The aim of this study was to identify the aspects that need to be analysed and discussed to implement an AI denoising PET/CT algorithm in clinical practice, based on the representations of Nuclear Medicine Technologists (NMT) from Western-Switzerland, highlighting the barriers and facilitators associated.

METHODS

Two focus groups were organised in June and September 2023, involving ten voluntary participants recruited from all types of medical imaging departments, forming a diverse sample of NMT. The interview guide followed the first stage of the revised model of Ottawa of Research Use. A content analysis was performed following the three-stage approach described by Wanlin. Ethics cleared the study.

RESULTS

Clinical practice, workload, knowledge and resources were de 4 themes identified as necessary to be thought before implementing an AI denoising PET/CT algorithm by ten NMT participants (aged 31-60), not familiar with this AI tool. The main barriers to implement this algorithm included workflow challenges, resistance from professionals and lack of education; while the main facilitators were explanations and the availability of support to ask questions such as a "local champion".

CONCLUSION

To implement a denoising algorithm in PET/CT, several aspects of clinical practice need to be thought to reduce the barriers to its implementation such as the procedures, the workload and the available resources. Participants emphasised also the importance of clear explanations, education, and support for successful implementation.

IMPLICATIONS FOR PRACTICE

To facilitate the implementation of AI tools in clinical practice, it is important to identify the barriers and propose strategies that can mitigate it.

Clinical Integration of Artificial Intelligence for Breast Imaging

This article describes an approach to planning and implementing artificial intelligence products in a breast screening service. It highlights the importance of an in-depth understanding of the end-to-end workflow and effective project planning by a multidisciplinary team. It discusses the need for monitoring to ensure that performance is stable and meets expectations, as well as focusing on the potential for inadvertantly generating inequality. New cross-discipline roles and expertise will be needed to enhance service delivery.

Technology-enabled patient care in medical radiation sciences: the two sides of the coin

This is an exciting time to be working in healthcare and medical radiation sciences. This article discusses the potential benefits and risks of new technological interventions for patient benefit and outlines the need for co-production, governance and education to ensure these are used for advancing patients' well-being.

The impact of hepatic and splenic volumetric assessment in imaging for chronic liver disease: a narrative review

Chronic liver disease is responsible for significant morbidity and mortality worldwide. Abdominal computed tomography (CT) and magnetic resonance imaging (MRI) can fully visualise the liver and adjacent structures in the upper abdomen providing a reproducible assessment of the liver and biliary system and can detect features of portal hypertension. Subjective interpretation of CT and MRI in the assessment of liver parenchyma for early and advanced stages of fibrosis (pre-cirrhosis), as well as severity of portal hypertension, is limited. Quantitative and reproducible measurements of hepatic and splenic volumes have been shown to correlate with fibrosis staging, clinical outcomes, and mortality. In this review, we will explore the role of volumetric measurements in relation to diagnosis, assessment of severity and prediction of outcomes in chronic liver disease patients. We conclude that volumetric analysis of the liver and spleen can provide important information in such patients, has the potential to stratify patients' stage of hepatic fibrosis and disease severity, and can provide critical prognostic information. CRITICAL RELEVANCE STATEMENT: This review highlights the role of volumetric measurements of the liver and spleen using CT and MRI in relation to diagnosis, assessment of severity, and prediction of outcomes in chronic liver disease patients. KEY POINTS: Volumetry of the liver and spleen using CT and MRI correlates with hepatic fibrosis stages and cirrhosis. Volumetric measurements correlate with chronic liver disease outcomes. Fully automated methods for volumetry are required for implementation into routine clinical practice.

Radiologists' perspectives on the workflow integration of an artificial intelligence-based computer-aided detection system: A qualitative study

In healthcare, artificial intelligence (AI) is expected to improve work processes, yet most research focuses on the technical features of AI rather than its real-world clinical implementation. To evaluate the implementation process of an AI-based computer-aided detection system (AI-CAD) for prostate MRI readings, we interviewed German radiologists in a pre-post design. We embedded our findings in the Model of Workflow Integration and the Technology Acceptance Model to analyze workflow effects, facilitators, and barriers. The most prominent barriers were: (i) a time delay in the work process, (ii) additional work steps to be taken, and (iii) an unstable performance of the AI-CAD. Most frequently named facilitators were (i) good self-organization, and (ii) good usability of the software. Our results underline the importance of a holistic approach to AI implementation considering the sociotechnical work system and provide valuable insights into key factors of the successful adoption of AI technologies in work systems.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever‑growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi‑society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

Ethical use of artificial intelligence to prevent sudden cardiac death: an interview study of patient perspectives

BACKGROUND

The emergence of artificial intelligence (AI) in medicine has prompted the development of numerous ethical guidelines, while the involvement of patients in the creation of these documents lags behind. As part of the European PROFID project we explore patient perspectives on the ethical implications of AI in care for patients at increased risk of sudden cardiac death (SCD).

AIM

Explore perspectives of patients on the ethical use of AI, particularly in clinical decision-making regarding the implantation of an implantable cardioverter-defibrillator (ICD).

METHODS

Semi-structured, future scenario-based interviews were conducted among patients who had either an ICD and/or a heart condition with increased risk of SCD in Germany (n = 9) and the Netherlands (n = 15). We used the principles of the European Commission's Ethics Guidelines for Trustworthy AI to structure the interviews.

RESULTS

Six themes arose from the interviews: the ability of AI to rectify human doctors' limitations; the objectivity of data; whether AI can serve as second opinion; AI explainability and patient trust; the importance of the 'human touch'; and the personalization of care. Overall, our results reveal a strong desire among patients for more personalized and patient-centered care in the context of ICD implantation. Participants in our study express significant concerns about the further loss of the 'human touch' in healthcare when AI is introduced in clinical settings. They believe that this aspect of care is currently inadequately recognized in clinical practice. Participants attribute to doctors the responsibility of evaluating AI recommendations for clinical relevance and aligning them with patients' individual contexts and values, in consultation with the patient.

CONCLUSION

The 'human touch' patients exclusively ascribe to human medical practitioners extends beyond sympathy and kindness, and has clinical relevance in medical decision-making. Because this cannot be replaced by AI, we suggest that normative research into the 'right to a human doctor' is needed. Furthermore, policies on patient-centered AI integration in clinical practice should encompass the ethics of everyday practice rather than only principle-based ethics. We suggest that an empirical ethics approach grounded in ethnographic research is exceptionally well-suited to pave the way forward.

Attitudes towards artificial intelligence in emergency medicine

OBJECTIVE

To assess Australian and New Zealand emergency clinicians' attitudes towards the use of artificial intelligence (AI) in emergency medicine.

METHODS

We undertook a qualitative interview-based study based on grounded theory. Participants were recruited through ED internal mailing lists, the Australasian College for Emergency Medicine Bulletin, and the research teams' personal networks. Interviews were transcribed, coded and themes presented.

RESULTS

Twenty-five interviews were conducted between July 2021 and May 2022. Thematic saturation was achieved after 22 interviews. Most participants were from either Western Australia (52%) or Victoria (16%) and were consultants (96%). More participants reported feeling optimistic (10/25) than neutral (6/25), pessimistic (2/25) or mixed (7/25) towards the use of AI in the ED. A minority expressed scepticism regarding the feasibility or value of implementing AI into the ED. Multiple potential risks and ethical issues were discussed by participants including skill loss from overreliance on AI, algorithmic bias, patient privacy and concerns over liability. Participants also discussed perceived inadequacies in existing information technology systems. Participants felt that AI technologies would be used as decision support tools and not replace the roles of emergency clinicians. Participants were not concerned about the impact of AI on their job security. Most (17/25) participants thought that AI would impact emergency medicine within the next 10 years.

CONCLUSIONS

Emergency clinicians interviewed were generally optimistic about the use of AI in emergency medicine, so long as it is used as a decision support tool and they maintain the ability to override its recommendations.

Artificial Intelligence in Radiology: Opportunities and Challenges

Artificial intelligence's (AI) emergence in radiology elicits both excitement and uncertainty. AI holds promise for improving radiology with regards to clinical practice, education, and research opportunities. Yet, AI systems are trained on select datasets that can contain bias and inaccuracies. Radiologists must understand these limitations and engage with AI developers at every step of the process - from algorithm initiation and design to development and implementation - to maximize benefit and minimize harm that can be enabled by this technology.

A multinational study on artificial intelligence adoption: Clinical implementers' perspectives

BACKGROUND

Despite substantial progress in AI research for healthcare, translating research achievements to AI systems in clinical settings is challenging and, in many cases, unsatisfactory. As a result, many AI investments have stalled at the prototype level, never reaching clinical settings.

OBJECTIVE

To improve the chances of future AI implementation projects succeeding, we analyzed the experiences of clinical AI system implementers to better understand the challenges and success factors in their implementations.

METHODS

Thirty-seven implementers of clinical AI from European and North and South American countries were interviewed. Semi-structured interviews were transcribed and analyzed qualitatively with the framework method, identifying the success factors and the reasons for challenges as well as documenting proposals from implementers to improve AI adoption in clinical settings.

RESULTS

We gathered the implementers' requirements for facilitating AI adoption in the clinical setting. The main findings include 1) the lesser importance of AI explainability in favor of proper clinical validation studies, 2) the need to actively involve clinical practitioners, and not only clinical researchers, in the inception of AI research projects, 3) the need for better information structures and processes to manage data access and the ethical approval of AI projects, 4) the need for better support for regulatory compliance and avoidance of duplications in data management approval bodies, 5) the need to increase both clinicians' and citizens' literacy as respects the benefits and limitations of AI, and 6) the need for better funding schemes to support the implementation, embedding, and validation of AI in the clinical workflow, beyond pilots.

CONCLUSION

Participants in the interviews are positive about the future of AI in clinical settings. At the same time, they proposenumerous measures to transfer research advancesinto implementations that will benefit healthcare personnel. Transferring AI research into benefits for healthcare workers and patients requires adjustments in regulations, data access procedures, education, funding schemes, and validation of AI systems.

External validation of the RSNA 2020 pulmonary embolism detection challenge winning deep learning algorithm

PURPOSE

To evaluate the diagnostic performance and generalizability of the winning DL algorithm of the RSNA 2020 PE detection challenge to a local population using CTPA data from two hospitals.

MATERIALS AND METHODS

Consecutive CTPA images from patients referred for suspected PE were retrospectively analysed. The winning RSNA 2020 DL algorithm was retrained on the RSNA-STR Pulmonary Embolism CT (RSPECT) dataset. The algorithm was tested in hospital A on multidetector CT (MDCT) images of 238 patients and in hospital B on spectral detector CT (SDCT) and virtual monochromatic images (VMI) of 114 patients. The output of the DL algorithm was compared with a reference standard, which included a consensus reading by at least two experienced cardiothoracic radiologists for both hospitals. Areas under the receiver operating characteristic curve (AUCs) were calculated. Sensitivity and specificity were determined using the maximum Youden index.

RESULTS

According to the reference standard, PE was present in 73 patients (30.7%) in hospital A and 33 patients (29.0%) in hospital B. For the DL algorithm the AUC was 0.96 (95% CI 0.92-0.98) in hospital A, 0.89 (95% CI 0.81-0.94) for conventional reconstruction in hospital B and 0.87 (95% CI 0.80-0.93) for VMI.

CONCLUSION

The RSNA 2020 pulmonary embolism detection on CTPA challenge winning DL algorithm, retrained on the RSPECT dataset, showed high diagnostic accuracy on MDCT images. A somewhat lower performance was observed on SDCT images, which suggest additional training on novel CT technology may improve generalizability of this DL algorithm.

Advancements in Pancreatic Cancer Detection: Integrating Biomarkers, Imaging Technologies, and Machine Learning for Early Diagnosis

Artificial intelligence (AI) has come to play a pivotal role in revolutionizing medical practices, particularly in the field of pancreatic cancer detection and management. As a leading cause of cancer-related deaths, pancreatic cancer warrants innovative approaches due to its typically advanced stage at diagnosis and dismal survival rates. Present detection methods, constrained by limitations in accuracy and efficiency, underscore the necessity for novel solutions. AI-driven methodologies present promising avenues for enhancing early detection and prognosis forecasting. Through the analysis of imaging data, biomarker profiles, and clinical information, AI algorithms excel in discerning subtle abnormalities indicative of pancreatic cancer with remarkable precision. Moreover, machine learning (ML) algorithms facilitate the amalgamation of diverse data sources to optimize patient care. However, despite its huge potential, the implementation of AI in pancreatic cancer detection faces various challenges. Issues such as the scarcity of comprehensive datasets, biases in algorithm development, and concerns regarding data privacy and security necessitate thorough scrutiny. While AI offers immense promise in transforming pancreatic cancer detection and management, ongoing research and collaborative efforts are indispensable in overcoming technical hurdles and ethical dilemmas. This review delves into the evolution of AI, its application in pancreatic cancer detection, and the challenges and ethical considerations inherent in its integration.

Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines

The Consolidated Standards of Reporting Trials extension for Artificial Intelligence interventions (CONSORT-AI) was published in September 2020. Since its publication, several randomised controlled trials (RCTs) of AI interventions have been published but their completeness and transparency of reporting is unknown. This systematic review assesses the completeness of reporting of AI RCTs following publication of CONSORT-AI and provides a comprehensive summary of RCTs published in recent years. 65 RCTs were identified, mostly conducted in China (37%) and USA (18%). Median concordance with CONSORT-AI reporting was 90% (IQR 77-94%), although only 10 RCTs explicitly reported its use. Several items were consistently under-reported, including algorithm version, accessibility of the AI intervention or code, and references to a study protocol. Only 3 of 52 included journals explicitly endorsed or mandated CONSORT-AI. Despite a generally high concordance amongst recent AI RCTs, some AI-specific considerations remain systematically poorly reported. Further encouragement of CONSORT-AI adoption by journals and funders may enable more complete adoption of the full CONSORT-AI guidelines.

Value Creation Through Artificial Intelligence and Cardiovascular Imaging: A Scientific Statement From the American Heart Association

Multiple applications for machine learning and artificial intelligence (AI) in cardiovascular imaging are being proposed and developed. However, the processes involved in implementing AI in cardiovascular imaging are highly diverse, varying by imaging modality, patient subtype, features to be extracted and analyzed, and clinical application. This article establishes a framework that defines value from an organizational perspective, followed by value chain analysis to identify the activities in which AI might produce the greatest incremental value creation. The various perspectives that should be considered are highlighted, including clinicians, imagers, hospitals, patients, and payers. Integrating the perspectives of all health care stakeholders is critical for creating value and ensuring the successful deployment of AI tools in a real-world setting. Different AI tools are summarized, along with the unique aspects of AI applications to various cardiac imaging modalities, including cardiac computed tomography, magnetic resonance imaging, and positron emission tomography. AI is applicable and has the potential to add value to cardiovascular imaging at every step along the patient journey, from selecting the more appropriate test to optimizing image acquisition and analysis, interpreting the results for classification and diagnosis, and predicting the risk for major adverse cardiac events.

Radiomics and artificial intelligence for risk stratification of pulmonary nodules: Ready for primetime?

Pulmonary nodules are ubiquitously found on computed tomography (CT) imaging either incidentally or via lung cancer screening and require careful diagnostic evaluation and management to both diagnose malignancy when present and avoid unnecessary biopsy of benign lesions. To engage in this complex decision-making, clinicians must first risk stratify pulmonary nodules to determine what the best course of action should be. Recent developments in imaging technology, computer processing power, and artificial intelligence algorithms have yielded radiomics-based computer-aided diagnosis tools that use CT imaging data including features invisible to the naked human eye to predict pulmonary nodule malignancy risk and are designed to be used as a supplement to routine clinical risk assessment. These tools vary widely in their algorithm construction, internal and external validation populations, intended-use populations, and commercial availability. While several clinical validation studies have been published, robust clinical utility and clinical effectiveness data are not yet currently available. However, there is reason for optimism as ongoing and future studies aim to target this knowledge gap, in the hopes of improving the diagnostic process for patients with pulmonary nodules.

Radiology AI Deployment and Assessment Rubric (RADAR) to bring value-based AI into radiological practice

OBJECTIVE

To provide a comprehensive framework for value assessment of artificial intelligence (AI) in radiology.

METHODS

This paper presents the RADAR framework, which has been adapted from Fryback and Thornbury's imaging efficacy framework to facilitate the valuation of radiology AI from conception to local implementation. Local efficacy has been newly introduced to underscore the importance of appraising an AI technology within its local environment. Furthermore, the RADAR framework is illustrated through a myriad of study designs that help assess value.

RESULTS

RADAR presents a seven-level hierarchy, providing radiologists, researchers, and policymakers with a structured approach to the comprehensive assessment of value in radiology AI. RADAR is designed to be dynamic and meet the different valuation needs throughout the AI's lifecycle. Initial phases like technical and diagnostic efficacy (RADAR-1 and RADAR-2) are assessed pre-clinical deployment via in silico clinical trials and cross-sectional studies. Subsequent stages, spanning from diagnostic thinking to patient outcome efficacy (RADAR-3 to RADAR-5), require clinical integration and are explored via randomized controlled trials and cohort studies. Cost-effectiveness efficacy (RADAR-6) takes a societal perspective on financial feasibility, addressed via health-economic evaluations. The final level, RADAR-7, determines how prior valuations translate locally, evaluated through budget impact analysis, multi-criteria decision analyses, and prospective monitoring.

CONCLUSION

The RADAR framework offers a comprehensive framework for valuing radiology AI. Its layered, hierarchical structure, combined with a focus on local relevance, aligns RADAR seamlessly with the principles of value-based radiology.

CRITICAL RELEVANCE STATEMENT

The RADAR framework advances artificial intelligence in radiology by delineating a much-needed framework for comprehensive valuation.

KEYPOINTS

• Radiology artificial intelligence lacks a comprehensive approach to value assessment. • The RADAR framework provides a dynamic, hierarchical method for thorough valuation of radiology AI. • RADAR advances clinical radiology by bridging the artificial intelligence implementation gap.

Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

Navigating the doctor-patient-AI relationship - a mixed-methods study of physician attitudes toward artificial intelligence in primary care

BACKGROUND

Artificial intelligence (AI) is a rapidly advancing field that is beginning to enter the practice of medicine. Primary care is a cornerstone of medicine and deals with challenges such as physician shortage and burnout which impact patient care. AI and its application via digital health is increasingly presented as a possible solution. However, there is a scarcity of research focusing on primary care physician (PCP) attitudes toward AI. This study examines PCP views on AI in primary care. We explore its potential impact on topics pertinent to primary care such as the doctor-patient relationship and clinical workflow. By doing so, we aim to inform primary care stakeholders to encourage successful, equitable uptake of future AI tools. Our study is the first to our knowledge to explore PCP attitudes using specific primary care AI use cases rather than discussing AI in medicine in general terms.

METHODS

From June to August 2023, we conducted a survey among 47 primary care physicians affiliated with a large academic health system in Southern California. The survey quantified attitudes toward AI in general as well as concerning two specific AI use cases. Additionally, we conducted interviews with 15 survey respondents.

RESULTS

Our findings suggest that PCPs have largely positive views of AI. However, attitudes often hinged on the context of adoption. While some concerns reported by PCPs regarding AI in primary care focused on technology (accuracy, safety, bias), many focused on people-and-process factors (workflow, equity, reimbursement, doctor-patient relationship).

CONCLUSION

Our study offers nuanced insights into PCP attitudes towards AI in primary care and highlights the need for primary care stakeholder alignment on key issues raised by PCPs. AI initiatives that fail to address both the technological and people-and-process concerns raised by PCPs may struggle to make an impact.

A holistic approach to implementing artificial intelligence in radiology

OBJECTIVE

Despite the widespread recognition of the importance of artificial intelligence (AI) in healthcare, its implementation is often limited. This article aims to address this implementation gap by presenting insights from an in-depth case study of an organisation that approached AI implementation with a holistic approach.

MATERIALS AND METHODS

We conducted a longitudinal, qualitative case study of the implementation of AI in radiology at a large academic medical centre in the Netherlands for three years. Collected data consists of 43 days of work observations, 30 meeting observations, 18 interviews and 41 relevant documents. Abductive reasoning was used for systematic data analysis, which revealed three change initiative themes responding to specific AI implementation challenges.

RESULTS

This study identifies challenges of implementing AI in radiology at different levels and proposes a holistic approach to tackle those challenges. At the technology level, there is the issue of multiple narrow AI applications with no standard use interface; at the workflow level, AI results allow limited interaction with radiologists; at the people and organisational level, there are divergent expectations and limited experience with AI. The case of Southern illustrates that organisations can reap more benefits from AI implementation by investing in long-term initiatives that holistically align both social and technological aspects of clinical practice.

CONCLUSION

This study highlights the importance of a holistic approach to AI implementation that addresses challenges spanning technology, workflow, and organisational levels. Aligning change initiatives between these different levels has proven to be important to facilitate wide-scale implementation of AI in clinical practice.

CRITICAL RELEVANCE STATEMENT

Adoption of artificial intelligence is crucial for future-ready radiological care. This case study highlights the importance of a holistic approach that addresses technological, workflow, and organisational aspects, offering practical insights and solutions to facilitate successful AI adoption in clinical practice.

KEY POINTS

1. Practical and actionable insights into successful AI implementation in radiology are lacking. 2. Aligning technology, workflow, organisational aspects is crucial for a successful AI implementation 3. Holistic approach aids organisations to create sustainable value through AI implementation.

Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones.This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.Key points • The incorporation of artificial intelligence (AI) in radiological practice demands increased monitoring of its utility and safety.• Cooperation between developers, clinicians, and regulators will allow all involved to address ethical issues and monitor AI performance.• AI can fulfil its promise to advance patient well-being if all steps from development to integration in healthcare are rigorously evaluated.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement from the ACR, CAR, ESR, RANZCR and RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. This article is simultaneously published in Insights into Imaging (DOI 10.1186/s13244-023-01541-3), Journal of Medical Imaging and Radiation Oncology (DOI 10.1111/1754-9485.13612), Canadian Association of Radiologists Journal (DOI 10.1177/08465371231222229), Journal of the American College of Radiology (DOI 10.1016/j.jacr.2023.12.005), and Radiology: Artificial Intelligence (DOI 10.1148/ryai.230513). Keywords: Artificial Intelligence, Radiology, Automation, Machine Learning Published under a CC BY 4.0 license. ©The Author(s) 2024. Editor's Note: The RSNA Board of Directors has endorsed this article. It has not undergone review or editing by this journal.

Understanding the factors influencing acceptability of AI in medical imaging domains among healthcare professionals: A scoping review

BACKGROUND

Artificial intelligence (AI) technology has the potential to transform medical practice within the medical imaging industry and materially improve productivity and patient outcomes. However, low acceptability of AI as a digital healthcare intervention among medical professionals threatens to undermine user uptake levels, hinder meaningful and optimal value-added engagement, and ultimately prevent these promising benefits from being realised. Understanding the factors underpinning AI acceptability will be vital for medical institutions to pinpoint areas of deficiency and improvement within their AI implementation strategies. This scoping review aims to survey the literature to provide a comprehensive summary of the key factors influencing AI acceptability among healthcare professionals in medical imaging domains and the different approaches which have been taken to investigate them.

METHODS

A systematic literature search was performed across five academic databases including Medline, Cochrane Library, Web of Science, Compendex, and Scopus from January 2013 to September 2023. This was done in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines. Overall, 31 articles were deemed appropriate for inclusion in the scoping review.

RESULTS

The literature has converged towards three overarching categories of factors underpinning AI acceptability including: user factors involving trust, system understanding, AI literacy, and technology receptiveness; system usage factors entailing value proposition, self-efficacy, burden, and workflow integration; and socio-organisational-cultural factors encompassing social influence, organisational readiness, ethicality, and perceived threat to professional identity. Yet, numerous studies have overlooked a meaningful subset of these factors that are integral to the use of medical AI systems such as the impact on clinical workflow practices, trust based on perceived risk and safety, and compatibility with the norms of medical professions. This is attributable to reliance on theoretical frameworks or ad-hoc approaches which do not explicitly account for healthcare-specific factors, the novelties of AI as software as a medical device (SaMD), and the nuances of human-AI interaction from the perspective of medical professionals rather than lay consumer or business end users.

CONCLUSION

This is the first scoping review to survey the health informatics literature around the key factors influencing the acceptability of AI as a digital healthcare intervention in medical imaging contexts. The factors identified in this review suggest that existing theoretical frameworks used to study AI acceptability need to be modified to better capture the nuances of AI deployment in healthcare contexts where the user is a healthcare professional influenced by expert knowledge and disciplinary norms. Increasing AI acceptability among medical professionals will critically require designing human-centred AI systems which go beyond high algorithmic performance to consider accessibility to users with varying degrees of AI literacy, clinical workflow practices, the institutional and deployment context, and the cultural, ethical, and safety norms of healthcare professions. As investment into AI for healthcare increases, it would be valuable to conduct a systematic review and meta-analysis of the causal contribution of these factors to achieving high levels of AI acceptability among medical professionals.

Trust and stakeholder perspectives on the implementation of AI tools in clinical radiology

OBJECTIVES

To define requirements that condition trust in artificial intelligence (AI) as clinical decision support in radiology from the perspective of various stakeholders and to explore ways to fulfil these requirements.

METHODS

Semi-structured interviews were conducted with twenty-five respondents-nineteen directly involved in the development, implementation, or use of AI applications in radiology and six working with AI in other areas of healthcare. We designed the questions to explore three themes: development and use of AI, professional decision-making, and management and organizational procedures connected to AI. The transcribed interviews were analysed in an iterative coding process from open coding to theoretically informed thematic coding.

RESULTS

We identified four aspects of trust that relate to reliability, transparency, quality verification, and inter-organizational compatibility. These aspects fall under the categories of substantial and procedural requirements.

CONCLUSIONS

Development of appropriate levels of trust in AI in healthcare is complex and encompasses multiple dimensions of requirements. Various stakeholders will have to be involved in developing AI solutions for healthcare and radiology to fulfil these requirements.

CLINICAL RELEVANCE STATEMENT

For AI to achieve advances in radiology, it must be given the opportunity to support, rather than replace, human expertise. Support requires trust. Identification of aspects and conditions for trust allows developing AI implementation strategies that facilitate advancing the field.

KEY POINTS

• Dimensions of procedural and substantial demands that need to be fulfilled to foster appropriate levels of trust in AI in healthcare are conditioned on aspects related to reliability, transparency, quality verification, and inter-organizational compatibility. •Creating the conditions for trust to emerge requires the involvement of various stakeholders, who will have to compensate the problem's inherent complexity by finding and promoting well-defined solutions.

Added value of an artificial intelligence algorithm in reducing the number of missed incidental acute pulmonary embolism in routine portal venous phase chest CT

OBJECTIVES

The purpose of this study was to evaluate the incremental value of artificial intelligence (AI) compared to the diagnostic accuracy of radiologists alone in detecting incidental acute pulmonary embolism (PE) on routine portal venous contrast-enhanced chest computed tomography (CT).

METHODS

CTs of 3089 consecutive patients referred to the radiology department for a routine contrast-enhanced chest CT between 27-5-2020 and 31-12-2020, were retrospectively analysed by a CE-certified and FDA-approved AI algorithm. The diagnostic performance of the AI was compared to the initial report. To determine the reference standard, discordant findings were independently evaluated by two readers. In case of disagreement, another experienced cardiothoracic radiologist with knowledge of the initial report and the AI output adjudicated.

RESULTS

The prevalence of acute incidental PE in the reference standard was 2.2% (67 of 3089 patients). In 25 cases, AI detected initially unreported PE. This included three cases concerning central/lobar PE. Sensitivity of the AI algorithm was significantly higher than the outcome of the initial report (respectively 95.5% vs. 62.7%, p < 0.001), whereas specificity was very high for both (respectively 99.6% vs 99.9%, p = 0.012). The AI algorithm only showed a slightly higher amount of false-positive findings (11 vs. 2), resulting in a significantly lower PPV (85.3% vs. 95.5%, p = 0.047).

CONCLUSION

The AI algorithm showed high diagnostic accuracy in diagnosing incidental PE, detecting an additional 25 cases of initially unreported PE, accounting for 37.3% of all positive cases.

CLINICAL RELEVANCE STATEMENT

Radiologist support from AI algorithms in daily practice can prevent missed incidental acute PE on routine chest CT, without a high burden of false-positive cases.

KEY POINTS

• Incidental pulmonary embolism is often missed by radiologists in non-diagnostic scans with suboptimal contrast opacification within the pulmonary trunk. • An artificial intelligence algorithm showed higher sensitivity detecting incidental pulmonary embolism on routine portal venous chest CT compared to the initial report. • Implementation of artificial intelligence support in routine daily practice will reduce the number of missed incidental pulmonary embolism.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

Value Proposition of FDA-Approved Artificial Intelligence Algorithms for Neuroimaging

PURPOSE

The number of FDA-cleared artificial intelligence (AI) algorithms for neuroimaging has grown in the past decade. The adoption of these algorithms into clinical practice depends largely on whether this technology provides value in the clinical setting. The objective of this study was to analyze trends in FDA-cleared AI algorithms for neuroimaging and understand their value proposition as advertised by the AI developers and vendors.

METHODS

A list of AI algorithms cleared by the FDA for neuroimaging between May 2008 and August 2022 was extracted from the ACR Data Science Institute AI Central database. Product information for each device was collected from the database. For each device, information on the advertised value as presented on the developer's website was collected.

RESULTS

A total of 59 AI neuroimaging algorithms were cleared by the FDA between May 2008 and August 2022. Most of these algorithms (24 of 59) were compatible with noncontrast CT, 21 with MRI, 9 with CT perfusion, 8 with CT angiography, 3 with MR perfusion, and 2 with PET. Six algorithms were compatible with multiple imaging techniques. Of the 59 algorithms, websites were located that discussed the product value for 55 algorithms. The most widely advertised value proposition was improved quality of care (38 of 55 [69.1%]). A total of 24 algorithms (43.6%) proposed saving user time, 9 (15.7%) advertised decreased costs, and 6 (10.9%) described increased revenue. Product websites for 26 algorithms (43.6%) showed user testimonials advertising the value of the technology.

CONCLUSIONS

The results of this study indicate a wide range of value propositions advertised by developers and vendors of AI algorithms for neuroimaging. Most vendors advertised that their products would improve patient care. Further research is necessary to determine whether the value claimed by developers is actually demonstrated in clinical practice.

Multi-stakeholder preferences for the use of artificial intelligence in healthcare: A systematic review and thematic analysis

INTRODUCTION

Despite the proliferation of Artificial Intelligence (AI) technology over the last decade, clinician, patient, and public perceptions of its use in healthcare raise a number of ethical, legal and social questions. We systematically review the literature on attitudes towards the use of AI in healthcare from patients, the general public and health professionals' perspectives to understand these issues from multiple perspectives.

METHODOLOGY

A search for original research articles using qualitative, quantitative, and mixed methods published between 1 Jan 2001 to 24 Aug 2021 was conducted on six bibliographic databases. Data were extracted and classified into different themes representing views on: (i) knowledge and familiarity of AI, (ii) AI benefits, risks, and challenges, (iii) AI acceptability, (iv) AI development, (v) AI implementation, (vi) AI regulations, and (vii) Human - AI relationship.

RESULTS

The final search identified 7,490 different records of which 105 publications were selected based on predefined inclusion/exclusion criteria. While the majority of patients, the general public and health professionals generally had a positive attitude towards the use of AI in healthcare, all groups indicated some perceived risks and challenges. Commonly perceived risks included data privacy; reduced professional autonomy; algorithmic bias; healthcare inequities; and greater burnout to acquire AI-related skills. While patients had mixed opinions on whether healthcare workers suffer from job loss due to the use of AI, health professionals strongly indicated that AI would not be able to completely replace them in their professions. Both groups shared similar doubts about AI's ability to deliver empathic care. The need for AI validation, transparency, explainability, and patient and clinical involvement in the development of AI was emphasised. To help successfully implement AI in health care, most participants envisioned that an investment in training and education campaigns was necessary, especially for health professionals. Lack of familiarity, lack of trust, and regulatory uncertainties were identified as factors hindering AI implementation. Regarding AI regulations, key themes included data access and data privacy. While the general public and patients exhibited a willingness to share anonymised data for AI development, there remained concerns about sharing data with insurance or technology companies. One key domain under this theme was the question of who should be held accountable in the case of adverse events arising from using AI.

CONCLUSIONS

While overall positivity persists in attitudes and preferences toward AI use in healthcare, some prevalent problems require more attention. There is a need to go beyond addressing algorithm-related issues to look at the translation of legislation and guidelines into practice to ensure fairness, accountability, transparency, and ethics in AI.

Technical/Algorithm, Stakeholder, and Society (TASS) barriers to the application of artificial intelligence in medicine: A systematic review

INTRODUCTION

The use of artificial intelligence (AI), particularly machine learning and predictive analytics, has shown great promise in health care. Despite its strong potential, there has been limited use in health care settings. In this systematic review, we aim to determine the main barriers to successful implementation of AI in healthcare and discuss potential ways to overcome these challenges.

METHODS

We conducted a literature search in PubMed (1/1/2001-1/1/2023). The search was restricted to publications in the English language, and human study subjects. We excluded articles that did not discuss AI, machine learning, predictive analytics, and barriers to the use of these techniques in health care. Using grounded theory methodology, we abstracted concepts to identify major barriers to AI use in medicine.

RESULTS

We identified a total of 2,382 articles. After reviewing the 306 included papers, we developed 19 major themes, which we categorized into three levels: the Technical/Algorithm, Stakeholder, and Social levels (TASS). These themes included: Lack of Explainability, Need for Validation Protocols, Need for Standards for Interoperability, Need for Reporting Guidelines, Need for Standardization of Performance Metrics, Lack of Plan for Updating Algorithm, Job Loss, Skills Loss, Workflow Challenges, Loss of Patient Autonomy and Consent, Disturbing the Patient-Clinician Relationship, Lack of Trust in AI, Logistical Challenges, Lack of strategic plan, Lack of Cost-effectiveness Analysis and Proof of Efficacy, Privacy, Liability, Bias and Social Justice, and Education.

CONCLUSION

We identified 19 major barriers to the use of AI in healthcare and categorized them into three levels: the Technical/Algorithm, Stakeholder, and Social levels (TASS). Future studies should expand on barriers in pediatric care and focus on developing clearly defined protocols to overcome these barriers.

Can incorrect artificial intelligence (AI) results impact radiologists, and if so, what can we do about it? A multi-reader pilot study of lung cancer detection with chest radiography

OBJECTIVE

To examine whether incorrect AI results impact radiologist performance, and if so, whether human factors can be optimized to reduce error.

METHODS

Multi-reader design, 6 radiologists interpreted 90 identical chest radiographs (follow-up CT needed: yes/no) on four occasions (09/20-01/22). No AI result was provided for session 1. Sham AI results were provided for sessions 2-4, and AI for 12 cases were manipulated to be incorrect (8 false positives (FP), 4 false negatives (FN)) (0.87 ROC-AUC). In the Delete AI (No Box) condition, radiologists were told AI results would not be saved for the evaluation. In Keep AI (No Box) and Keep AI (Box), radiologists were told results would be saved. In Keep AI (Box), the ostensible AI program visually outlined the region of suspicion. AI results were constant between conditions.

RESULTS

Relative to the No AI condition (FN = 2.7%, FP = 51.4%), FN and FPs were higher in the Keep AI (No Box) (FN = 33.0%, FP = 86.0%), Delete AI (No Box) (FN = 26.7%, FP = 80.5%), and Keep AI (Box) (FN = to 20.7%, FP = 80.5%) conditions (all ps < 0.05). FNs were higher in the Keep AI (No Box) condition (33.0%) than in the Keep AI (Box) condition (20.7%) (p = 0.04). FPs were higher in the Keep AI (No Box) (86.0%) condition than in the Delete AI (No Box) condition (80.5%) (p = 0.03).

CONCLUSION

Incorrect AI causes radiologists to make incorrect follow-up decisions when they were correct without AI. This effect is mitigated when radiologists believe AI will be deleted from the patient's file or a box is provided around the region of interest.

CLINICAL RELEVANCE STATEMENT

When AI is wrong, radiologists make more errors than they would have without AI. Based on human factors psychology, our manuscript provides evidence for two AI implementation strategies that reduce the deleterious effects of incorrect AI.

KEY POINTS

• When AI provided incorrect results, false negative and false positive rates among the radiologists increased. • False positives decreased when AI results were deleted, versus kept, in the patient's record. • False negatives and false positives decreased when AI visually outlined the region of suspicion.

Radiology Residents' Perceptions of Artificial Intelligence: Nationwide Cross-Sectional Survey Study

BACKGROUND

Artificial intelligence (AI) is transforming various fields, with health care, especially diagnostic specialties such as radiology, being a key but controversial battleground. However, there is limited research systematically examining the response of "human intelligence" to AI.

OBJECTIVE

This study aims to comprehend radiologists' perceptions regarding AI, including their views on its potential to replace them, its usefulness, and their willingness to accept it. We examine the influence of various factors, encompassing demographic characteristics, working status, psychosocial aspects, personal experience, and contextual factors.

METHODS

Between December 1, 2020, and April 30, 2021, a cross-sectional survey was completed by 3666 radiology residents in China. We used multivariable logistic regression models to examine factors and associations, reporting odds ratios (ORs) and 95% CIs.

RESULTS

In summary, radiology residents generally hold a positive attitude toward AI, with 29.90% (1096/3666) agreeing that AI may reduce the demand for radiologists, 72.80% (2669/3666) believing AI improves disease diagnosis, and 78.18% (2866/3666) feeling that radiologists should embrace AI. Several associated factors, including age, gender, education, region, eye strain, working hours, time spent on medical images, resilience, burnout, AI experience, and perceptions of residency support and stress, significantly influence AI attitudes. For instance, burnout symptoms were associated with greater concerns about AI replacement (OR 1.89; P<.001), less favorable views on AI usefulness (OR 0.77; P=.005), and reduced willingness to use AI (OR 0.71; P<.001). Moreover, after adjusting for all other factors, perceived AI replacement (OR 0.81; P<.001) and AI usefulness (OR 5.97; P<.001) were shown to significantly impact the intention to use AI.

CONCLUSIONS

This study profiles radiology residents who are accepting of AI. Our comprehensive findings provide insights for a multidimensional approach to help physicians adapt to AI. Targeted policies, such as digital health care initiatives and medical education, can be developed accordingly.

A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare

Artificial intelligence (AI) is expected to improve healthcare outcomes by facilitating early diagnosis, reducing the medical administrative burden, aiding drug development, personalising medical and oncological management, monitoring healthcare parameters on an individual basis, and allowing clinicians to spend more time with their patients. In the post-pandemic world where there is a drive for efficient delivery of healthcare and manage long waiting times for patients to access care, AI has an important role in supporting clinicians and healthcare systems to streamline the care pathways and provide timely and high-quality care for the patients. Despite AI technologies being used in healthcare for some decades, and all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. This literature review highlighted barriers in six key areas: ethical, technological, liability and regulatory, workforce, social, and patient safety barriers. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare will enable clinical staff and healthcare leaders to overcome the identified hurdles and incorporate AI technologies for the benefit of patients and clinical staff.

Artificial intelligence and the work-health interface: A research agenda for a technologically transforming world of work

The labor market is undergoing a rapid artificial intelligence (AI) revolution. There is currently limited empirical scholarship that focuses on how AI adoption affects employment opportunities and work environments in ways that shape worker health, safety, well-being and equity. In this article, we present an agenda to guide research examining the implications of AI on the intersection between work and health. To build the agenda, a full day meeting was organized and attended by 50 participants including researchers from diverse disciplines and applied stakeholders. Facilitated meeting discussions aimed to set research priorities related to workplace AI applications and its impact on the health of workers, including critical research questions, methodological approaches, data needs, and resource requirements. Discussions also aimed to identify groups of workers and working contexts that may benefit from AI adoption as well as those that may be disadvantaged by AI. Discussions were synthesized into four research agenda areas: (1) examining the impact of stronger AI on human workers; (2) advancing responsible and healthy AI; (3) informing AI policy for worker health, safety, well-being, and equitable employment; and (4) understanding and addressing worker and employer knowledge needs regarding AI applications. The agenda provides a roadmap for researchers to build a critical evidence base on the impact of AI on workers and workplaces, and will ensure that worker health, safety, well-being, and equity are at the forefront of workplace AI system design and adoption.

An experimental machine learning study investigating the decision-making process of students and qualified radiographers when interpreting radiographic images

AI is becoming more prevalent in healthcare and is predicted to be further integrated into workflows to ease the pressure on an already stretched service. The National Health Service in the UK has prioritised AI and Digital health as part of its Long-Term Plan. Few studies have examined the human interaction with such systems in healthcare, despite reports of biases being present with the use of AI in other technologically advanced fields, such as finance and aviation. Understanding is needed of how certain user characteristics may impact how radiographers engage with AI systems in use in the clinical setting to mitigate against problems before they arise. The aim of this study is to determine correlations of skills, confidence in AI and perceived knowledge amongst student and qualified radiographers in the UK healthcare system. A machine learning based AI model was built to predict if the interpreter was either a student (n = 67) or a qualified radiographer (n = 39) in advance, using important variables from a feature selection technique named Boruta. A survey, which required the participant to interpret a series of plain radiographic examinations with and without AI assistance, was created on the Qualtrics survey platform and promoted via social media (Twitter/LinkedIn), therefore adopting convenience, snowball sampling This survey was open to all UK radiographers, including students and retired radiographers. Pearson's correlation analysis revealed that males who were proficient in their profession were more likely than females to trust AI. Trust in AI was negatively correlated with age and with level of experience. A machine learning model was built, the best model predicted the image interpreter to be qualified radiographers with 0.93 area under curve and a prediction accuracy of 93%. Further testing in prospective validation cohorts using a larger sample size is required to determine the clinical utility of the proposed machine learning model.

Evaluation of acceptance, attitude, and knowledge towards artificial intelligence and its application from the point of view of physicians and nurses: A provincial survey study in Iran: A cross-sectional descriptive-analytical study

Background and Aims

The prospect of using artificial intelligence (AI) in healthcare is bright and promising, and its use can have a significant impact on cost reduction and decrease the possibility of error and negligence among healthcare workers. This study aims to investigate the level of knowledge, attitude, and acceptance among Iranian physicians and nurses.

Methods

This cross-sectional descriptive-analytical study was conducted in eight public university hospitals located in Tehran on 400 physicians and nurses. To conduct the study, convenient sampling was used with the help of researcher-made questionnaires. Statistical analysis was done by SPSS 21 The mean and standard deviation and Chi-square and Fisher's exact tests were used.

Results

In this study, the level of knowledge among the research subjects was average (14.66 ± 4.53), the level of their attitude toward AI was relatively favorable (47.81 ± 6.74), and their level of acceptance of AI was average (103.19 ± 13.70). Moreover, from the participant's perspective, AI in medicine is most widely used in increasing the accuracy of diagnostic tests (86.5%), identifying drug interactions (82.75%), and helping to analyze medical tests and imaging (80%). There was a statistically significant relationship between the variable of acceptance of AI and the participant's level of education (p = 0.028), participation in an AI training course (p = 0.022), and the hospital department where they worked (p < 0.001).

Conclusion

In this study, both the knowledge and the acceptance of the participants towards AI were proved to be at an average level and the attitude towards AI was relatively favorable, which is in contrast with the very rapid and inevitable expansion of AI. Although our participants were aware of the growing use of AI in medicine, they had a cautious attitude toward this.

Analyzing Barriers and Enablers for the Acceptance of Artificial Intelligence Innovations into Radiology Practice: A Scoping Review

OBJECTIVES

This scoping review was conducted to determine the barriers and enablers associated with the acceptance of artificial intelligence/machine learning (AI/ML)-enabled innovations into radiology practice from a physician's perspective.

METHODS

A systematic search was performed using Ovid Medline and Embase. Keywords were used to generate refined queries with the inclusion of computer-aided diagnosis, artificial intelligence, and barriers and enablers. Three reviewers assessed the articles, with a fourth reviewer used for disagreements. The risk of bias was mitigated by including both quantitative and qualitative studies.

RESULTS

An electronic search from January 2000 to 2023 identified 513 studies. Twelve articles were found to fulfill the inclusion criteria: qualitative studies (n = 4), survey studies (n = 7), and randomized controlled trials (RCT) (n = 1). Among the most common barriers to AI implementation into radiology practice were radiologists' lack of acceptance and trust in AI innovations; a lack of awareness, knowledge, and familiarity with the technology; and perceived threat to the professional autonomy of radiologists. The most important identified AI implementation enablers were high expectations of AI's potential added value; the potential to decrease errors in diagnosis; the potential to increase efficiency when reaching a diagnosis; and the potential to improve the quality of patient care.

CONCLUSIONS

This scoping review found that few studies have been designed specifically to identify barriers and enablers to the acceptance of AI in radiology practice. The majority of studies have assessed the perception of AI replacing radiologists, rather than other barriers or enablers in the adoption of AI. To comprehensively evaluate the potential advantages and disadvantages of integrating AI innovations into radiology practice, gathering more robust research evidence on stakeholder perspectives and attitudes is essential.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.