Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

Facilitating public involvement in research about healthcare AI: A scoping review of empirical methods

OBJECTIVE

With the recent increase in research into public views on healthcare artificial intelligence (HCAI), the objective of this review is to examine the methods of empirical studies on public views on HCAI. We map how studies provided participants with information about HCAI, and we examine the extent to which studies framed publics as active contributors to HCAI governance.

MATERIALS AND METHODS

We searched 5 academic databases and Google Advanced for empirical studies investigating public views on HCAI. We extracted information including study aims, research instruments, and recommendations.

RESULTS

Sixty-two studies were included. Most were quantitative (N = 42). Most (N = 47) reported providing participants with background information about HCAI. Despite this, studies often reported participants' lack of prior knowledge about HCAI as a limitation. Over three quarters (N = 48) of the studies made recommendations that envisaged public views being used to guide governance of AI.

DISCUSSION

Provision of background information is an important component of facilitating research with publics on HCAI. The high proportion of studies reporting participants' lack of knowledge about HCAI as a limitation reflects the need for more guidance on how information should be presented. A minority of studies adopted technocratic positions that construed publics as passive beneficiaries of AI, rather than as active stakeholders in HCAI design and implementation.

CONCLUSION

This review draws attention to how public roles in HCAI governance are constructed in empirical studies. To facilitate active participation, we recommend that research with publics on HCAI consider methodological designs that expose participants to diverse information sources.

Frameworks for procurement, integration, monitoring, and evaluation of artificial intelligence tools in clinical settings: A systematic review

Research on the applications of artificial intelligence (AI) tools in medicine has increased exponentially over the last few years but its implementation in clinical practice has not seen a commensurate increase with a lack of consensus on implementing and maintaining such tools. This systematic review aims to summarize frameworks focusing on procuring, implementing, monitoring, and evaluating AI tools in clinical practice. A comprehensive literature search, following PRSIMA guidelines was performed on MEDLINE, Wiley Cochrane, Scopus, and EBSCO databases, to identify and include articles recommending practices, frameworks or guidelines for AI procurement, integration, monitoring, and evaluation. From the included articles, data regarding study aim, use of a framework, rationale of the framework, details regarding AI implementation involving procurement, integration, monitoring, and evaluation were extracted. The extracted details were then mapped on to the Donabedian Plan, Do, Study, Act cycle domains. The search yielded 17,537 unique articles, out of which 47 were evaluated for inclusion based on their full texts and 25 articles were included in the review. Common themes extracted included transparency, feasibility of operation within existing workflows, integrating into existing workflows, validation of the tool using predefined performance indicators and improving the algorithm and/or adjusting the tool to improve performance. Among the four domains (Plan, Do, Study, Act) the most common domain was Plan (84%, n = 21), followed by Study (60%, n = 15), Do (52%, n = 13), & Act (24%, n = 6). Among 172 authors, only 1 (0.6%) was from a low-income country (LIC) and 2 (1.2%) were from lower-middle-income countries (LMICs). Healthcare professionals cite the implementation of AI tools within clinical settings as challenging owing to low levels of evidence focusing on integration in the Do and Act domains. The current healthcare AI landscape calls for increased data sharing and knowledge translation to facilitate common goals and reap maximum clinical benefit.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial

OBJECTIVES

We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2).

MATERIALS AND METHODS

A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected.

RESULTS

Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians.

DISCUSSION

Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved.

CONCLUSION

The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century

As healthcare systems around the world face challenges such as escalating costs, limited access, and growing demand for personalized care, artificial intelligence (AI) is emerging as a key force for transformation. This review is motivated by the urgent need to harness AI's potential to mitigate these issues and aims to critically assess AI's integration in different healthcare domains. We explore how AI empowers clinical decision-making, optimizes hospital operation and management, refines medical image analysis, and revolutionizes patient care and monitoring through AI-powered wearables. Through several case studies, we review how AI has transformed specific healthcare domains and discuss the remaining challenges and possible solutions. Additionally, we will discuss methodologies for assessing AI healthcare solutions, ethical challenges of AI deployment, and the importance of data privacy and bias mitigation for responsible technology use. By presenting a critical assessment of AI's transformative potential, this review equips researchers with a deeper understanding of AI's current and future impact on healthcare. It encourages an interdisciplinary dialogue between researchers, clinicians, and technologists to navigate the complexities of AI implementation, fostering the development of AI-driven solutions that prioritize ethical standards, equity, and a patient-centered approach.

Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study

BACKGROUND

Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation.

OBJECTIVE

This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England.

METHODS

This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote.

RESULTS

Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024.

CONCLUSIONS

The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50568.

The Impact of Digital Hospitals on Patient and Clinician Experience: Systematic Review and Qualitative Evidence Synthesis

BACKGROUND

The digital transformation of health care is advancing rapidly. A well-accepted framework for health care improvement is the Quadruple Aim: improved clinician experience, improved patient experience, improved population health, and reduced health care costs. Hospitals are attempting to improve care by using digital technologies, but the effectiveness of these technologies is often only measured against cost and quality indicators, and less is known about the clinician and patient experience.

OBJECTIVE

This study aims to conduct a systematic review and qualitative evidence synthesis to assess the clinician and patient experience of digital hospitals.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and ENTREQ (Enhancing the Transparency in Reporting the Synthesis of Qualitative Research) guidelines were followed. The PubMed, Embase, Scopus, CINAHL, and PsycINFO databases were searched from January 2010 to June 2022. Studies that explored multidisciplinary clinician or adult inpatient experiences of digital hospitals (with a full electronic medical record) were included. Study quality was assessed using the Mixed Methods Appraisal Tool. Data synthesis was performed narratively for quantitative studies. Qualitative evidence synthesis was performed via (1) automated machine learning text analytics using Leximancer (Leximancer Pty Ltd) and (2) researcher-led inductive synthesis to generate themes.

RESULTS

A total of 61 studies (n=39, 64% quantitative; n=15, 25% qualitative; and n=7, 11% mixed methods) were included. Most studies (55/61, 90%) investigated clinician experiences, whereas few (10/61, 16%) investigated patient experiences. The study populations ranged from 8 to 3610 clinicians, 11 to 34,425 patients, and 5 to 2836 hospitals. Quantitative outcomes indicated that clinicians had a positive overall satisfaction (17/24, 71% of the studies) with digital hospitals, and most studies (11/19, 58%) reported a positive sentiment toward usability. Data accessibility was reported positively, whereas adaptation, clinician-patient interaction, and workload burnout were reported negatively. The effects of digital hospitals on patient safety and clinicians' ability to deliver patient care were mixed. The qualitative evidence synthesis of clinician experience studies (18/61, 30%) generated 7 themes: inefficient digital documentation, inconsistent data quality, disruptions to conventional health care relationships, acceptance, safety versus risk, reliance on hybrid (digital and paper) workflows, and patient data privacy. There was weak evidence of a positive association between digital hospitals and patient satisfaction scores.

CONCLUSIONS

Clinicians' experience of digital hospitals appears positive according to high-level indicators (eg, overall satisfaction and data accessibility), but the qualitative evidence synthesis revealed substantive tensions. There is insufficient evidence to draw a definitive conclusion on the patient experience within digital hospitals, but indications appear positive or agnostic. Future research must prioritize equitable investigation and definition of the digital clinician and patient experience to achieve the Quadruple Aim of health care.

Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review

Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.

Perceptions on artificial intelligence-based decision-making for coexisting multiple long-term health conditions: protocol for a qualitative study with patients and healthcare professionals

INTRODUCTION

Coexisting multiple health conditions is common among older people, a population that is increasing globally. The potential for polypharmacy, adverse events, drug interactions and development of additional health conditions complicates prescribing decisions for these patients. Artificial intelligence (AI)-generated decision-making tools may help guide clinical decisions in the context of multiple health conditions, by determining which of the multiple medication options is best. This study aims to explore the perceptions of healthcare professionals (HCPs) and patients on the use of AI in the management of multiple health conditions.

METHODS AND ANALYSIS

A qualitative study will be conducted using semistructured interviews. Adults (≥18 years) with multiple health conditions living in the West Midlands of England and HCPs with experience in caring for patients with multiple health conditions will be eligible and purposively sampled. Patients will be identified from Clinical Practice Research Datalink (CPRD) Aurum; CPRD will contact general practitioners who will in turn, send a letter to patients inviting them to take part. Eligible HCPs will be recruited through British HCP bodies and known contacts. Up to 30 patients and 30 HCPs will be recruited, until data saturation is achieved. Interviews will be in-person or virtual, audio recorded and transcribed verbatim. The topic guide is designed to explore participants' attitudes towards AI-informed clinical decision-making to augment clinician-directed decision-making, the perceived advantages and disadvantages of both methods and attitudes towards risk management. Case vignettes comprising a common decision pathway for patients with multiple health conditions will be presented during each interview to invite participants' opinions on how their experiences compare. Data will be analysed thematically using the Framework Method.

ETHICS AND DISSEMINATION

This study has been approved by the National Health Service Research Ethics Committee (Reference: 22/SC/0210). Written informed consent or verbal consent will be obtained prior to each interview. The findings from this study will be disseminated through peer-reviewed publications, conferences and lay summaries.

Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study

BACKGROUND

Machine learning (ML)-driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes.

OBJECTIVE

This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care.

METHODS

A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient's primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team.

RESULTS

Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups.

CONCLUSIONS

Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians' needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD.

Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study

BACKGROUND

Chatbots are being piloted to draft responses to patient questions, but patients' ability to distinguish between provider and chatbot responses and patients' trust in chatbots' functions are not well established.

OBJECTIVE

This study aimed to assess the feasibility of using ChatGPT (Chat Generative Pre-trained Transformer) or a similar artificial intelligence-based chatbot for patient-provider communication.

METHODS

A survey study was conducted in January 2023. Ten representative, nonadministrative patient-provider interactions were extracted from the electronic health record. Patients' questions were entered into ChatGPT with a request for the chatbot to respond using approximately the same word count as the human provider's response. In the survey, each patient question was followed by a provider- or ChatGPT-generated response. Participants were informed that 5 responses were provider generated and 5 were chatbot generated. Participants were asked-and incentivized financially-to correctly identify the response source. Participants were also asked about their trust in chatbots' functions in patient-provider communication, using a Likert scale from 1-5.

RESULTS

A US-representative sample of 430 study participants aged 18 and older were recruited on Prolific, a crowdsourcing platform for academic studies. In all, 426 participants filled out the full survey. After removing participants who spent less than 3 minutes on the survey, 392 respondents remained. Overall, 53.3% (209/392) of respondents analyzed were women, and the average age was 47.1 (range 18-91) years. The correct classification of responses ranged between 49% (192/392) to 85.7% (336/392) for different questions. On average, chatbot responses were identified correctly in 65.5% (1284/1960) of the cases, and human provider responses were identified correctly in 65.1% (1276/1960) of the cases. On average, responses toward patients' trust in chatbots' functions were weakly positive (mean Likert score 3.4 out of 5), with lower trust as the health-related complexity of the task in the questions increased.

CONCLUSIONS

ChatGPT responses to patient questions were weakly distinguishable from provider responses. Laypeople appear to trust the use of chatbots to answer lower-risk health questions. It is important to continue studying patient-chatbot interaction as chatbots move from administrative to more clinical roles in health care.

Evaluating the translation of implementation science to clinical artificial intelligence: a bibliometric study of qualitative research

Introduction

Whilst a theoretical basis for implementation research is seen as advantageous, there is little clarity over if and how the application of theories, models or frameworks (TMF) impact implementation outcomes. Clinical artificial intelligence (AI) continues to receive multi-stakeholder interest and investment, yet a significant implementation gap remains. This bibliometric study aims to measure and characterize TMF application in qualitative clinical AI research to identify opportunities to improve research practice and its impact on clinical AI implementation.

Methods

Qualitative research of stakeholder perspectives on clinical AI published between January 2014 and October 2022 was systematically identified. Eligible studies were characterized by their publication type, clinical and geographical context, type of clinical AI studied, data collection method, participants and application of any TMF. Each TMF applied by eligible studies, its justification and mode of application was characterized.

Results

Of 202 eligible studies, 70 (34.7%) applied a TMF. There was an 8-fold increase in the number of publications between 2014 and 2022 but no significant increase in the proportion applying TMFs. Of the 50 TMFs applied, 40 (80%) were only applied once, with the Technology Acceptance Model applied most frequently (n = 9). Seven TMFs were novel contributions embedded within an eligible study. A minority of studies justified TMF application (n = 51,58.6%) and it was uncommon to discuss an alternative TMF or the limitations of the one selected (n = 11,12.6%). The most common way in which a TMF was applied in eligible studies was data analysis (n = 44,50.6%). Implementation guidelines or tools were explicitly referenced by 2 reports (1.0%).

Conclusion

TMFs have not been commonly applied in qualitative research of clinical AI. When TMFs have been applied there has been (i) little consensus on TMF selection (ii) limited description of selection rationale and (iii) lack of clarity over how TMFs inform research. We consider this to represent an opportunity to improve implementation science's translation to clinical AI research and clinical AI into practice by promoting the rigor and frequency of TMF application. We recommend that the finite resources of the implementation science community are diverted toward increasing accessibility and engagement with theory informed practices. The considered application of theories, models and frameworks (TMF) are thought to contribute to the impact of implementation science on the translation of innovations into real-world care. The frequency and nature of TMF use are yet to be described within digital health innovations, including the prominent field of clinical AI. A well-known implementation gap, coined as the "AI chasm" continues to limit the impact of clinical AI on real-world care. From this bibliometric study of the frequency and quality of TMF use within qualitative clinical AI research, we found that TMFs are usually not applied, their selection is highly varied between studies and there is not often a convincing rationale for their selection. Promoting the rigor and frequency of TMF use appears to present an opportunity to improve the translation of clinical AI into practice.

Commentary: Patient Perspectives on Artificial Intelligence; What have We Learned and How Should We Move Forward?

Artificial intelligence (AI) in healthcare has now begun to make its contributions to real-world patient care with varying degrees of both public and clinical acceptability around it. The heavy investment from governments, industry and academia needed to reach this point has helped to surface different perspectives on AI. As clinical AI applications become a reality, however, there is an increasing need to harness and integrate patient perspectives, which address the distinct needs of different populations, healthcare systems and clinical problems more closely. Despite this need, patient perspectives on AI implementation have little presence in academic literature and within implementation science and are not sufficiently considered throughout the MedTech and eHealthtech product development cycle, which brings its own challenges and opportunities. This joint patient expert/clinician commentary aims to briefly summarise views on AI. It reflects upon recommendations on how stakeholders such as clinicians and Health & MedTech small and medium-sized enterprises (SMEs) can make practical usage of these views. The recommendations of the authors centre around how to work better with patients to enable both product centric and patient centric innovation and person-centred care.

The Clinical Researcher Journey in the Artificial Intelligence Era: The PAC-MAN’s Challenge

Artificial intelligence (AI) is a powerful tool that can assist researchers and clinicians in various settings. However, like any technology, it must be used with caution and awareness as there are numerous potential pitfalls. To provide a creative analogy, we have likened research to the PAC-MAN classic arcade video game. Just as the protagonist of the game is constantly seeking data, researchers are constantly seeking information that must be acquired and managed within the constraints of the research rules. In our analogy, the obstacles that researchers face are represented by “ghosts”, which symbolize major ethical concerns, low-quality data, legal issues, and educational challenges. In short, clinical researchers need to meticulously collect and analyze data from various sources, often navigating through intricate and nuanced challenges to ensure that the data they obtain are both precise and pertinent to their research inquiry. Reflecting on this analogy can foster a deeper comprehension of the significance of employing AI and other powerful technologies with heightened awareness and attentiveness.

Safety and efficacy of an artificial intelligence-enabled decision tool for treatment decisions in neovascular age-related macular degeneration and an exploration of clinical pathway integration and implementation: protocol for a multi-methods validation study

INTRODUCTION

Neovascular age-related macular degeneration (nAMD) management is one of the largest single-disease contributors to hospital outpatient appointments. Partial automation of nAMD treatment decisions could reduce demands on clinician time. Established artificial intelligence (AI)-enabled retinal imaging analysis tools, could be applied to this use-case, but are not yet validated for it. A primary qualitative investigation of stakeholder perceptions of such an AI-enabled decision tool is also absent. This multi-methods study aims to establish the safety and efficacy of an AI-enabled decision tool for nAMD treatment decisions and understand where on the clinical pathway it could sit and what factors are likely to influence its implementation.

METHODS AND ANALYSIS

Single-centre retrospective imaging and clinical data will be collected from nAMD clinic visits at a National Health Service (NHS) teaching hospital ophthalmology service, including judgements of nAMD disease stability or activity made in real-world consultant-led-care. Dataset size will be set by a power calculation using the first 127 randomly sampled eligible clinic visits. An AI-enabled retinal segmentation tool and a rule-based decision tree will independently analyse imaging data to report nAMD stability or activity for each of these clinic visits. Independently, an external reading centre will receive both clinical and imaging data to generate an enhanced reference standard for each clinic visit. The non-inferiority of the relative negative predictive value of AI-enabled reports on disease activity relative to consultant-led-care judgements will then be tested. In parallel, approximately 40 semi-structured interviews will be conducted with key nAMD service stakeholders, including patients. Transcripts will be coded using a theoretical framework and thematic analysis will follow.

ETHICS AND DISSEMINATION

NHS Research Ethics Committee and UK Health Research Authority approvals are in place (21/NW/0138). Informed consent is planned for interview participants only. Written and oral dissemination is planned to public, clinical, academic and commercial stakeholders.

Artificial intelligence in healthcare: Complementing, not replacing, doctors and healthcare providers

The utilization of artificial intelligence (AI) in clinical practice has increased and is evidently contributing to improved diagnostic accuracy, optimized treatment planning, and improved patient outcomes. The rapid evolution of AI, especially generative AI and large language models (LLMs), have reignited the discussions about their potential impact on the healthcare industry, particularly regarding the role of healthcare providers. Concerning questions, "can AI replace doctors?" and "will doctors who are using AI replace those who are not using it?" have been echoed. To shed light on this debate, this article focuses on emphasizing the augmentative role of AI in healthcare, underlining that AI is aimed to complement, rather than replace, doctors and healthcare providers. The fundamental solution emerges with the human-AI collaboration, which combines the cognitive strengths of healthcare providers with the analytical capabilities of AI. A human-in-the-loop (HITL) approach ensures that the AI systems are guided, communicated, and supervised by human expertise, thereby maintaining safety and quality in healthcare services. Finally, the adoption can be forged further by the organizational process informed by the HITL approach to improve multidisciplinary teams in the loop. AI can create a paradigm shift in healthcare by complementing and enhancing the skills of healthcare providers, ultimately leading to improved service quality, patient outcomes, and a more efficient healthcare system.