Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

Embracing Generative Artificial Intelligence in Clinical Research and Beyond: Opportunities, Challenges, and Solutions

To explore threats and opportunities and to chart a path for safely navigating the rapid changes that generative artificial intelligence (AI) will bring to clinical research, the Duke Clinical Research Institute convened a multidisciplinary think tank in January 2024. Leading experts from academia, industry, nonprofits, and government agencies highlighted the potential opportunities of generative AI in automation of documentation, strengthening of participant and community engagement, and improvement of trial accuracy and efficiency. Challenges include technical hurdles, ethical dilemmas, and regulatory uncertainties. Success is expected to require establishing rigorous data management and security protocols, fostering integrity and trust among stakeholders, and sharing information about the safety and effectiveness of AI applications. Meeting insights point towards a future where, through collaboration and transparency, generative AI will help to shorten the translational pipeline and increase the inclusivity and equitability of clinical research.

Evolution of radiology staff perspectives during artificial intelligence (AI) implementation for expedited lung cancer triage

AIMS

To investigate radiology staff perceptions of an AI tool for chest radiography triage, flagging findings suspicious for lung cancer to expedite same-day CT chest examination studies.

MATERIALS AND METHODS

Surveys were distributed to all radiology staff at three time points: at pre-implementation, one month and also seven months post-implementation of artificial intelligence (AI). Survey questions captured feedback on AI use and patient impact.

RESULTS

Survey response rates at the three time periods were 23.1% (45/195), 14.9% (29/195) and 27.2% (53/195), respectively. Most respondents initially anticipated AI to be time-saving for the department and patient (50.8%), but this shifted to faster follow-up care for patients after AI implementation (51.7%). From the free text comments, early apprehension about job role changes evolved into frustration regarding technical integration challenges after implementation. This later transitioned to a more balanced view of recognised patient benefits versus minor ongoing logistical issues by the late post-implementation stage. There was majority disagreement across all survey periods that AI could be considered to be used autonomously (53.3-72.5%), yet acceptance grew for personal AI usage if staff were to be patients themselves (from 31.1% pre-implementation to 47.2% post-implementation).

CONCLUSION

Successful AI integration in radiology demands active staff engagement, addressing concerns to transform initial mixed excitement and resistance into constructive adaptation. Continual feedback is vital for refining AI deployment strategies, ensuring its beneficial and sustainable incorporation into clinical care pathways.

The Use of AI in Mental Health Services to Support Decision-Making: Scoping Review

BACKGROUND

Recent advancements in artificial intelligence (AI) have changed the care processes in mental health, particularly in decision-making support for health care professionals and individuals with mental health problems. AI systems provide support in several domains of mental health, including early detection, diagnostics, treatment, and self-care. The use of AI systems in care flows faces several challenges in relation to decision-making support, stemming from technology, end-user, and organizational perspectives with the AI disruption of care processes.

OBJECTIVE

This study aims to explore the use of AI systems in mental health to support decision-making, focusing on 3 key areas: the characteristics of research on AI systems in mental health; the current applications, decisions, end users, and user flow of AI systems to support decision-making; and the evaluation of AI systems for the implementation of decision-making support, including elements influencing the long-term use.

METHODS

A scoping review of empirical evidence was conducted across 5 databases: PubMed, Scopus, PsycINFO, Web of Science, and CINAHL. The searches were restricted to peer-reviewed articles published in English after 2011. The initial screening at the title and abstract level was conducted by 2 reviewers, followed by full-text screening based on the inclusion criteria. Data were then charted and prepared for data analysis.

RESULTS

Of a total of 1217 articles, 12 (0.99%) met the inclusion criteria. These studies predominantly originated from high-income countries. The AI systems were used in health care, self-care, and hybrid care contexts, addressing a variety of mental health problems. Three types of AI systems were identified in terms of decision-making support: diagnostic and predictive AI, treatment selection AI, and self-help AI. The dynamics of the type of end-user interaction and system design were diverse in complexity for the integration and use of the AI systems to support decision-making in care processes. The evaluation of the use of AI systems highlighted several challenges impacting the implementation and functionality of the AI systems in care processes, including factors affecting accuracy, increase of demand, trustworthiness, patient-physician communication, and engagement with the AI systems.

CONCLUSIONS

The design, development, and implementation of AI systems to support decision-making present substantial challenges for the sustainable use of this technology in care processes. The empirical evidence shows that the evaluation of the use of AI systems in mental health is still in its early stages, with need for more empirically focused research on real-world use. The key aspects requiring further investigation include the evaluation of the use of AI-supported decision-making from human-AI interaction and human-computer interaction perspectives, longitudinal implementation studies of AI systems in mental health to assess the use, and the integration of shared decision-making in AI systems.

Health Care Professionals and Data Scientists' Perspectives on a Machine Learning System to Anticipate and Manage the Risk of Decompensation From Patients With Heart Failure: Qualitative Interview Study

BACKGROUND

Heart failure (HF) is a significant global health problem, affecting approximately 64.34 million people worldwide. The worsening of HF, also known as HF decompensation, is a major factor behind hospitalizations, contributing to substantial health care costs related to this condition.

OBJECTIVE

This study aimed to explore the perspectives of health care professionals and data scientists regarding the relevance, challenges, and potential benefits of using machine learning (ML) models to predict decompensation from patients with HF.

METHODS

A total of 13 individual, semistructured, qualitative interviews were conducted in Portugal between October 31, 2022, and June 23, 2023. Participants represented different health care specialties and were selected from different contexts and regions of the country to ensure a comprehensive understanding of the topic. Data saturation was determined as the point at which no new themes emerged from participants' perspectives, ensuring a sufficient sample size for analysis. The interviews were audio recorded, transcribed, and analyzed using MAXQDA (VERBI Software GmbH) through a reflexive thematic analysis. Two researchers (JS and AH) coded the interviews to ensure the consistency of the codes. Ethical approval was granted by the NOVA National School of Public Health ethics committee (CEENSP 14/2022), and informed consent was obtained from all participants.

RESULTS

The participants recognized the potential benefits of ML models for early detection, risk stratification, and personalized care of patients with HF. The importance of selecting appropriate variables for model development, such as rapid weight gain and symptoms, was emphasized. The use of wearables for recording vital signs was considered necessary, although challenges related to adoption among older patients were identified. Risk stratification emerged as a crucial aspect, with the model needing to identify patients at high-, medium-, and low-risk levels. Participants emphasized the need for a response model involving health care professionals to validate ML-generated alerts and determine appropriate interventions.

CONCLUSIONS

The study's findings highlight ML models' potential benefits and challenges for predicting HF decompensation. The relevance of ML models for improving patient outcomes, reducing health care costs, and promoting patient engagement in disease management is highlighted. Adequate variable selection, risk stratification, and response models were identified as essential components for the effective implementation of ML models in health care. In addition, the study identified technical, regulatory and ethical, and adoption and acceptance challenges that need to be overcome for the successful integration of ML models into clinical workflows. Interpretation of the findings suggests that future research should focus on more extensive and diverse samples, incorporate the patient perspective, and explore the impact of ML models on patient outcomes and personalized care in HF management. Incorporation of this study's findings into practice is expected to contribute to developing and implementing ML-based predictive models that positively impact HF management.

AI Interventions to Alleviate Healthcare Shortages and Enhance Work Conditions in Critical Care: Qualitative Analysis

BACKGROUND

The escalating global scarcity of skilled health care professionals is a critical concern, further exacerbated by rising stress levels and clinician burnout rates. Artificial intelligence (AI) has surfaced as a potential resource to alleviate these challenges. Nevertheless, it is not taken for granted that AI will inevitably augment human performance, as ill-designed systems may inadvertently impose new burdens on health care workers, and implementation may be challenging. An in-depth understanding of how AI can effectively enhance rather than impair work conditions is therefore needed.

OBJECTIVE

This research investigates the efficacy of AI in alleviating stress and enriching work conditions, using intensive care units (ICUs) as a case study. Through a sociotechnical system lens, we delineate how AI systems, tasks, and responsibilities of ICU nurses and physicians can be co-designed to foster motivating, resilient, and health-promoting work.

METHODS

We use the sociotechnical system framework COMPASS (Complementary Analysis of Sociotechnical Systems) to assess 5 job characteristics: autonomy, skill diversity, flexibility, problem-solving opportunities, and task variety. The qualitative analysis is underpinned by extensive workplace observation in 6 ICUs (approximately 559 nurses and physicians), structured interviews with work unit leaders (n=12), and a comparative analysis of data science experts' and clinicians' evaluation of the optimal levels of human-AI teaming.

RESULTS

The results indicate that AI holds the potential to positively impact work conditions for ICU nurses and physicians in four key areas. First, autonomy is vital for stress reduction, motivation, and performance improvement. AI systems that ensure transparency, predictability, and human control can reinforce or amplify autonomy. Second, AI can encourage skill diversity and competence development, thus empowering clinicians to broaden their skills, increase the polyvalence of tasks across professional boundaries, and improve interprofessional cooperation. However, careful consideration is required to avoid the deskilling of experienced professionals. Third, AI automation can expand flexibility by relieving clinicians from administrative duties, thereby concentrating their efforts on patient care. Remote monitoring and improved scheduling can help integrate work with other life domains. Fourth, while AI may reduce problem-solving opportunities in certain areas, it can open new pathways, particularly for nurses. Finally, task identity and variety are essential job characteristics for intrinsic motivation and worker engagement but could be compromised depending on how AI tools are designed and implemented.

CONCLUSIONS

This study demonstrates AI's capacity to mitigate stress and improve work conditions for ICU nurses and physicians, thereby contributing to resolving health care staffing shortages. AI solutions that are thoughtfully designed in line with the principles for good work design can enhance intrinsic motivation, learning, and worker well-being, thus providing strategic value for hospital management, policy makers, and health care professionals alike.

Exploring geriatric nurses' perspectives on the adoption of AI in elderly care a qualitative study

This phenomenological study explored the perspectives of geriatric nurses on the adoption of artificial intelligence (AI) in elderly care. Thematic analysis of semi-structured interviews with 17 nurses revealed perceived benefits, challenges, ethical considerations, and practical implications. Participants acknowledged AI's potential for improving diagnostic accuracy, personalized care, continuous monitoring, and data pattern insights. However, concerns were raised regarding workflow integration, cost barriers, resistance to change, data privacy, diminishment of human elements, and the need for ethical guidelines. A cautious optimism was expressed, emphasizing the importance of addressing practical challenges, maintaining the human touch, and fostering a collaborative approach. The findings highlight the need for comprehensive training, user-centered design, ethical frameworks, and strategies to overcome financial and implementation barriers. Future research should focus on evaluating the impact of AI implementation on patient outcomes and nursing experiences.

Allied Health Professionals' Perceptions of Artificial Intelligence in the Clinical Setting: Cross-Sectional Survey

BACKGROUND

Artificial intelligence (AI) has the potential to address growing logistical and economic pressures on the health care system by reducing risk, increasing productivity, and improving patient safety; however, implementing digital health technologies can be disruptive. Workforce perception is a powerful indicator of technology use and acceptance, however, there is little research available on the perceptions of allied health professionals (AHPs) toward AI in health care.

OBJECTIVE

This study aimed to explore AHP perceptions of AI and the opportunities and challenges for its use in health care delivery.

METHODS

A cross-sectional survey was conducted at a health service in, Queensland, Australia, using the Shinners Artificial Intelligence Perception tool.

RESULTS

A total of 231 (22.1%) participants from 11 AHPs responded to the survey. Participants were mostly younger than 40 years (157/231, 67.9%), female (189/231, 81.8%), working in a clinical role (196/231, 84.8%) with a median of 10 years' experience in their profession. Most participants had not used AI (185/231, 80.1%), had little to no knowledge about AI (201/231, 87%), and reported workforce knowledge and skill as the greatest challenges to incorporating AI in health care (178/231, 77.1%). Age (P=.01), profession (P=.009), and AI knowledge (P=.02) were strong predictors of the perceived professional impact of AI. AHPs generally felt unprepared for the implementation of AI in health care, with concerns about a lack of workforce knowledge on AI and losing valued tasks to AI. Prior use of AI (P=.02) and years of experience as a health care professional (P=.02) were significant predictors of perceived preparedness for AI. Most participants had not received education on AI (190/231, 82.3%) and desired training (170/231, 73.6%) and believed AI would improve health care. Ideas and opportunities suggested for the use of AI within the allied health setting were predominantly nonclinical, administrative, and to support patient assessment tasks, with a view to improving efficiencies and increasing clinical time for direct patient care.

CONCLUSIONS

Education and experience with AI are needed in health care to support its implementation across allied health, the second largest workforce in health. Industry and academic partnerships with clinicians should not be limited to AHPs with high AI literacy as clinicians across all knowledge levels can identify many opportunities for AI in health care.

"How I would like AI used for my imaging": children and young persons' perspectives

OBJECTIVES

Artificial intelligence (AI) tools are becoming more available in modern healthcare, particularly in radiology, although less attention has been paid to applications for children and young people. In the development of these, it is critical their views are heard.

MATERIALS AND METHODS

A national, online survey was publicised to UK schools, universities and charity partners encouraging any child or young adult to participate. The survey was "live" for one year (June 2022 to 2023). Questions about views of AI in general, and in specific circumstances (e.g. bone fractures) were asked.

RESULTS

One hundred and seventy-one eligible responses were received, with a mean age of 19 years (6-23 years) with representation across all 4 UK nations. Most respondents agreed or strongly agreed they wanted to know the accuracy of an AI tool that was being used (122/171, 71.3%), that accuracy was more important than speed (113/171, 66.1%), and that AI should be used with human oversight (110/171, 64.3%). Many respondents (73/171, 42.7%) felt AI would be more accurate at finding problems on bone X-rays than humans, with almost all respondents who had sustained a missed fracture strongly agreeing with that sentiment (12/14, 85.7%).

CONCLUSIONS

Children and young people in our survey had positive views regarding AI, and felt it should be integrated into modern healthcare, but expressed a preference for a "medical professional in the loop" and accuracy of findings over speed. Key themes regarding information on AI performance and governance were raised and should be considered prior to future AI implementation for paediatric healthcare.

CLINICAL RELEVANCE STATEMENT

Artificial intelligence (AI) integration into clinical practice must consider all stakeholders, especially paediatric patients who have largely been ignored. Children and young people favour AI involvement with human oversight, seek assurances for safety, accuracy, and clear accountability in case of failures.

KEY POINTS

Paediatric patient's needs and voices are often overlooked in AI tool design and deployment. Children and young people approved of AI, if paired with human oversight and reliability. Children and young people are stakeholders for developing and deploying AI tools in paediatrics.

Implementation Strategy for Artificial Intelligence in Radiotherapy: Can Implementation Science Help?

PURPOSE

Artificial intelligence (AI) applications in radiotherapy (RT) are expected to save time and improve quality, but implementation remains limited. Therefore, we used implementation science to develop a format for designing an implementation strategy for AI. This study aimed to (1) apply this format to develop an AI implementation strategy for our center; (2) identify insights gained to enhance AI implementation using this format; and (3) assess the feasibility and acceptability of this format to design a center-specific implementation strategy for departments aiming to implement AI.

METHODS

We created an AI-implementation strategy for our own center using implementation science methods. This included a stakeholder analysis, literature review, and interviews to identify facilitators and barriers, and designed strategies to overcome the barriers. These methods were subsequently used in a workshop with teams from seven Dutch RT centers to develop their own AI-implementation plans. The applicability, appropriateness, and feasibility were evaluated by the workshop participants, and relevant insights for AI implementation were summarized.

RESULTS

The stakeholder analysis identified internal (physicians, physicists, RT technicians, information technology, and education) and external (patients and representatives) stakeholders. Barriers and facilitators included concerns about opacity, privacy, data quality, legal aspects, knowledge, trust, stakeholder involvement, ethics, and multidisciplinary collaboration, all integrated into our implementation strategy. The workshop evaluation showed high acceptability (18 participants [90%]), appropriateness (17 participants [85%]), and feasibility (15 participants [75%]) of the implementation strategy. Sixteen participants fully agreed with the format.

CONCLUSION

Our study highlights the need for a collaborative approach to implement AI in RT. We designed a strategy to overcome organizational challenges, improve AI integration, and enhance patient care. Workshop feedback indicates the proposed methods are useful for multiple RT centers. Insights gained by applying the methods highlight the importance of multidisciplinary collaboration in the development and implementation of AI.

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

Intervention design for artificial intelligence-enabled macular service implementation: a primary qualitative study

BACKGROUND

Neovascular age-related macular degeneration (nAMD) is one of the largest single-disease contributors to hospital outpatient appointments. Challenges in finding the clinical capacity to meet this demand can lead to sight-threatening delays in the macular services that provide treatment. Clinical artificial intelligence (AI) technologies pose one opportunity to rebalance demand and capacity in macular services. However, there is a lack of evidence to guide early-adopters seeking to use AI as a solution to demand-capacity imbalance. This study aims to provide guidance for these early adopters on how AI-enabled macular services may best be implemented by exploring what will influence the outcome of AI implementation and why.

METHODS

Thirty-six semi-structured interviews were conducted with participants. Data were analysed with the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework to identify factors likely to influence implementation outcomes. These factors and the primary data then underwent a secondary analysis using the Fit between Individuals, Technology and Task (FITT) framework to propose an actionable intervention.

RESULTS

nAMD treatment should be initiated at face-to-face appointments with clinicians who recommend year-long periods of AI-enabled scheduling of treatments. This aims to maintain or enhance the quality of patient communication, whilst reducing consultation frequency. Appropriately trained photographers should take on the additional roles of inputting retinal imaging into the AI device and overseeing its communication to clinical colleagues, while ophthalmologists assume clinical oversight and consultation roles. Interoperability to facilitate this intervention would best be served by imaging equipment that can send images to the cloud securely for analysis by AI tools. Picture Archiving and Communication Software (PACS) should have the capability to output directly into electronic medical records (EMR) familiar to clinical and administrative staff.

CONCLUSION

There are many enablers to implementation and few of the remaining barriers relate directly to the AI technology itself. The proposed intervention requires local tailoring and prospective evaluation but can support early adopters in optimising the chances of success from initial efforts to implement AI-enabled macular services.

PROTOCOL REGISTRATION

Hogg HDJ, Brittain K, Teare D, Talks J, Balaskas K, Keane P, Maniatopoulos G. Safety and efficacy of an artificial intelligence-enabled decision tool for treatment decisions in neovascular age-related macular degeneration and an exploration of clinical pathway integration and implementation: protocol for a multi-methods validation study. BMJ Open. 2023 Feb 1;13(2):e069443. https://doi.org/10.1136/bmjopen-2022-069443 . PMID: 36725098; PMCID: PMC9896175.

Implementation of Machine Learning Applications in Health Care Organizations: Systematic Review of Empirical Studies

BACKGROUND

There is a growing enthusiasm for machine learning (ML) among academics and health care practitioners. Despite the transformative potential of ML-based applications for patient care, their uptake and implementation in health care organizations are sporadic. Numerous challenges currently impede or delay the widespread implementation of ML in clinical practice, and limited knowledge is available regarding how these challenges have been addressed.

OBJECTIVE

This work aimed to (1) examine the characteristics of ML-based applications and the implementation process in clinical practice, using the Consolidated Framework for Implementation Research (CFIR) for theoretical guidance and (2) synthesize the strategies adopted by health care organizations to foster successful implementation of ML.

METHODS

A systematic literature review was conducted based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The search was conducted in PubMed, Scopus, and Web of Science over a 10-year period (2013-2023). The search strategy was built around 4 blocks of keywords (artificial intelligence, implementation, health care, and study type). Only empirical studies documenting the implementation of ML applications in clinical settings were considered. The implementation process was investigated using a thematic analysis and coding procedure.

RESULTS

Thirty-four studies were selected for data synthesis. Selected papers were relatively recent, with only 9% (3/34) of records published before 2019. ML-based applications were implemented mostly within hospitals (29/34, 85%). In terms of clinical workflow, ML-based applications supported mostly prognosis (20/34, 59%) and diagnosis (10/34, 29%). The implementation efforts were analyzed using CFIR domains. As for the inner setting domain, access to knowledge and information (12/34, 35%), information technology infrastructure (11/34, 32%), and organizational culture (9/34, 26%) were among the most observed dimensions influencing the success of implementation. As for the ML innovation itself, factors deemed relevant were its design (15/34, 44%), the relative advantage with respect to existing clinical practice (14/34, 41%), and perceived complexity (14/34, 41%). As for the other domains (ie, processes, roles, and outer setting), stakeholder engagement (12/34, 35%), reflecting and evaluating practices (11/34, 32%), and the presence of implementation leaders (9/34, 26%) were the main factors identified as important.

CONCLUSIONS

This review sheds some light on the factors that are relevant and that should be accounted for in the implementation process of ML-based applications in health care. While the relevance of ML-specific dimensions, like trust, emerges clearly across several implementation domains, the evidence from this review highlighted that relevant implementation factors are not necessarily specific for ML but rather transversal for digital health technologies. More research is needed to further clarify the factors that are relevant to implementing ML-based applications at the organizational level and to support their uptake within health care organizations.

TRIAL REGISTRATION

PROSPERO 403873; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=403873.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/47971.

Health Care Professionals' Experience of Using AI: Systematic Review With Narrative Synthesis

BACKGROUND

There has been a substantial increase in the development of artificial intelligence (AI) tools for clinical decision support. Historically, these were mostly knowledge-based systems, but recent advances include non-knowledge-based systems using some form of machine learning. The ability of health care professionals to trust technology and understand how it benefits patients or improves care delivery is known to be important for their adoption of that technology. For non-knowledge-based AI tools for clinical decision support, these issues are poorly understood.

OBJECTIVE

The aim of this study is to qualitatively synthesize evidence on the experiences of health care professionals in routinely using non-knowledge-based AI tools to support their clinical decision-making.

METHODS

In June 2023, we searched 4 electronic databases, MEDLINE, Embase, CINAHL, and Web of Science, with no language or date limit. We also contacted relevant experts and searched reference lists of the included studies. We included studies of any design that reported the experiences of health care professionals using non-knowledge-based systems for clinical decision support in their work settings. We completed double independent quality assessment for all included studies using the Mixed Methods Appraisal Tool. We used a theoretically informed thematic approach to synthesize the findings.

RESULTS

After screening 7552 titles and 182 full-text articles, we included 25 studies conducted in 9 different countries. Most of the included studies were qualitative (n=13), and the remaining were quantitative (n=9) and mixed methods (n=3). Overall, we identified 7 themes: health care professionals' understanding of AI applications, level of trust and confidence in AI tools, judging the value added by AI, data availability and limitations of AI, time and competing priorities, concern about governance, and collaboration to facilitate the implementation and use of AI. The most frequently occurring are the first 3 themes. For example, many studies reported that health care professionals were concerned about not understanding the AI outputs or the rationale behind them. There were issues with confidence in the accuracy of the AI applications and their recommendations. Some health care professionals believed that AI provided added value and improved decision-making, and some reported that it only served as a confirmation of their clinical judgment, while others did not find it useful at all.

CONCLUSIONS

Our review identified several important issues documented in various studies on health care professionals' use of AI tools in real-world health care settings. Opinions of health care professionals regarding the added value of AI tools for supporting clinical decision-making varied widely, and many professionals had concerns about their understanding of and trust in this technology. The findings of this review emphasize the need for concerted efforts to optimize the integration of AI tools in real-world health care settings.

TRIAL REGISTRATION

PROSPERO CRD42022336359; https://tinyurl.com/2yunvkmb.

"It happened to be the perfect thing": experiences of generative AI chatbots for mental health

The global mental health crisis underscores the need for accessible, effective interventions. Chatbots based on generative artificial intelligence (AI), like ChatGPT, are emerging as novel solutions, but research on real-life usage is limited. We interviewed nineteen individuals about their experiences using generative AI chatbots for mental health. Participants reported high engagement and positive impacts, including better relationships and healing from trauma and loss. We developed four themes: (1) a sense of 'emotional sanctuary', (2) 'insightful guidance', particularly about relationships, (3) the 'joy of connection', and (4) comparisons between the 'AI therapist' and human therapy. Some themes echoed prior research on rule-based chatbots, while others seemed novel to generative AI. Participants emphasised the need for better safety guardrails, human-like memory and the ability to lead the therapeutic process. Generative AI chatbots may offer mental health support that feels meaningful to users, but further research is needed on safety and effectiveness.

Applying the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability Framework Across Implementation Stages to Identify Key Strategies to Facilitate Clinical Decision Support System Integration Within a Large Metropolitan Health Service: Interview and Focus Group Study

BACKGROUND

Computerized clinical decision support systems (CDSSs) enhance patient care through real-time, evidence-based guidance for health care professionals. Despite this, the effective implementation of these systems for health services presents multifaceted challenges, leading to inappropriate use and abandonment over the course of time. Using the Non-Adoption, Abandonment, Scale-Up, Spread, and Sustainability (NASSS) framework, this qualitative study examined CDSS adoption in a metropolitan health service, identifying determinants across implementation stages to optimize CDSS integration into health care practice.

OBJECTIVE

This study aims to identify the theory-informed (NASSS) determinants, which included multiple CDSS interventions across a 2-year period, both at the health-service level and at the individual hospital setting, that either facilitate or hinder the application of CDSSs within a metropolitan health service. In addition, this study aimed to map these determinants onto specific stages of the implementation process, thereby developing a system-level understanding of CDSS application across implementation stages.

METHODS

Participants involved in various stages of the implementation process were recruited (N=30). Participants took part in interviews and focus groups. We used a hybrid inductive-deductive qualitative content analysis and a framework mapping approach to categorize findings into barriers, enablers, or neutral determinants aligned to NASSS framework domains. These determinants were also mapped to implementation stages using the Active Implementation Framework stages approach.

RESULTS

Participants comprised clinical adopters (14/30, 47%), organizational champions (5/30, 16%), and those with roles in organizational clinical informatics (5/30, 16%). Most determinants were mapped to the organization level, technology, and adopter subdomains. However, the study findings also demonstrated a relative lack of long-term implementation planning. Consequently, determinants were not uniformly distributed across the stages of implementation, with 61.1% (77/126) identified in the exploration stage, 30.9% (39/126) in the full implementation stage, and 4.7% (6/126) in the installation stages. Stakeholders engaged in more preimplementation and full-scale implementation activities, with fewer cycles of monitoring and iteration activities identified.

CONCLUSIONS

These findings addressed a substantial knowledge gap in the literature using systems thinking principles to identify the interdependent dynamics of CDSS implementation. A lack of sustained implementation strategies (ie, training and longer-term, adopter-level championing) weakened the sociotechnical network between developers and adopters, leading to communication barriers. More rigorous implementation planning, encompassing all 4 implementation stages, may, in a way, help in addressing the barriers identified and enhancing enablers.

Women's views on using artificial intelligence in breast cancer screening: A review and qualitative study to guide breast screening services

As breast screening services move towards use of healthcare AI (HCAI) for screen reading, research on public views of HCAI can inform more person-centered implementation. We synthesise reviews of public views of HCAI in general, and review primary studies of women's views of AI in breast screening. People generally appear open to HCAI and its potential benefits, despite a wide range of concerns; similarly, women are open towards AI in breast screening because of the potential benefits, but are concerned about a wide range of risks. Women want radiologists to remain central; oversight, evaluation and performance, care, equity and bias, transparency, and accountability are key issues; women may be less tolerant of AI error than of human error. Using our recent Australian primary study, we illustrate both the value of informing participants before collecting data, and women's views. The 40 screening-age women in this study stipulated four main conditions on breast screening AI implementation: 1) maintaining human control; 2) strong evidence of performance; 3) supporting familiarisation with AI; and 4) providing adequate reasons for introducing AI. Three solutions were offered to support familiarisation: transparency and information; slow and staged implementation; and allowing women to opt-out of AI reading. We provide recommendations to guide both implementation of AI in healthcare and research on public views of HCAI. Breast screening services should be transparent about AI use and share information about breast screening AI with women. Implementation should be slow and staged, providing opt-out options if possible. Screening services should demonstrate strong governance to maintain clinician control, demonstrate excellent AI system performance, assure data protection and bias mitigation, and give good reasons to justify implementation. When these measures are put in place, women are more likely to see HCAI use in breast screening as legitimate and acceptable.

Ethical framework for artificial intelligence in healthcare research: A path to integrity

The integration of Artificial Intelligence (AI) into healthcare research promises unprecedented advancements in medical diagnostics, treatment personalization, and patient care management. However, these innovations also bring forth significant ethical challenges that must be addressed to maintain public trust, ensure patient safety, and uphold data integrity. This article sets out to introduce a detailed framework designed to steer governance and offer a systematic method for assuring that AI applications in healthcare research are developed and executed with integrity and adherence to medical research ethics.

The Human-Technology Continuum

BACKGROUND AND OBJECTIVES

Managers in health care today face an array of digital technologies that assist or augment certain human tasks. But these technologies are often fraught and present challenges to managers, whose competencies must evolve to keep pace with technological advancements.

METHODS

Drawing on theory about technology, work, and organizations, we present a human-technology continuum to facilitate this discussion for managers. Furthermore, we illustrate how managerial competencies are linked to the entire human-technology continuum, rather than to specific technologies, using diabetes management examples.

RESULTS

The human-technology continuum indicates that augmentative technologies are layered onto assistive ones in health care settings. This suggests that technological advancements not only enhance but alter managerial competencies.

CONCLUSIONS

Digital technology stretches the boundaries of managers' day-to-day work in health care. Therefore, we make the following suggestions so the managers can be responsive to ongoing digital transformations: restructuring work, training the workforce, neutralizing threats, establishing ethical boundaries, and building partnerships.

Clinical Evaluation of Artificial Intelligence-Enabled Interventions

Artificial intelligence (AI) health technologies are increasingly available for use in real-world care. This emerging opportunity is accompanied by a need for decision makers and practitioners across healthcare systems to evaluate the safety and effectiveness of these interventions against the needs of their own setting. To meet this need, high-quality evidence regarding AI-enabled interventions must be made available, and decision makers in varying roles and settings must be empowered to evaluate that evidence within the context in which they work. This article summarizes good practices across four stages of evidence generation for AI health technologies: study design, study conduct, study reporting, and study appraisal.

Facilitating public involvement in research about healthcare AI: A scoping review of empirical methods

OBJECTIVE

With the recent increase in research into public views on healthcare artificial intelligence (HCAI), the objective of this review is to examine the methods of empirical studies on public views on HCAI. We map how studies provided participants with information about HCAI, and we examine the extent to which studies framed publics as active contributors to HCAI governance.

MATERIALS AND METHODS

We searched 5 academic databases and Google Advanced for empirical studies investigating public views on HCAI. We extracted information including study aims, research instruments, and recommendations.

RESULTS

Sixty-two studies were included. Most were quantitative (N = 42). Most (N = 47) reported providing participants with background information about HCAI. Despite this, studies often reported participants' lack of prior knowledge about HCAI as a limitation. Over three quarters (N = 48) of the studies made recommendations that envisaged public views being used to guide governance of AI.

DISCUSSION

Provision of background information is an important component of facilitating research with publics on HCAI. The high proportion of studies reporting participants' lack of knowledge about HCAI as a limitation reflects the need for more guidance on how information should be presented. A minority of studies adopted technocratic positions that construed publics as passive beneficiaries of AI, rather than as active stakeholders in HCAI design and implementation.

CONCLUSION

This review draws attention to how public roles in HCAI governance are constructed in empirical studies. To facilitate active participation, we recommend that research with publics on HCAI consider methodological designs that expose participants to diverse information sources.

Frameworks for procurement, integration, monitoring, and evaluation of artificial intelligence tools in clinical settings: A systematic review

Research on the applications of artificial intelligence (AI) tools in medicine has increased exponentially over the last few years but its implementation in clinical practice has not seen a commensurate increase with a lack of consensus on implementing and maintaining such tools. This systematic review aims to summarize frameworks focusing on procuring, implementing, monitoring, and evaluating AI tools in clinical practice. A comprehensive literature search, following PRSIMA guidelines was performed on MEDLINE, Wiley Cochrane, Scopus, and EBSCO databases, to identify and include articles recommending practices, frameworks or guidelines for AI procurement, integration, monitoring, and evaluation. From the included articles, data regarding study aim, use of a framework, rationale of the framework, details regarding AI implementation involving procurement, integration, monitoring, and evaluation were extracted. The extracted details were then mapped on to the Donabedian Plan, Do, Study, Act cycle domains. The search yielded 17,537 unique articles, out of which 47 were evaluated for inclusion based on their full texts and 25 articles were included in the review. Common themes extracted included transparency, feasibility of operation within existing workflows, integrating into existing workflows, validation of the tool using predefined performance indicators and improving the algorithm and/or adjusting the tool to improve performance. Among the four domains (Plan, Do, Study, Act) the most common domain was Plan (84%, n = 21), followed by Study (60%, n = 15), Do (52%, n = 13), & Act (24%, n = 6). Among 172 authors, only 1 (0.6%) was from a low-income country (LIC) and 2 (1.2%) were from lower-middle-income countries (LMICs). Healthcare professionals cite the implementation of AI tools within clinical settings as challenging owing to low levels of evidence focusing on integration in the Do and Act domains. The current healthcare AI landscape calls for increased data sharing and knowledge translation to facilitate common goals and reap maximum clinical benefit.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial

OBJECTIVES

We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2).

MATERIALS AND METHODS

A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected.

RESULTS

Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians.

DISCUSSION

Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved.

CONCLUSION

The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century

As healthcare systems around the world face challenges such as escalating costs, limited access, and growing demand for personalized care, artificial intelligence (AI) is emerging as a key force for transformation. This review is motivated by the urgent need to harness AI's potential to mitigate these issues and aims to critically assess AI's integration in different healthcare domains. We explore how AI empowers clinical decision-making, optimizes hospital operation and management, refines medical image analysis, and revolutionizes patient care and monitoring through AI-powered wearables. Through several case studies, we review how AI has transformed specific healthcare domains and discuss the remaining challenges and possible solutions. Additionally, we will discuss methodologies for assessing AI healthcare solutions, ethical challenges of AI deployment, and the importance of data privacy and bias mitigation for responsible technology use. By presenting a critical assessment of AI's transformative potential, this review equips researchers with a deeper understanding of AI's current and future impact on healthcare. It encourages an interdisciplinary dialogue between researchers, clinicians, and technologists to navigate the complexities of AI implementation, fostering the development of AI-driven solutions that prioritize ethical standards, equity, and a patient-centered approach.

Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study

BACKGROUND

Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation.

OBJECTIVE

This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England.

METHODS

This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote.

RESULTS

Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024.

CONCLUSIONS

The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50568.

The Impact of Digital Hospitals on Patient and Clinician Experience: Systematic Review and Qualitative Evidence Synthesis

BACKGROUND

The digital transformation of health care is advancing rapidly. A well-accepted framework for health care improvement is the Quadruple Aim: improved clinician experience, improved patient experience, improved population health, and reduced health care costs. Hospitals are attempting to improve care by using digital technologies, but the effectiveness of these technologies is often only measured against cost and quality indicators, and less is known about the clinician and patient experience.

OBJECTIVE

This study aims to conduct a systematic review and qualitative evidence synthesis to assess the clinician and patient experience of digital hospitals.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and ENTREQ (Enhancing the Transparency in Reporting the Synthesis of Qualitative Research) guidelines were followed. The PubMed, Embase, Scopus, CINAHL, and PsycINFO databases were searched from January 2010 to June 2022. Studies that explored multidisciplinary clinician or adult inpatient experiences of digital hospitals (with a full electronic medical record) were included. Study quality was assessed using the Mixed Methods Appraisal Tool. Data synthesis was performed narratively for quantitative studies. Qualitative evidence synthesis was performed via (1) automated machine learning text analytics using Leximancer (Leximancer Pty Ltd) and (2) researcher-led inductive synthesis to generate themes.

RESULTS

A total of 61 studies (n=39, 64% quantitative; n=15, 25% qualitative; and n=7, 11% mixed methods) were included. Most studies (55/61, 90%) investigated clinician experiences, whereas few (10/61, 16%) investigated patient experiences. The study populations ranged from 8 to 3610 clinicians, 11 to 34,425 patients, and 5 to 2836 hospitals. Quantitative outcomes indicated that clinicians had a positive overall satisfaction (17/24, 71% of the studies) with digital hospitals, and most studies (11/19, 58%) reported a positive sentiment toward usability. Data accessibility was reported positively, whereas adaptation, clinician-patient interaction, and workload burnout were reported negatively. The effects of digital hospitals on patient safety and clinicians' ability to deliver patient care were mixed. The qualitative evidence synthesis of clinician experience studies (18/61, 30%) generated 7 themes: inefficient digital documentation, inconsistent data quality, disruptions to conventional health care relationships, acceptance, safety versus risk, reliance on hybrid (digital and paper) workflows, and patient data privacy. There was weak evidence of a positive association between digital hospitals and patient satisfaction scores.

CONCLUSIONS

Clinicians' experience of digital hospitals appears positive according to high-level indicators (eg, overall satisfaction and data accessibility), but the qualitative evidence synthesis revealed substantive tensions. There is insufficient evidence to draw a definitive conclusion on the patient experience within digital hospitals, but indications appear positive or agnostic. Future research must prioritize equitable investigation and definition of the digital clinician and patient experience to achieve the Quadruple Aim of health care.

Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review

Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.

Perceptions on artificial intelligence-based decision-making for coexisting multiple long-term health conditions: protocol for a qualitative study with patients and healthcare professionals

INTRODUCTION

Coexisting multiple health conditions is common among older people, a population that is increasing globally. The potential for polypharmacy, adverse events, drug interactions and development of additional health conditions complicates prescribing decisions for these patients. Artificial intelligence (AI)-generated decision-making tools may help guide clinical decisions in the context of multiple health conditions, by determining which of the multiple medication options is best. This study aims to explore the perceptions of healthcare professionals (HCPs) and patients on the use of AI in the management of multiple health conditions.

METHODS AND ANALYSIS

A qualitative study will be conducted using semistructured interviews. Adults (≥18 years) with multiple health conditions living in the West Midlands of England and HCPs with experience in caring for patients with multiple health conditions will be eligible and purposively sampled. Patients will be identified from Clinical Practice Research Datalink (CPRD) Aurum; CPRD will contact general practitioners who will in turn, send a letter to patients inviting them to take part. Eligible HCPs will be recruited through British HCP bodies and known contacts. Up to 30 patients and 30 HCPs will be recruited, until data saturation is achieved. Interviews will be in-person or virtual, audio recorded and transcribed verbatim. The topic guide is designed to explore participants' attitudes towards AI-informed clinical decision-making to augment clinician-directed decision-making, the perceived advantages and disadvantages of both methods and attitudes towards risk management. Case vignettes comprising a common decision pathway for patients with multiple health conditions will be presented during each interview to invite participants' opinions on how their experiences compare. Data will be analysed thematically using the Framework Method.

ETHICS AND DISSEMINATION

This study has been approved by the National Health Service Research Ethics Committee (Reference: 22/SC/0210). Written informed consent or verbal consent will be obtained prior to each interview. The findings from this study will be disseminated through peer-reviewed publications, conferences and lay summaries.

Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study

BACKGROUND

Machine learning (ML)-driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes.

OBJECTIVE

This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care.

METHODS

A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient's primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team.

RESULTS

Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups.

CONCLUSIONS

Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians' needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD.

Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study

BACKGROUND

Chatbots are being piloted to draft responses to patient questions, but patients' ability to distinguish between provider and chatbot responses and patients' trust in chatbots' functions are not well established.

OBJECTIVE

This study aimed to assess the feasibility of using ChatGPT (Chat Generative Pre-trained Transformer) or a similar artificial intelligence-based chatbot for patient-provider communication.

METHODS

A survey study was conducted in January 2023. Ten representative, nonadministrative patient-provider interactions were extracted from the electronic health record. Patients' questions were entered into ChatGPT with a request for the chatbot to respond using approximately the same word count as the human provider's response. In the survey, each patient question was followed by a provider- or ChatGPT-generated response. Participants were informed that 5 responses were provider generated and 5 were chatbot generated. Participants were asked-and incentivized financially-to correctly identify the response source. Participants were also asked about their trust in chatbots' functions in patient-provider communication, using a Likert scale from 1-5.

RESULTS

A US-representative sample of 430 study participants aged 18 and older were recruited on Prolific, a crowdsourcing platform for academic studies. In all, 426 participants filled out the full survey. After removing participants who spent less than 3 minutes on the survey, 392 respondents remained. Overall, 53.3% (209/392) of respondents analyzed were women, and the average age was 47.1 (range 18-91) years. The correct classification of responses ranged between 49% (192/392) to 85.7% (336/392) for different questions. On average, chatbot responses were identified correctly in 65.5% (1284/1960) of the cases, and human provider responses were identified correctly in 65.1% (1276/1960) of the cases. On average, responses toward patients' trust in chatbots' functions were weakly positive (mean Likert score 3.4 out of 5), with lower trust as the health-related complexity of the task in the questions increased.

CONCLUSIONS

ChatGPT responses to patient questions were weakly distinguishable from provider responses. Laypeople appear to trust the use of chatbots to answer lower-risk health questions. It is important to continue studying patient-chatbot interaction as chatbots move from administrative to more clinical roles in health care.

Evaluating the translation of implementation science to clinical artificial intelligence: a bibliometric study of qualitative research

Introduction

Whilst a theoretical basis for implementation research is seen as advantageous, there is little clarity over if and how the application of theories, models or frameworks (TMF) impact implementation outcomes. Clinical artificial intelligence (AI) continues to receive multi-stakeholder interest and investment, yet a significant implementation gap remains. This bibliometric study aims to measure and characterize TMF application in qualitative clinical AI research to identify opportunities to improve research practice and its impact on clinical AI implementation.

Methods

Qualitative research of stakeholder perspectives on clinical AI published between January 2014 and October 2022 was systematically identified. Eligible studies were characterized by their publication type, clinical and geographical context, type of clinical AI studied, data collection method, participants and application of any TMF. Each TMF applied by eligible studies, its justification and mode of application was characterized.

Results

Of 202 eligible studies, 70 (34.7%) applied a TMF. There was an 8-fold increase in the number of publications between 2014 and 2022 but no significant increase in the proportion applying TMFs. Of the 50 TMFs applied, 40 (80%) were only applied once, with the Technology Acceptance Model applied most frequently (n = 9). Seven TMFs were novel contributions embedded within an eligible study. A minority of studies justified TMF application (n = 51,58.6%) and it was uncommon to discuss an alternative TMF or the limitations of the one selected (n = 11,12.6%). The most common way in which a TMF was applied in eligible studies was data analysis (n = 44,50.6%). Implementation guidelines or tools were explicitly referenced by 2 reports (1.0%).

Conclusion

TMFs have not been commonly applied in qualitative research of clinical AI. When TMFs have been applied there has been (i) little consensus on TMF selection (ii) limited description of selection rationale and (iii) lack of clarity over how TMFs inform research. We consider this to represent an opportunity to improve implementation science's translation to clinical AI research and clinical AI into practice by promoting the rigor and frequency of TMF application. We recommend that the finite resources of the implementation science community are diverted toward increasing accessibility and engagement with theory informed practices. The considered application of theories, models and frameworks (TMF) are thought to contribute to the impact of implementation science on the translation of innovations into real-world care. The frequency and nature of TMF use are yet to be described within digital health innovations, including the prominent field of clinical AI. A well-known implementation gap, coined as the "AI chasm" continues to limit the impact of clinical AI on real-world care. From this bibliometric study of the frequency and quality of TMF use within qualitative clinical AI research, we found that TMFs are usually not applied, their selection is highly varied between studies and there is not often a convincing rationale for their selection. Promoting the rigor and frequency of TMF use appears to present an opportunity to improve the translation of clinical AI into practice.

Commentary: Patient Perspectives on Artificial Intelligence; What have We Learned and How Should We Move Forward?

Artificial intelligence (AI) in healthcare has now begun to make its contributions to real-world patient care with varying degrees of both public and clinical acceptability around it. The heavy investment from governments, industry and academia needed to reach this point has helped to surface different perspectives on AI. As clinical AI applications become a reality, however, there is an increasing need to harness and integrate patient perspectives, which address the distinct needs of different populations, healthcare systems and clinical problems more closely. Despite this need, patient perspectives on AI implementation have little presence in academic literature and within implementation science and are not sufficiently considered throughout the MedTech and eHealthtech product development cycle, which brings its own challenges and opportunities. This joint patient expert/clinician commentary aims to briefly summarise views on AI. It reflects upon recommendations on how stakeholders such as clinicians and Health & MedTech small and medium-sized enterprises (SMEs) can make practical usage of these views. The recommendations of the authors centre around how to work better with patients to enable both product centric and patient centric innovation and person-centred care.

The Clinical Researcher Journey in the Artificial Intelligence Era: The PAC-MAN’s Challenge

Artificial intelligence (AI) is a powerful tool that can assist researchers and clinicians in various settings. However, like any technology, it must be used with caution and awareness as there are numerous potential pitfalls. To provide a creative analogy, we have likened research to the PAC-MAN classic arcade video game. Just as the protagonist of the game is constantly seeking data, researchers are constantly seeking information that must be acquired and managed within the constraints of the research rules. In our analogy, the obstacles that researchers face are represented by “ghosts”, which symbolize major ethical concerns, low-quality data, legal issues, and educational challenges. In short, clinical researchers need to meticulously collect and analyze data from various sources, often navigating through intricate and nuanced challenges to ensure that the data they obtain are both precise and pertinent to their research inquiry. Reflecting on this analogy can foster a deeper comprehension of the significance of employing AI and other powerful technologies with heightened awareness and attentiveness.

Safety and efficacy of an artificial intelligence-enabled decision tool for treatment decisions in neovascular age-related macular degeneration and an exploration of clinical pathway integration and implementation: protocol for a multi-methods validation study

INTRODUCTION

Neovascular age-related macular degeneration (nAMD) management is one of the largest single-disease contributors to hospital outpatient appointments. Partial automation of nAMD treatment decisions could reduce demands on clinician time. Established artificial intelligence (AI)-enabled retinal imaging analysis tools, could be applied to this use-case, but are not yet validated for it. A primary qualitative investigation of stakeholder perceptions of such an AI-enabled decision tool is also absent. This multi-methods study aims to establish the safety and efficacy of an AI-enabled decision tool for nAMD treatment decisions and understand where on the clinical pathway it could sit and what factors are likely to influence its implementation.

METHODS AND ANALYSIS

Single-centre retrospective imaging and clinical data will be collected from nAMD clinic visits at a National Health Service (NHS) teaching hospital ophthalmology service, including judgements of nAMD disease stability or activity made in real-world consultant-led-care. Dataset size will be set by a power calculation using the first 127 randomly sampled eligible clinic visits. An AI-enabled retinal segmentation tool and a rule-based decision tree will independently analyse imaging data to report nAMD stability or activity for each of these clinic visits. Independently, an external reading centre will receive both clinical and imaging data to generate an enhanced reference standard for each clinic visit. The non-inferiority of the relative negative predictive value of AI-enabled reports on disease activity relative to consultant-led-care judgements will then be tested. In parallel, approximately 40 semi-structured interviews will be conducted with key nAMD service stakeholders, including patients. Transcripts will be coded using a theoretical framework and thematic analysis will follow.

ETHICS AND DISSEMINATION

NHS Research Ethics Committee and UK Health Research Authority approvals are in place (21/NW/0138). Informed consent is planned for interview participants only. Written and oral dissemination is planned to public, clinical, academic and commercial stakeholders.

Artificial intelligence in healthcare: Complementing, not replacing, doctors and healthcare providers

The utilization of artificial intelligence (AI) in clinical practice has increased and is evidently contributing to improved diagnostic accuracy, optimized treatment planning, and improved patient outcomes. The rapid evolution of AI, especially generative AI and large language models (LLMs), have reignited the discussions about their potential impact on the healthcare industry, particularly regarding the role of healthcare providers. Concerning questions, "can AI replace doctors?" and "will doctors who are using AI replace those who are not using it?" have been echoed. To shed light on this debate, this article focuses on emphasizing the augmentative role of AI in healthcare, underlining that AI is aimed to complement, rather than replace, doctors and healthcare providers. The fundamental solution emerges with the human-AI collaboration, which combines the cognitive strengths of healthcare providers with the analytical capabilities of AI. A human-in-the-loop (HITL) approach ensures that the AI systems are guided, communicated, and supervised by human expertise, thereby maintaining safety and quality in healthcare services. Finally, the adoption can be forged further by the organizational process informed by the HITL approach to improve multidisciplinary teams in the loop. AI can create a paradigm shift in healthcare by complementing and enhancing the skills of healthcare providers, ultimately leading to improved service quality, patient outcomes, and a more efficient healthcare system.