Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities

The use of data-driven technologies such as Artificial Intelligence (AI) and Machine Learning (ML) is growing in healthcare. However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use. To address these gaps and tackle a common challenge faced by healthcare delivery organizations, a case-based workshop was organized, and a framework was developed to evaluate the potential impact of implementing an AI solution on health equity. The Health Equity Across the AI Lifecycle (HEAAL) is co-designed with extensive engagement of clinical, operational, technical, and regulatory leaders across healthcare delivery organizations and ecosystem partners in the US. It assesses 5 equity assessment domains-accountability, fairness, fitness for purpose, reliability and validity, and transparency-across the span of eight key decision points in the AI adoption lifecycle. It is a process-oriented framework containing 37 step-by-step procedures for evaluating an existing AI solution and 34 procedures for evaluating a new AI solution in total. Within each procedure, it identifies relevant key stakeholders and data sources used to conduct the procedure. HEAAL guides how healthcare delivery organizations may mitigate the potential risk of AI solutions worsening health inequities. It also informs how much resources and support are required to assess the potential impact of AI solutions on health inequities.

The algorithm journey map: a tangible approach to implementing AI solutions in healthcare

When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This "Algorithm Journey Map" enumerates all social and technical activities throughout the AI solution's procurement, development, integration, and full lifecycle management. In addition to mapping the "who?" and "what?" of the adoption of the AI tool, we also show several 'lessons learned' throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles-in algorithmic systems.

Development and Temporal Validation of a Machine Learning Model to Predict Clinical Deterioration

OBJECTIVES

Early warning scores detecting clinical deterioration in pediatric inpatients have wide-ranging performance and use a limited number of clinical features. This study developed a machine learning model leveraging multiple static and dynamic clinical features from the electronic health record to predict the composite outcome of unplanned transfer to the ICU within 24 hours and inpatient mortality within 48 hours in hospitalized children.

METHODS

Using a retrospective development cohort of 17 630 encounters across 10 388 patients, 2 machine learning models (light gradient boosting machine [LGBM] and random forest) were trained on 542 features and compared with our institutional Pediatric Early Warning Score (I-PEWS).

RESULTS

The LGBM model significantly outperformed I-PEWS based on receiver operating characteristic curve (AUROC) for the composite outcome of ICU transfer or mortality for both internal validation and temporal validation cohorts (AUROC 0.785 95% confidence interval [0.780-0.791] vs 0.708 [0.701-0.715] for temporal validation) as well as lead-time before deterioration events (median 11 hours vs 3 hours; P = .004). However, LGBM performance as evaluated by precision recall curve was lesser in the temporal validation cohort with associated decreased positive predictive value (6% vs 29%) and increased number needed to evaluate (17 vs 3) compared with I-PEWS.

CONCLUSIONS

Our electronic health record based machine learning model demonstrated improved AUROC and lead-time in predicting clinical deterioration in pediatric inpatients 24 to 48 hours in advance compared with I-PEWS. Further work is needed to optimize model positive predictive value to allow for integration into clinical practice.

Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study

BACKGROUND

Machine learning (ML)-driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes.

OBJECTIVE

This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care.

METHODS

A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient's primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team.

RESULTS

Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups.

CONCLUSIONS

Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians' needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD.

Feasibility of Implementation of a Mobile Digital Personal Health Record to Coordinate Care for Children and Youth With Special Health Care Needs in Primary Care: Protocol for a Mixed Methods Study

BACKGROUND

Electronic health record (EHR)-integrated digital personal health records (PHRs) via Fast Healthcare Interoperability Resources (FHIR) are promising digital health tools to support care coordination (CC) for children and youth with special health care needs but remain widely unadopted; as their adoption grows, mixed methods and implementation research could guide real-world implementation and evaluation.

OBJECTIVE

This study (1) evaluates the feasibility of an FHIR-enabled digital PHR app for CC for children and youth with special health care needs, (2) characterizes determinants of implementation, and (3) explores associations between adoption and patient- or family-reported outcomes.

METHODS

This nonrandomized, single-arm, prospective feasibility trial will test an FHIR-enabled digital PHR app's use among families of children and youth with special health care needs in primary care settings. Key app features are FHIR-enabled access to structured data from the child's medical record, families' abilities to longitudinally track patient- or family-centered care goals, and sharing progress toward care goals with the child's primary care provider via a clinician dashboard. We shall enroll 40 parents or caregivers of children and youth with special health care needs to use the app for 6 months. Inclusion criteria for children and youth with special health care needs are age 0-16 years; primary care at a participating site; complex needs benefiting from CC; high hospitalization risk in the next 6 months; English speaking; having requisite technology at home (internet access, Apple iOS mobile device); and an active web-based EHR patient portal account to which a parent or caregiver has full proxy access. Digital prescriptions will be used to disseminate study recruitment materials directly to eligible participants via their existing EHR patient portal accounts. We will apply an intervention mixed methods design to link quantitative and qualitative (semistructured interviews and family engagement panels with parents of children and youth with special health care needs) data and characterize implementation determinants. Two CC frameworks (Pediatric Care Coordination Framework; Patient-Centered Medical Home) and 2 evaluation frameworks (Consolidated Framework for Implementation Research; Technology Acceptance Model) provide theoretical foundations for this study.

RESULTS

Participant recruitment began in fall 2022, before which we identified >300 potentially eligible patients in EHR data. A family engagement panel in fall 2021 generated formative feedback from family partners. Integrated analysis of pretrial quantitative and qualitative data informed family-centered enhancements to study procedures.

CONCLUSIONS

Our findings will inform how to integrate an FHIR-enabled digital PHR app for children and youth with special health care needs into clinical care. Mixed methods and implementation research will help strengthen implementation in diverse clinical settings. The study is positioned to advance knowledge of how to use digital health innovations for improving care and outcomes for children and youth with special health care needs and their families.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05513235; https://clinicaltrials.gov/study/NCT05513235.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46847.

Risk of in-hospital Deterioration for Children with Single Ventricle Physiology

Children with single ventricle physiology (SV) are at high risk of in-hospital morbidity and mortality. Identifying children at risk for deterioration may allow for earlier escalation of care and subsequently decreased mortality.We conducted a retrospective chart review of all admissions to the pediatric cardiology non-ICU service from 2014 to 2018 for children < 18 years old. We defined clinical deterioration as unplanned transfer to the ICU or inpatient mortality. We selected children with SV by diagnosis codes and defined infants as children < 1 year old. We compared demographic, vital sign, and lab values between infants with and without a deterioration event. We evaluated vital sign and medical therapy changes before deterioration events.Among infants with SV (129 deterioration events over 225 admissions, overall 25% with hypoplastic left heart syndrome), those who deteriorated were younger (p = 0.001), had lower baseline oxygen saturation (p = 0.022), and higher baseline respiratory rate (p = 0.022), heart rate (p = 0.023), and hematocrit (p = 0.008). Median Duke Pediatric Early Warning Score increased prior to deterioration (p < 0.001). Deterioration was associated with administration of additional oxygen support (p = 0.012), a fluid bolus (p < 0.001), antibiotics (p < 0.001), vasopressor support (p = 0.009), and red blood cell transfusion (p < 0.001).Infants with SV are at high risk for deterioration. Integrating baseline and dynamic patient data from the electronic health record to identify the highest risk patients may allow for earlier detection and intervention to prevent clinical deterioration.

Accelerating health system innovation: principles and practices from the Duke Institute for Health Innovation

The Duke Institute for Health Innovation (DIHI) was launched in 2013. Frontline staff members submit proposals for innovation projects that align with strategic priorities set by organizational leadership. Funded projects receive operational and technical support from institute staff members and a transdisciplinary network of collaborators to develop and implement solutions as part of routine clinical care, ranging from machine learning algorithms to mobile applications. DIHI's operations are shaped by four guiding principles: build to show value, build to integrate, build to scale, and build responsibly. Between 2013 and 2021, more than 600 project proposals have been submitted to DIHI. More than 85 innovation projects, both through the application process and other strategic partnerships, have been supported and implemented. DIHI's funding has incubated 12 companies, engaged more than 300 faculty members, staff members, and students, and contributed to more than 50 peer-reviewed publications. DIHI's practices can serve as a model for other health systems to systematically source, develop, implement, and scale innovations.

Use of convolutional neural networks in skin lesion analysis using real world image and non-image data

Background

Understanding performance of convolutional neural networks (CNNs) for binary (benign vs. malignant) lesion classification based on real world images is important for developing a meaningful clinical decision support (CDS) tool.

Methods

We developed a CNN based on real world smartphone images with histopathological ground truth and tested the utility of structured electronic health record (EHR) data on model performance. Model accuracy was compared against three board-certified dermatologists for clinical validity.

Results

At a classification threshold of 0.5, the sensitivity was 79 vs. 77 vs. 72%, and specificity was 64 vs. 65 vs. 57% for image-alone vs. combined image and clinical data vs. clinical data-alone models, respectively. The PPV was 68 vs. 69 vs. 62%, AUC was 0.79 vs. 0.79 vs. 0.69, and AP was 0.78 vs. 0.79 vs. 0.64 for image-alone vs. combined data vs. clinical data-alone models. Older age, male sex, and number of prior dermatology visits were important positive predictors for malignancy in the clinical data-alone model.

Conclusion

Additional clinical data did not significantly improve CNN image model performance. Model accuracy for predicting malignant lesions was comparable to dermatologists (model: 71.31% vs. 3 dermatologists: 77.87, 69.88, and 71.93%), validating clinical utility. Prospective validation of the model in primary care setting will enhance understanding of the model’s clinical utility.

Preliminary results of a clinical research and innovation scholarship to prepare medical students to lead innovations in health care

There is consensus amongst national organizations to integrate health innovation and augmented intelligence (AI) into medical education. However, there is scant evidence to guide policymakers and medical educators working to revise curricula. This study presents academic, operational, and domain understanding outcomes for the first three cohorts of participants in a clinical research and innovation scholarship program.

Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

Background

Machine learning models have the potential to improve diagnostic accuracy and management of acute conditions. Despite growing efforts to evaluate and validate such models, little is known about how to best translate and implement these products as part of routine clinical care.

Objective

This study aims to explore the factors influencing the integration of a machine learning sepsis early warning system (Sepsis Watch) into clinical workflows.

Methods

We conducted semistructured interviews with 15 frontline emergency department physicians and rapid response team nurses who participated in the Sepsis Watch quality improvement initiative. Interviews were audio recorded and transcribed. We used a modified grounded theory approach to identify key themes and analyze qualitative data.

Results

A total of 3 dominant themes emerged: perceived utility and trust, implementation of Sepsis Watch processes, and workforce considerations. Participants described their unfamiliarity with machine learning models. As a result, clinician trust was influenced by the perceived accuracy and utility of the model from personal program experience. Implementation of Sepsis Watch was facilitated by the easy-to-use tablet application and communication strategies that were developed by nurses to share model outputs with physicians. Barriers included the flow of information among clinicians and gaps in knowledge about the model itself and broader workflow processes.

Conclusions

This study generated insights into how frontline clinicians perceived machine learning models and the barriers to integrating them into clinical workflows. These findings can inform future efforts to implement machine learning interventions in real-world settings and maximize the adoption of these interventions.

Implementation of Electronic Patient-Reported Outcomes in Routine Cancer Care at an Academic Center: Identifying Opportunities and Challenges

PURPOSE:

Electronic patient-reported outcomes (ePROs) can help clinicians proactively assess and manage their patients’ symptoms. Despite known benefits, there is limited adoption of ePROs into routine clinical care as a result of workflow and technologic challenges. This study identifies oncologists’ perspectives on factors that affect integration of ePROs into clinical workflows.

METHODS:

We conducted semistructured qualitative interviews with 16 oncologists from a large academic medical center, across diverse subspecialties and cancer types. Oncologists were asked how they currently use or could imagine using ePROs before, during, and after a patient visit. We used an inductive approach to thematically analyze these qualitative data.

RESULTS:

Results were categorized into the following three main themes: (1) selection and development of ePRO tool, (2) contextual drivers of adoption, and (3) patient-facing concerns. Respondents preferred diagnosis-based ePRO tools over more general symptom screeners. Although they noted information overload as a potential barrier, respondents described strong data visualization and ease of use as facilitators. Contextual drivers of oncologist adoption include identifying target early adopters, incentivizing uptake through use of ePRO data to support billing and documentation, and emphasizing benefits for patient care and efficiency. Respondents also indicated the need to focus on patient-facing issues, such as patient response rate, timing of survey distribution, and validity and reliability of responses.

DISCUSSION:

Respondents identified several barriers and facilitators to successful uptake of ePROs. Understanding oncologists’ perspectives is essential to inform both practice-level implementation strategies and policy-level decisions to include ePROs in alternative payment models for cancer care.

Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study

Background

Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature.

Objective

This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care.

Methods

In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch.

Results

Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch.

Conclusions

Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

Integration of electronic patient-reported outcomes into clinical workflows within the Epic electronic medical record

102

Background: Electronic patient-reported outcome measures (ePROs) offer a new strategy for symptom assessment that can improve quality of life and prolong survival in routine cancer care. However, ePRO systems are often separate from existing electronic medical records (EMRs) and not well integrated into oncology clinics. In this pilot project, we assessed the feasibility and utility of integrating ePROs into our existing EMR and clinical workflows. Methods: The 10-question Edmonton Symptom Assessment Scale (ESAS) was integrated into the Epic EMR at three outpatient clinics in the Duke Cancer Institute. Patients with active MyChart accounts were offered the ESAS survey prior to their visit, via the patient portal. ePRO data were routed to clinicians in tabular and graphical formats. A “SmartPhrase” facilitated easy data integration into clinical notes. We subsequently interviewed clinicians and optimized workflows. Several patient engagement strategies were used, including automated messages, phone call reminders, and electronic tablets, to increase response rate. Results: It was feasible to quickly customize and activate an ePRO in Epic. Over 10 months, 161 patients completed 208 ePRO surveys. Initially, 10-20% of eligible patients completed the MyChart questionnaire. Patient engagement strategies, including phone calls and personalized MyChart messages, had little effect. Ultimately, tablets were introduced in the clinic check-in process, increasing response rates to >90%. Clinicians reported positive regard for the system, and an impact on patient symptom management. Clinician workflow optimization resulted in minimal “clicks” in the EMR, and the SmartPhrase was used in 128 clinical notes. Conclusions: Integration of ePROs into the clinical setting poses three challenges: technical implementation, workflow optimization, and patient engagement. While technical implementation is important, it was the easiest to solve, with patient engagement as the greatest barrier. Clinicians value an integrated ePRO system that automatically routes data to the clinical note. The key to successful ePRO integration is in ease of use for both patients and clinicians.