Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities

The use of data-driven technologies such as Artificial Intelligence (AI) and Machine Learning (ML) is growing in healthcare. However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use. To address these gaps and tackle a common challenge faced by healthcare delivery organizations, a case-based workshop was organized, and a framework was developed to evaluate the potential impact of implementing an AI solution on health equity. The Health Equity Across the AI Lifecycle (HEAAL) is co-designed with extensive engagement of clinical, operational, technical, and regulatory leaders across healthcare delivery organizations and ecosystem partners in the US. It assesses 5 equity assessment domains-accountability, fairness, fitness for purpose, reliability and validity, and transparency-across the span of eight key decision points in the AI adoption lifecycle. It is a process-oriented framework containing 37 step-by-step procedures for evaluating an existing AI solution and 34 procedures for evaluating a new AI solution in total. Within each procedure, it identifies relevant key stakeholders and data sources used to conduct the procedure. HEAAL guides how healthcare delivery organizations may mitigate the potential risk of AI solutions worsening health inequities. It also informs how much resources and support are required to assess the potential impact of AI solutions on health inequities.

Establishing priorities for implementation of large language models in pathology and laboratory medicine

Artificial intelligence and machine learning have numerous applications in pathology and laboratory medicine. The release of ChatGPT prompted speculation regarding the potentially transformative role of large-language models (LLMs) in academic pathology, laboratory medicine, and pathology education. Because of the potential to improve LLMs over the upcoming years, pathology and laboratory medicine clinicians are encouraged to embrace this technology, identify pathways by which LLMs may support our missions in education, clinical practice, and research, participate in the refinement of AI modalities, and design user-friendly interfaces that integrate these tools into our most important workflows. Challenges regarding the use of LLMs, which have already received considerable attention in a general sense, are also reviewed herein within the context of the pathology field and are important to consider as LLM applications are identified and operationalized.

Large Language Models and Medical Education: Preparing for a Rapid Transformation in How Trainees Will Learn to Be Doctors

Artificial intelligence has the potential to revolutionize health care but has yet to be widely implemented. In part, this may be because, to date, we have focused on easily predicted rather than easily actionable problems. Large language models (LLMs) represent a paradigm shift in our approach to artificial intelligence because they are easily accessible and already being tested by frontline clinicians, who are rapidly identifying possible use cases. LLMs in health care have the potential to reduce clerical work, bridge gaps in patient education, and more. As we enter this era of healthcare delivery, LLMs will present both opportunities and challenges in medical education. Future models should be developed to support trainees to develop skills in clinical reasoning, encourage evidence-based medicine, and offer case-based training opportunities. LLMs may also change what we continue teaching trainees with regard to clinical documentation. Finally, trainees can help us train and develop the LLMs of the future as we consider the best ways to incorporate LLMs into medical education. Ready or not, LLMs will soon be integrated into various aspects of clinical practice, and we must work closely with students and educators to make sure these models are also built with trainees in mind to responsibly chaperone medical education into the next era.

Gender Disparities in Critical Care Procedure Training of Internal Medicine Residents

Background

Procedural training is a required competency in internal medicine (IM) residency, yet limited data exist on residents' experience of procedural training.

Objectives

We sought to understand how gender impacts access to procedural training among IM residents.

Methods

A mixed-methods, explanatory sequential study was performed. Procedure volume for IM residents between 2016 and 2020 was assessed at two large academic residencies (Program A and Program B: 399 residents and 4,020 procedures). Procedural rates and actual versus expected procedure volume by gender were compared, with separate analyses by clinical environment (intensive care unit [ICU] or structured procedural service). Semistructured gender-congruent focus groups were conducted. Topics included identity formation as a proceduralist and the resident procedural learning experience, including perceived gender bias in procedure allocation.

Results

Compared with men, women residents performed disproportionately fewer ICU procedures per month at Program A (1.4 vs. 2.7; P < 0.05) but not at Program B (0.36 vs. 0.54; P = 0.23). At Program A, women performed only 47% of ICU procedures, significantly fewer than the 54% they were expected to perform on the basis of their time on ICU rotations (P < 0.001). For equal gender distribution of procedural volume at Program A, 11% of the procedures performed by men would have needed to have been performed by women instead. Gender was not associated with differences in the Program A structured procedural service (53% observed vs. 52% expected; P = 0.935), Program B structured procedural service (40% observed vs. 43% expected; P = 0.174), or in Program B ICUs (33% observed vs. 34% expected; P = 0.656). Focus group analysis identified that women from both residencies perceived that assertiveness was required for procedural training in unstructured learning environments. Residents felt that gender influenced access to procedural opportunities, ability to self-advocate for procedural experience, identity formation as a proceduralist, and confidence in acquiring procedural skills.

Conclusion

Gender disparities in access to procedural training during ICU rotations were seen at one institution but not another. There were ubiquitous perceptions that assertiveness was important to access procedural opportunities. We hypothesize that structured allocation of procedures would mitigate disparities by allowing all residents to access procedural training regardless of self-advocacy. Residency programs should adopt structured procedural training programs to counteract inequities.

Protecting reproductive health information in the post-Roe era: interoperability strategies for healthcare institutions

On June 24, 2022, the US Supreme Court ended constitutional protections for abortion, resulting in wide variability in access from severe restrictions in many states and fewer restrictions in others. Healthcare institutions capture information about patients' pregnancy and abortion care and, due to interoperability, may share it in ways that expose their providers and patients to social stigma and potential legal jeopardy in states with severe restrictions. In this article, we describe sources of risk to patients and providers that arise from interoperability and specify actions that institutions can take to reduce that risk. Institutions have significant power to define their practices for how and where care is documented, how patients are identified, where data are sent or hosted, and how patients are counseled, and thus should protect patients' privacy and ability to receive medical care that is safe and legal where it is performed.

Characterizing styles of clinical note production and relationship to clinical work hours among first-year residents

OBJECTIVE

To characterize variation in clinical documentation production patterns, how this variation relates to individual resident behavior preferences, and how these choices relate to work hours.

MATERIALS AND METHODS

We used unsupervised machine learning with clinical note metadata for 1265 progress notes written for 279 patient encounters by 50 first-year residents on the Hospital Medicine service in 2018 to uncover distinct note-level and user-level production patterns. We examined average and 95% confidence intervals of median user daily work hours measured from audit log data for each user-level production pattern.

RESULTS

Our analysis revealed 10 distinct note-level and 5 distinct user-level production patterns (user styles). Note production patterns varied in when writing occurred and in how dispersed writing was through the day. User styles varied in which note production pattern(s) dominated. We observed suggestive trends in work hours for different user styles: residents who preferred producing notes in dispersed sessions had higher median daily hours worked while residents who preferred producing notes in the morning or in a single uninterrupted session had lower median daily hours worked.

DISCUSSION

These relationships suggest that note writing behaviors should be further investigated to understand what practices could be targeted to reduce documentation burden and derivative outcomes such as resident work hour violations.

CONCLUSION

Clinical note documentation is a time-consuming activity for physicians; we identify substantial variation in how first-year residents choose to do this work and suggestive trends between user preferences and work hours.

Estimation of Surgical Resident Duty Hours and Workload in Real Time Using Electronic Health Record Data

OBJECTIVE

To explore the use of electronic health record (EHR) data to estimate surgery resident duty hours and monitor real time workload.

DESIGN

Retrospective analysis of resident duty hours logged using a voluntary global positioning system (GPS)-based smartphone application compared to duty hour estimates by an EHR-based algorithm. The algorithm estimated duty hours using EHR activity data and operating room logs. A dashboard was developed through Plan-Do-Study-Act cycles for real-time monitoring of workload.

SETTING

Single tertiary/quaternary medical center general surgery residency program with approximately 90 categorical, preliminary, and integrated residents at eight clinical sites.

PARTICIPANTS

Categorical, preliminary, and integrated surgery residents of all clinical years who volunteered to pilot a GPS application to track duty hours.

RESULTS

Of 2,623 work periods by 59 residents were logged with both methods. EHR-estimated work periods started later than GPS logs (median 0.3 hours, interquartile range [IQR] -0.1 - 0.3); EHR-estimated work periods ended earlier than GPS logs (median 0.1 hours, IQR -0.7 - 0.3); and EHR-estimated duty hour totals were less than totals logged by GPS (median -0.3 hours, IQR -0.8 - +0.1). Overall correlation between weekly duty hours logged by EHR and GPS was 0.79. Correlations between the 2 systems stratified from PGY-1 through PGY-5 were 0.76, 0.64, 0.82, 0.87, and 0.83, respectively. The algorithm identified six 80-hour workweek violations (averaged over 4 weeks), while GPS logs identified 8. EHR-based duty hours and operational data were integrated into a dashboard to enable real time monitoring of resident workloads.

CONCLUSIONS

EHR-based estimation of surgical resident duty hours has good correlation with GPS-based assessment of duty hours and identifies most workweek duty hour violations. This approach allows for dynamic workload monitoring and may be combined with operational data to anticipate and prevent duty hour violations, thereby optimizing learning.

Automating Measurement of Trainee Work Hours

BACKGROUND

Medical training programs across the country are bound to a set of work hour regulations, generally monitored via self-report.

OBJECTIVE

We developed a computational method to automate measurement of intern and resident work hours, which we validated against self-report.

DESIGN, SETTING, AND PARTICIPANTS

We included all electronic health record (EHR) access log data between July 1, 2018, and June 30, 2019, for trainees enrolled in the internal medicine training program. We inferred the duration of continuous in-hospital work hours by linking EHR sessions that occurred within 5 hours as "on-campus" work and further accounted for "out-of-hospital" work which might be taking place at home.

MAIN OUTCOMES AND MEASURES

We compared daily work hours estimated through the computational method with self-report and calculated the mean absolute error between the two groups. We used the computational method to estimate average weekly work hours across the rotation and the percentage of rotations where average work hours exceed the 80-hour workweek.

RESULTS

The mean absolute error between self-reported and EHR-derived daily work hours for first- (PGY-1), second- (PGY-2), and third- (PGY-3) year trainees were 1.27, 1.51, and 1.51 hours, respectively. Using this computational method, we estimated average (SD) weekly work hours of 57.0 (21.7), 69.9 (12.2), and 64.1 (16.3) for PGY-1, PGY-2, and PGY-3 residents.

CONCLUSION

EHR log data can be used to accurately approximate self-report of work hours, accounting for both in-hospital and out-of-hospital work. Automation will reduce trainees' clerical work, improve consistency and comparability of data, and provide more complete and timely data that training programs need.

Decreasing Hypoglycemia following Insulin Administration for Inpatient Hyperkalemia

BACKGROUND

Acute hyperkalemia (serum potassium ≥ 5.1 mEq/L) is often treated with a bolus of IV insulin. This treatment may result in iatrogenic hypoglycemia (glucose < 70 mg/dl).

OBJECTIVES

The aims of this study were to accurately determine the frequency of iatrogenic hypoglycemia following insulin treatment for hyperkalemia, and to develop an electronic health record (EHR) orderset to decrease the risk for iatrogenic hypoglycemia.

DESIGN

This study was an observational, prospective study.

SETTING

The setting for this study was a university hospital.

PATIENTS

All nonobstetric adult inpatients in all acute and intensive care units were eligible.

INTERVENTION

Implementation of a hyperkalemia orderset (Orderset 1.1) with glucose checks before and then one, two, four, and six hours after regular intravenous insulin administration. Based on the results from Orderset 1.1, Orderset 1.2 was developed and introduced to include weight-based dosing of insulin options, alerts identifying patients at higher risk of hypoglycemia, and tools to guide decision-making based on the preinsulin blood glucose level.

MEASUREMENTS

Patient demographics, weight, diabetes history, potassium level, renal function, and glucose levels were recorded before, and then glucose levels were measured again at one, two, four, and six hours after insulin was administered.

RESULTS

The iatrogenic hypoglycemia rate identified with mandatory glucose checks in Orderset 1.1 was 21%; 92% of these occurred within three hours posttreatment. Risk factors for hypoglycemia included decreased renal function (serum creatinine >2.5 mg/dl), a high dose of insulin (>0.14 units/kg), and re-treatment with blood glucose < 140 mg/dl. After the introduction of Orderset 1.2, the rate of iatrogenic hypoglycemia decreased to 10%.

CONCLUSIONS

The use of an EHR orderset for treating hyperkalemia may reduce the risk of iatrogenic hypoglycemia in patients receiving insulin while still adequately lowering their potassium.

Automated and flexible identification of complex disease: building a model for systemic lupus erythematosus using noisy labeling

Accurate and efficient identification of complex chronic conditions in the electronic health record (EHR) is an important but challenging task that has historically relied on tedious clinician review and oversimplification of the disease. Here we adapt methods that allow for automated "noisy labeling" of positive and negative controls to create a "silver standard" for machine learning to automate identification of systemic lupus erythematosus (SLE). Our final model, which includes both structured data as well as text processing of clinical notes, outperformed all existing algorithms for SLE (AUC 0.97). In addition, we demonstrate how the probabilistic outputs of this model can be adapted to various clinical needs, selecting high thresholds when specificity is the priority and lower thresholds when a more inclusive patient population is desired. Deploying a similar methodology to other complex diseases has the potential to dramatically simplify the landscape of population identification in the EHR.

MeSH terms

Electronic Health Records, Machine Learning, Lupus Erythematosus, Phenotype, Algorithms.

Using Spatial and Temporal Mapping to Identify Nosocomial Disease Transmission of Clostridium difficile

This study evaluates the use of spatial and temporal mapping to identify nosocomial disease transmission of Clostridium difficile.

Correlation of hypogammaglobulinaemia with proteinuria, and the relationship between hypogammaglobulinaemia and infection in active lupus nephritis

Objective

To evaluate hypogammaglobulinaemia and risk of serious infectious adverse events in active lupus nephritis.

Methods

The Abatacept and Cyclophosphamide Combination Efficacy and Safety Study (ACCESS) compared abatacept with placebo in participants with lupus nephritis undergoing treatment with Euro-Lupus Nephritis low-dose cyclophosphamide. Serum IgG levels were assessed prior to initiation of treatment and throughout the trial. Hypogammaglobulinaemia was defined as IgG <450 mg/dL.

Results

Hypogammaglobulinaemia was observed in 16/102 (15.7%) participants prior to initiation of induction therapy for active lupus nephritis. Participants with nephrotic range proteinuria were more likely to have hypogammaglobulinaemia, and serum IgG levels were inversely correlated with urine protein to creatinine ratio (r=−0.42, p<0.0001). Following initiation of treatment for active lupus nephritis, additional participants developed hypogammaglobulinaemia by weeks 2–4. Serum IgG levels then increased, and all but one participant had serum IgG ≥450 mg/dL at 24 weeks. Hypogammaglobulinaemia was not associated with an increased risk of serious infectious adverse events.

Conclusions

In active lupus nephritis in ACCESS, hypogammaglobulinaemia was common and inversely correlated with proteinuria. Serum IgG levels were lowest in the weeks immediately following initiation of induction therapy, and subsequently improved by 24 weeks. Hypogammaglobulinaemia was not associated with serious infectious adverse events.

Trial registration

Cervical Spinal Fracture and Other Diagnoses Associated With Mortality in Hospitalized Ankylosing Spondylitis Patients

OBJECTIVE

Little data exist regarding mortality in ankylosing spondylitis (AS). We assessed diagnoses associated with in-hospital mortality in AS using a population-based inpatient data set.

METHODS

Data were abstracted from the Healthcare Cost and Utilization Project Nationwide Inpatient Sample between 2007 and 2011. We identified AS admissions using International Classification of Diseases, Ninth Revision, Clinical Modification code 720.0. In-hospital mortality was the primary outcome. Logistic regression was used to evaluate the association between top diagnoses and in-hospital mortality. We performed a secondary analysis from the same years in all patients (with and without AS) with cervical spine (C-spine) fracture to determine whether AS was an independent risk factor for mortality.

RESULTS

Between 2007 and 2011, we identified 12,484 admissions and 267 deaths in AS patients. C-spine fracture with spinal cord injury and sepsis had the highest odds of death, with odds ratios (ORs) of 13.43 (95% confidence interval [95% CI] 8.00-22.55, P < 0.0001) and 7.63 (95% CI 5.62-10.36, P < 0.0001), respectively. In the same time period, there were 53,606 C-spine fracture admissions, of which 408 were coded with AS. Among all C-spine fracture hospitalizations, an AS diagnosis was associated with inpatient death (OR 1.61 [95% CI 1.16-2.22], P = 0.004).

CONCLUSION

In AS patients admitted to the hospital, C-spine fracture is a leading cause of in-hospital mortality. Other diagnoses associated with mortality include sepsis, pneumonia, cardiovascular disease, and comorbid illnesses. Among all hospitalizations with C-spine fracture, AS was associated with increased odds of death. C-spine fracture-associated mortality warrants further study to elucidate risk factors in order to prevent such devastating fractures in AS patients.

National Lupus Hospitalization Trends Reveal Rising Rates of Herpes Zoster and Declines in Pneumocystis Pneumonia

OBJECTIVE

Infection is a leading cause of morbidity and mortality in systemic lupus erythematosus (SLE). Therapeutic practices have evolved over the past 15 years, but effects on infectious complications of SLE are unknown. We evaluated trends in hospitalizations for severe and opportunistic infections in a population-based SLE study.

METHODS

Data derive from the 2000 to 2011 United States National Inpatient Sample, including individuals who met a validated administrative definition of SLE. Primary outcomes were diagnoses of bacteremia, pneumonia, opportunistic fungal infection, herpes zoster, cytomegalovirus, or pneumocystis pneumonia (PCP). We used Poisson regression to determine whether infection rates were changing in SLE hospitalizations and used predictive marginals to generate annual adjusted rates of specific infections.

RESULTS

We identified 361,337 SLE hospitalizations from 2000 to 2011 meeting study inclusion criteria. Compared to non-SLE hospitalizations, SLE patients were younger (51 vs. 62 years), predominantly female (89% vs. 54%), and more likely to be racial/ethnic minorities. SLE diagnosis was significantly associated with all measured severe and opportunistic infections. From 2000 to 2011, adjusted SLE hospitalization rates for herpes zoster increased more than non-SLE rates: 54 to 79 per 10,000 SLE hospitalizations compared with 24 to 29 per 10,000 non-SLE hospitalizations. Conversely, SLE hospitalizations for PCP disproportionately decreased: 5.1 to 2.5 per 10,000 SLE hospitalizations compared with 0.9 to 1.3 per 10,000 non-SLE hospitalizations.

CONCLUSIONS

Among patients with SLE, herpes zoster hospitalizations are rising while PCP hospitalizations are declining. These trends likely reflect evolving SLE treatment strategies. Further research is needed to identify patients at greatest risk for infectious complications.

A population-based study of infection-related hospital mortality in patients with dermatomyositis/polymyositis

OBJECTIVE

Dermatomyositis (DM) and polymyositis (PM) are debilitating inflammatory myopathies associated with significant mortality. We evaluated the relative contribution of infection to hospital mortality in a large population-based study of individuals with PM/DM.

METHODS

Data derive from the 2007 to 2011 Healthcare Cost and Utilization Project National Inpatient Samples and include all hospital discharges that met a validated administrative definition of PM/DM. The primary outcome was hospital mortality. Variables for infections and comorbidities were generated from discharge diagnoses using validated administrative definitions. Logistic regression was used to investigate the relationship between infection and mortality in individuals with PM/DM, adjusting for sociodemographics, utilization variables, and comorbidities. Relative risks (RRs) were calculated to compare the overall prevalence of specific infections and associated mortality in PM/DM hospitalizations with those seen in the general hospitalized population.

RESULTS

A total of 15,407 hospitalizations with PM/DM met inclusion criteria for this study and inpatient mortality was 4.5% (700 deaths). In adjusted logistic regression analyses, infection (odds ratio [OR] 3.4, 95% confidence interval [95% CI] 2.9-4.0) was the strongest predictor of hospital mortality among individuals with PM/DM. Bacterial infection (OR 3.5, 95% CI 3.0-4.1), comprised primarily of pneumonia and bacteremia, and opportunistic fungal infections (OR 2.5, 95% CI 1.5-4.0) were independently associated with hospital mortality. The overall burden of infection in hospitalizations with PM/DM was significantly increased in comparison with the general hospitalized population (RR 1.5, 95% CI 1.4-1.6).

CONCLUSION

Among hospitalized individuals with PM/DM, infection is the leading cause of mortality. Strategies to mitigate infection risk in both the clinic and hospital settings should be evaluated to improve disease outcomes.

Cardiovascular disease and cognitive dysfunction in systemic lupus erythematosus

OBJECTIVE

Cognitive dysfunction and cardiovascular disease are common and debilitating manifestations of systemic lupus erythematosus (SLE). In this study, we evaluated the relationship between cardiovascular events, traditional cardiovascular risk factors, and SLE-specific risk factors as predictors of cognitive dysfunction in a large cohort of participants with SLE.

METHODS

Subjects included 694 participants from the Lupus Outcomes Study (LOS), a longitudinal study of SLE outcomes based on an annual telephone survey querying demographic and clinical variables. The Hopkins Verbal Learning Test-Revised and the Controlled Oral Word Association Test were administered to assess cognitive function. Multiple logistic regression was used to identify cardiovascular events (myocardial infarction, stroke), traditional cardiovascular risk factors (hypertension, hyperlipidemia, diabetes mellitus, obesity, smoking), and SLE-specific risk factors (antiphospholipid antibodies [aPL], disease activity, disease duration) associated with cognitive impairment in year 7 of the LOS.

RESULTS

The prevalence of cognitive impairment as measured by verbal memory and verbal fluency metrics was 15%. In adjusted multiple logistic regression analyses, aPL (odds ratio [OR] 2.10, 95% confidence interval [95% CI] 1.3-3.41), hypertension (OR 2.06, 95% CI 1.19-3.56), and a history of stroke (OR 2.27, 95% CI 1.16-4.43) were significantly associated with cognitive dysfunction. In additional analyses evaluating the association between these predictors and severity of cognitive impairment, stroke was significantly more prevalent in participants with severe impairment when compared to those with mild or moderate impairment (P = 0.036).

CONCLUSION

These results suggest that the presence of aPL, hypertension, and stroke are key variables associated with cognitive impairment, which may aid in identification of patients at greatest risk.

The relationship of some copper (II) complexes of facultative tetrathioethers to the coordination environment in the "blue" copper proteins

The facultative potentially tetradentate thioether ligands 1,2-bis(methylthioethylthio)ethane (2,2,2), 1,3-bis(2-methylthioethylthio)propane (2,3,2) and 1,2-bis(3-methylthiopropylthio)ethane (3,2,3) react with copper(II) salts to form Cu2(2,2,2)Cl4, Cu3(ligand)X6 (ligand = 2,3,2 and 3,2,3 X = Cl; ligand = 2,2,2 2,3,2 and 3,2,3 X = Br), and Cu(ligand)(ClO4)2. The stoichiometry and structures of these complexes are discussed in terms of the steric demands of the ligand and the nature of the halide. The [Cu(2,3,2)(ClO4)] ClO4 and [Cu(3,2,3)(ClO4) [ClO4 complexes have electronic spectra which exhibit the intense 600 nm band characteristic of the "blue" copper proteins. In fact, the spectrum of [Cu(2,3,2)(ClO4)]ClO4 is very similar to that of pseudomonas aeroginosa azurin.