Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

Artificial intelligence in early detection and prediction of pediatric/neonatal acute kidney injury: current status and future directions

Acute kidney injury (AKI) has a significant impact on the short-term and long-term clinical outcomes of pediatric and neonatal patients, and it is imperative in these populations to mitigate the pathways leading to AKI and be prepared for early diagnosis and treatment intervention of established AKI. Recently, artificial intelligence (AI) has provided more advent predictive models for early detection/prediction of AKI utilizing machine learning (ML). By providing strong detail and evidence from risk scores and electronic alerts, this review outlines a comprehensive and holistic insight into the current state of AI in AKI in pediatric/neonatal patients. In the pediatric population, AI models including XGBoost, logistic regression, support vector machines, decision trees, naïve Bayes, and risk stratification scores (Renal Angina Index (RAI), Nephrotoxic Injury Negated by Just-in-time Action (NINJA)) have shown success in predicting AKI using variables like serum creatinine, urine output, and electronic health record (EHR) alerts. Similarly, in the neonatal population, using the "Baby NINJA" model showed a decrease in nephrotoxic medication exposure by 42%, the rate of AKI by 78%, and the number of days with AKI by 68%. Furthermore, the "STARZ" risk stratification AI model showed a predictive ability of AKI within 7 days of NICU admission of AUC 0.93 and AUC of 0.96 in the validation and derivation cohorts, respectively. Many studies have reported the superiority of using biomarkers to predict AKI in pediatric patients and neonates as well. Future directions include the application of AI along with biomarkers (NGAL, CysC, OPN, IL-18, B2M, etc.) in a Labelbox configuration to create a more robust and accurate model for predicting and detecting pediatric/neonatal AKI.

Determining Prior Authorization Approval for Lumbar Stenosis Surgery With Machine Learning

STUDY DESIGN

Medical vignettes.

OBJECTIVES

Lumbar spinal stenosis (LSS) is a degenerative condition with a high prevalence in the elderly population, that is associated with a significant economic burden and often requires spinal surgery. Prior authorization of surgical candidates is required before patients can be covered by a health plan and must be approved by medical directors (MDs), which is often subjective and clinician specific. In this study, we hypothesized that the prediction accuracy of machine learning (ML) methods regarding surgical candidates is comparable to that of a panel of MDs.

METHODS

Based on patient demographic factors, previous therapeutic history, symptoms and physical examinations and imaging findings, we propose an ML which computes the probability of spinal surgical recommendations for LSS. The model implements a random forest model trained from medical vignette data reviewed by MDs. Sets of 400 and 100 medical vignettes reviewed by MDs were used for training and testing.

RESULTS

The predictive accuracy of the machine learning model was with a root mean square error (RMSE) between model predictions and ground truth of .1123, while the average RMSE between individual MD's recommendations and ground truth was .2661. For binary classification, the AUROC and Cohen's kappa were .959 and .801, while the corresponding average metrics based on individual MD's recommendations were .844 and .564, respectively.

CONCLUSIONS

Our results suggest that ML can be used to automate prior authorization approval of surgery for LSS with performance comparable to a panel of MDs.

Good machine learning practices: Learnings from the modern pharmaceutical discovery enterprise

Machine Learning (ML) and Artificial Intelligence (AI) have become an integral part of the drug discovery and development value chain. Many teams in the pharmaceutical industry nevertheless report the challenges associated with the timely, cost effective and meaningful delivery of ML and AI powered solutions for their scientists. We sought to better understand what these challenges were and how to overcome them by performing an industry wide assessment of the practices in AI and Machine Learning. Here we report results of the systematic business analysis of the personas in the modern pharmaceutical discovery enterprise in relation to their work with the AI and ML technologies. We identify 23 common business problems that individuals in these roles face when they encounter AI and ML technologies at work, and describe best practices (Good Machine Learning Practices) that address these issues.

The Effect of Artificial Intelligence on Patient-Physician Trust: Cross-Sectional Vignette Study

BACKGROUND

Clinical decision support systems (CDSSs) based on routine care data, using artificial intelligence (AI), are increasingly being developed. Previous studies focused largely on the technical aspects of using AI, but the acceptability of these technologies by patients remains unclear.

OBJECTIVE

We aimed to investigate whether patient-physician trust is affected when medical decision-making is supported by a CDSS.

METHODS

We conducted a vignette study among the patient panel (N=860) of the University Medical Center Utrecht, the Netherlands. Patients were randomly assigned into 4 groups-either the intervention or control groups of the high-risk or low-risk cases. In both the high-risk and low-risk case groups, a physician made a treatment decision with (intervention groups) or without (control groups) the support of a CDSS. Using a questionnaire with a 7-point Likert scale, with 1 indicating "strongly disagree" and 7 indicating "strongly agree," we collected data on patient-physician trust in 3 dimensions: competence, integrity, and benevolence. We assessed differences in patient-physician trust between the control and intervention groups per case using Mann-Whitney U tests and potential effect modification by the participant's sex, age, education level, general trust in health care, and general trust in technology using multivariate analyses of (co)variance.

RESULTS

In total, 398 patients participated. In the high-risk case, median perceived competence and integrity were lower in the intervention group compared to the control group but not statistically significant (5.8 vs 5.6; P=.16 and 6.3 vs 6.0; P=.06, respectively). However, the effect of a CDSS application on the perceived competence of the physician depended on the participant's sex (P=.03). Although no between-group differences were found in men, in women, the perception of the physician's competence and integrity was significantly lower in the intervention compared to the control group (P=.009 and P=.01, respectively). In the low-risk case, no differences in trust between the groups were found. However, increased trust in technology positively influenced the perceived benevolence and integrity in the low-risk case (P=.009 and P=.04, respectively).

CONCLUSIONS

We found that, in general, patient-physician trust was high. However, our findings indicate a potentially negative effect of AI applications on the patient-physician relationship, especially among women and in high-risk situations. Trust in technology, in general, might increase the likelihood of embracing the use of CDSSs by treating professionals.

Exploring the Role of Artificial Intelligence in Mental Healthcare: Current Trends and Future Directions - A Narrative Review for a Comprehensive Insight

Mental health is an essential component of the health and well-being of a person and community, and it is critical for the individual, society, and socio-economic development of any country. Mental healthcare is currently in the health sector transformation era, with emerging technologies such as artificial intelligence (AI) reshaping the screening, diagnosis, and treatment modalities of psychiatric illnesses. The present narrative review is aimed at discussing the current landscape and the role of AI in mental healthcare, including screening, diagnosis, and treatment. Furthermore, this review attempted to highlight the key challenges, limitations, and prospects of AI in providing mental healthcare based on existing works of literature. The literature search for this narrative review was obtained from PubMed, Saudi Digital Library (SDL), Google Scholar, Web of Science, and IEEE Xplore, and we included only English-language articles published in the last five years. Keywords used in combination with Boolean operators ("AND" and "OR") were the following: "Artificial intelligence", "Machine learning", Deep learning", "Early diagnosis", "Treatment", "interventions", "ethical consideration", and "mental Healthcare". Our literature review revealed that, equipped with predictive analytics capabilities, AI can improve treatment planning by predicting an individual's response to various interventions. Predictive analytics, which uses historical data to formulate preventative interventions, aligns with the move toward individualized and preventive mental healthcare. In the screening and diagnostic domains, a subset of AI, such as machine learning and deep learning, has been proven to analyze various mental health data sets and predict the patterns associated with various mental health problems. However, limited studies have evaluated the collaboration between healthcare professionals and AI in delivering mental healthcare, as these sensitive problems require empathy, human connections, and holistic, personalized, and multidisciplinary approaches. Ethical issues, cybersecurity, a lack of data analytics diversity, cultural sensitivity, and language barriers remain concerns for implementing this futuristic approach in mental healthcare. Considering these sensitive problems require empathy, human connections, and holistic, personalized, and multidisciplinary approaches, it is imperative to explore these aspects. Therefore, future comparative trials with larger sample sizes and data sets are warranted to evaluate different AI models used in mental healthcare across regions to fill the existing knowledge gaps.

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.

Accuracy and consistency of chatbots versus clinicians for answering pediatric dentistry questions: A pilot study

OBJECTIVES

Artificial Intelligence has applications such as Large Language Models (LLMs), which simulate human-like conversations. The potential of LLMs in healthcare is not fully evaluated. This pilot study assessed the accuracy and consistency of chatbots and clinicians in answering common questions in pediatric dentistry.

METHODS

Two expert pediatric dentists developed thirty true or false questions involving different aspects of pediatric dentistry. Publicly accessible chatbots (Google Bard, ChatGPT4, ChatGPT 3.5, Llama, Sage, Claude 2 100k, Claude-instant, Claude-instant-100k, and Google Palm) were employed to answer the questions (3 independent new conversations). Three groups of clinicians (general dentists, pediatric specialists, and students; n = 20/group) also answered. Responses were graded by two pediatric dentistry faculty members, along with a third independent pediatric dentist. Resulting accuracies (percentage of correct responses) were compared using analysis of variance (ANOVA), and post-hoc pairwise group comparisons were corrected using Tukey's HSD method. ACronbach's alpha was calculated to determine consistency.

RESULTS

Pediatric dentists were significantly more accurate (mean±SD 96.67 %± 4.3 %) than other clinicians and chatbots (p < 0.001). General dentists (88.0 % ± 6.1 %) also demonstrated significantly higher accuracy than chatbots (p < 0.001), followed by students (80.8 %±6.9 %). ChatGPT showed the highest accuracy (78 %±3 %) among chatbots. All chatbots except ChatGPT3.5 showed acceptable consistency (Cronbach alpha>0.7).

CLINICAL SIGNIFICANCE

Based on this pilot study, chatbots may be valuable adjuncts for educational purposes and for distributing information to patients. However, they are not yet ready to serve as substitutes for human clinicians in diagnostic decision-making.

CONCLUSION

In this pilot study, chatbots showed lower accuracy than dentists. Chatbots may not yet be recommended for clinical pediatric dentistry.

Contextualizing Positionality, Intersectionality, and Intelligence in the Anthropocene

The geological epoch of the Anthropocene has challenged traditional definitions of what intellectual abilities are necessary to creatively problem-solve, understand, and address contemporary societal and environmental crises. If we hope to make meaningful changes to how our society addresses these complex issues and pave the way for a better future for generations to come, we must advance traditional theories and measures of higher-order abilities to reflect equity and inclusion. To this end, we must address global issues by integrating the complexities of intersectional identities as they impact our understanding of what constitutes intelligence in individuals, groups, and diverse communities. This re-envisioning of intelligence presents new complexities for understanding and challenges for our field beyond the boundaries of what has been previously touted by many disciplines, including psychology. It is an opportunity to re-envision what it means to be intelligent in a diverse global context while also honoring and recognizing the value of difference, positionality, and other ways of knowing.

An evaluation of synthetic data augmentation for mitigating covariate bias in health data

Data bias is a major concern in biomedical research, especially when evaluating large-scale observational datasets. It leads to imprecise predictions and inconsistent estimates in standard regression models. We compare the performance of commonly used bias-mitigating approaches (resampling, algorithmic, and post hoc approaches) against a synthetic data-augmentation method that utilizes sequential boosted decision trees to synthesize under-represented groups. The approach is called synthetic minority augmentation (SMA). Through simulations and analysis of real health datasets on a logistic regression workload, the approaches are evaluated across various bias scenarios (types and severity levels). Performance was assessed based on area under the curve, calibration (Brier score), precision of parameter estimates, confidence interval overlap, and fairness. Overall, SMA produces the closest results to the ground truth in low to medium bias (50% or less missing proportion). In high bias (80% or more missing proportion), the advantage of SMA is not obvious, with no specific method consistently outperforming others.

Oral and oropharyngeal cancer screening and tobacco cessation discussions, NHANES 2011-2018

OBJECTIVE

Oral cavity and oropharyngeal cancer (OOPC) is a devastating disease often caught in late stages. People who use tobacco are at higher risk of OOPC. Tobacco cessation discussions and OOPC screenings are important factors in decreasing the risk of OOPC or its late stage diagnosis. As research on sex differences has been increasing-from research on biomedical to psychological and sociological determinants-there is a potential difference, by sex, as to whom is more likely to have a tobacco cessation discussion and OOPC screening. The objective of this study is to determine if there is an association of sex with tobacco cessation discussions and OOPC screenings conducted by dental healthcare professionals among participants who currently use tobacco.

METHOD

Data from 8 years of the National Health and Nutrition Examination Survey (2011-2018) were merged. Data from participants, ages 30 years and above, who self-reported current use of tobacco, a dental visit within the previous year and responsed to questions about oral cancer screening were analysed for frequency determination and logistic regression analysis. Having the combination of neither OOPC screening nor discussion about the benefits of not using tobacco was the outcome in the analysis.

RESULTS

There were 22.1% who had an OOPC screening by a dental professional within the previous year. Of the 41% who reported having had a conversation with a dental professional within the previous year about the benefits of tobacco cessation, 9.8% reported having both the conversation and OOPC screening. Males were less likely than females to have the combination of neither OOPC screening nor advice about tobacco cessation than females (adjusted odds ratio: 0.74; 95%CI: 0.57, 0.96).

CONCLUSION

There is an increased need for OOPC screening and the discussion of tobacco use by dental professionals among their patients who use tobacco, particularly for female patients.

Sex-specific differences in physiological parameters related to SARS-CoV-2 infections among a national cohort (COVI-GAPP study)

Considering sex as a biological variable in modern digital health solutions, we investigated sex-specific differences in the trajectory of four physiological parameters across a COVID-19 infection. A wearable medical device measured breathing rate, heart rate, heart rate variability, and wrist skin temperature in 1163 participants (mean age = 44.1 years, standard deviation [SD] = 5.6; 667 [57%] females). Participants reported daily symptoms and confounders in a complementary app. A machine learning algorithm retrospectively ingested daily biophysical parameters to detect COVID-19 infections. COVID-19 serology samples were collected from all participants at baseline and follow-up. We analysed potential sex-specific differences in physiology and antibody titres using multilevel modelling and t-tests. Over 1.5 million hours of physiological data were recorded. During the symptomatic period of infection, men demonstrated larger increases in skin temperature, breathing rate, and heart rate as well as larger decreases in heart rate variability than women. The COVID-19 infection detection algorithm performed similarly well for men and women. Our study belongs to the first research to provide evidence for differential physiological responses to COVID-19 between females and males, highlighting the potential of wearable technology to inform future precision medicine approaches.

Innovative applications of artificial intelligence during the COVID-19 pandemic

The COVID-19 pandemic has created unprecedented challenges worldwide. Artificial intelligence (AI) technologies hold tremendous potential for tackling key aspects of pandemic management and response. In the present review, we discuss the tremendous possibilities of AI technology in addressing the global challenges posed by the COVID-19 pandemic. First, we outline the multiple impacts of the current pandemic on public health, the economy, and society. Next, we focus on the innovative applications of advanced AI technologies in key areas such as COVID-19 prediction, detection, control, and drug discovery for treatment. Specifically, AI-based predictive analytics models can use clinical, epidemiological, and omics data to forecast disease spread and patient outcomes. Additionally, deep neural networks enable rapid diagnosis through medical imaging. Intelligent systems can support risk assessment, decision-making, and social sensing, thereby improving epidemic control and public health policies. Furthermore, high-throughput virtual screening enables AI to accelerate the identification of therapeutic drug candidates and opportunities for drug repurposing. Finally, we discuss future research directions for AI technology in combating COVID-19, emphasizing the importance of interdisciplinary collaboration. Though promising, barriers related to model generalization, data quality, infrastructure readiness, and ethical risks must be addressed to fully translate these innovations into real-world impacts. Multidisciplinary collaboration engaging diverse expertise and stakeholders is imperative for developing robust, responsible, and human-centered AI solutions against COVID-19 and future public health emergencies.

Proposing a Principle-Based Approach for Teaching AI Ethics in Medical Education

The use of artificial intelligence (AI) in medicine, potentially leading to substantial advancements such as improved diagnostics, has been of increasing scientific and societal interest in recent years. However, the use of AI raises new ethical challenges, such as an increased risk of bias and potential discrimination against patients, as well as misdiagnoses potentially leading to over- or underdiagnosis with substantial consequences for patients. Recognizing these challenges, current research underscores the importance of integrating AI ethics into medical education. This viewpoint paper aims to introduce a comprehensive set of ethical principles for teaching AI ethics in medical education. This dynamic and principle-based approach is designed to be adaptive and comprehensive, addressing not only the current but also emerging ethical challenges associated with the use of AI in medicine. This study conducts a theoretical analysis of the current academic discourse on AI ethics in medical education, identifying potential gaps and limitations. The inherent interconnectivity and interdisciplinary nature of these anticipated challenges are illustrated through a focused discussion on "informed consent" in the context of AI in medicine and medical education. This paper proposes a principle-based approach to AI ethics education, building on the 4 principles of medical ethics-autonomy, beneficence, nonmaleficence, and justice-and extending them by integrating 3 public health ethics principles-efficiency, common good orientation, and proportionality. The principle-based approach to teaching AI ethics in medical education proposed in this study offers a foundational framework for addressing the anticipated ethical challenges of using AI in medicine, recommended in the current academic discourse. By incorporating the 3 principles of public health ethics, this principle-based approach ensures that medical ethics education remains relevant and responsive to the dynamic landscape of AI integration in medicine. As the advancement of AI technologies in medicine is expected to increase, medical ethics education must adapt and evolve accordingly. The proposed principle-based approach for teaching AI ethics in medical education provides an important foundation to ensure that future medical professionals are not only aware of the ethical dimensions of AI in medicine but also equipped to make informed ethical decisions in their practice. Future research is required to develop problem-based and competency-oriented learning objectives and educational content for the proposed principle-based approach to teaching AI ethics in medical education.

The Appropriateness of Medical Devices Is Strongly Influenced by Sex and Gender

Until now, research has been performed mainly in men, with a low recruitment of women; consequentially, biological, physiological, and physio-pathological mechanisms are less understood in women. Obviously, without data obtained on women, it is impossible to apply the results of research appropriately to women. This issue also applies to medical devices (MDs), and numerous problems linked to scarce pre-market research and clinical trials on MDs were evidenced after their introduction to the market. Globally, some MDs are less efficient in women than in men and sometimes MDs are less safe for women than men, although recently there has been a small but significant decrease in the sex and gender gap. As an example, cardiac resynchronization defibrillators seem to produce more beneficial effects in women than in men. It is also important to remember that MDs can impact the health of healthcare providers and this could occur in a sex- and gender-dependent manner. Recently, MDs' complexity is rising, and to ensure their appropriate use they must have a sex-gender-sensitive approach. Unfortunately, the majority of physicians, healthcare providers, and developers of MDs still believe that the human population is only constituted by men. Therefore, to overcome the gender gap, a real collaboration between the inventors of MDs, health researchers, and health providers should be established to test MDs in female and male tissues, animals, and women.

What Goes In, Must Come Out: Generative Artificial Intelligence Does Not Present Algorithmic Bias Across Race and Gender in Medical Residency Specialties

Objective Artificial Intelligence (AI) has made significant inroads into various domains, including medicine, raising concerns about algorithmic bias. This study investigates the presence of biases in generative AI programs, with a specific focus on gender and racial representations across 19 medical residency specialties. Methodology This comparative study utilized DALL-E2 to generate faces representing 19 distinct residency training specialties, as identified by the Association of American Medical Colleges (AAMC), which were then compared to the AAMC's residency specialty breakdown with respect to race and gender. Results Our findings reveal an alignment between OpenAI's DALL-E2's predictions and the current demographic landscape of medical residents, suggesting an absence of algorithmic bias in this AI model. Conclusion This revelation gives rise to important ethical considerations. While AI excels at pattern recognition, it inherits and mirrors the biases present in its training data. To combat AI bias, addressing real-world disparities is imperative. Initiatives to promote inclusivity and diversity within medicine are commendable and contribute to reshaping medical education. This study underscores the need for ongoing efforts to dismantle barriers and foster inclusivity in historically male-dominated medical fields, particularly for underrepresented populations. Ultimately, our findings underscore the crucial role of real-world data quality in mitigating AI bias. As AI continues to shape healthcare and education, the pursuit of equitable, unbiased AI applications should remain at the forefront of these transformative endeavors.

Identifying and handling data bias within primary healthcare data using synthetic data generators

Advanced synthetic data generators can simulate data samples that closely resemble sensitive personal datasets while significantly reducing the risk of individual identification. The use of these advanced generators holds enormous potential in the medical field, as it allows for the simulation and sharing of sensitive patient data. This enables the development and rigorous validation of novel AI technologies for accurate diagnosis and efficient disease management. Despite the availability of massive ground truth datasets (such as UK-NHS databases that contain millions of patient records), the risk of biases being carried over to data generators still exists. These biases may arise from the under-representation of specific patient cohorts due to cultural sensitivities within certain communities or standardised data collection procedures. Machine learning models can exhibit bias in various forms, including the under-representation of certain groups in the data. This can lead to missing data and inaccurate correlations and distributions, which may also be reflected in synthetic data. Our paper aims to improve synthetic data generators by introducing probabilistic approaches to first detect difficult-to-predict data samples in ground truth data and then boost them when applying the generator. In addition, we explore strategies to generate synthetic data that can reduce bias and, at the same time, improve the performance of predictive models.

From Machine Learning to Patient Outcomes: A Comprehensive Review of AI in Pancreatic Cancer

Pancreatic cancer is a highly aggressive and difficult-to-detect cancer with a poor prognosis. Late diagnosis is common due to a lack of early symptoms, specific markers, and the challenging location of the pancreas. Imaging technologies have improved diagnosis, but there is still room for improvement in standardizing guidelines. Biopsies and histopathological analysis are challenging due to tumor heterogeneity. Artificial Intelligence (AI) revolutionizes healthcare by improving diagnosis, treatment, and patient care. AI algorithms can analyze medical images with precision, aiding in early disease detection. AI also plays a role in personalized medicine by analyzing patient data to tailor treatment plans. It streamlines administrative tasks, such as medical coding and documentation, and provides patient assistance through AI chatbots. However, challenges include data privacy, security, and ethical considerations. This review article focuses on the potential of AI in transforming pancreatic cancer care, offering improved diagnostics, personalized treatments, and operational efficiency, leading to better patient outcomes.

Role of Ethics in Developing AI-Based Applications in Medicine: Insights From Expert Interviews and Discussion of Implications

BACKGROUND

The integration of artificial intelligence (AI)-based applications in the medical field has increased significantly, offering potential improvements in patient care and diagnostics. However, alongside these advancements, there is growing concern about ethical considerations, such as bias, informed consent, and trust in the development of these technologies.

OBJECTIVE

This study aims to assess the role of ethics in the development of AI-based applications in medicine. Furthermore, this study focuses on the potential consequences of neglecting ethical considerations in AI development, particularly their impact on patients and physicians.

METHODS

Qualitative content analysis was used to analyze the responses from expert interviews. Experts were selected based on their involvement in the research or practical development of AI-based applications in medicine for at least 5 years, leading to the inclusion of 7 experts in the study.

RESULTS

The analysis revealed 3 main categories and 7 subcategories reflecting a wide range of views on the role of ethics in AI development. This variance underscores the subjectivity and complexity of integrating ethics into the development of AI in medicine. Although some experts view ethics as fundamental, others prioritize performance and efficiency, with some perceiving ethics as potential obstacles to technological progress. This dichotomy of perspectives clearly emphasizes the subjectivity and complexity surrounding the role of ethics in AI development, reflecting the inherent multifaceted nature of this issue.

CONCLUSIONS

Despite the methodological limitations impacting the generalizability of the results, this study underscores the critical importance of consistent and integrated ethical considerations in AI development for medical applications. It advocates further research into effective strategies for ethical AI development, emphasizing the need for transparent and responsible practices, consideration of diverse data sources, physician training, and the establishment of comprehensive ethical and legal frameworks.

Application of Artificial intelligence in COVID-19-related geriatric care: A scoping review

BACKGROUND

Older adults have been disproportionately affected by the COVID-19 pandemic. This scoping review aimed to summarize the current evidence of artificial intelligence (AI) use in the screening/monitoring, diagnosis, and/or treatment of COVID-19 among older adults.

METHOD

The review followed the Joanna Briggs Institute and Arksey and O'Malley frameworks. An information specialist performed a comprehensive search from the date of inception until May 2021, in six bibliographic databases. The selected studies considered all populations, and all AI interventions that had been used in COVID-19-related geriatric care. We focused on patient, healthcare provider, and healthcare system-related outcomes. The studies were restricted to peer-reviewed English publications. Two authors independently screened the titles and abstracts of the identified records, read the selected full texts, and extracted data from the included studies using a validated data extraction form. Disagreements were resolved by consensus, and if this was not possible, the opinion of a third reviewer was sought.

RESULTS

Six databases were searched , yielding 3,228 articles, of which 10 were included. The majority of articles used a single AI model to assess the association between patients' comorbidities and COVID-19 outcomes. Articles were mainly conducted in high-income countries, with limited representation of females in study participants, and insufficient reporting of participants' race and ethnicity.

DISCUSSION

This review highlighted how the COVID-19 pandemic has accelerated the application of AI to protect older populations, with most interventions in the pilot testing stage. Further work is required to measure effectiveness of these technologies in a larger scale, use more representative datasets for training of AI models, and expand AI applications to low-income countries.

Economic Evaluations and Equity in the Use of Artificial Intelligence in Imaging Exams for Medical Diagnosis in People With Skin, Neurological, and Pulmonary Diseases: Protocol for a Systematic Review

BACKGROUND

Traditional health care systems face long-standing challenges, including patient diversity, geographical disparities, and financial constraints. The emergence of artificial intelligence (AI) in health care offers solutions to these challenges. AI, a multidisciplinary field, enhances clinical decision-making. However, imbalanced AI models may enhance health disparities.

OBJECTIVE

This systematic review aims to investigate the economic performance and equity impact of AI in diagnostic imaging for skin, neurological, and pulmonary diseases. The research question is "To what extent does the use of AI in imaging exams for diagnosing skin, neurological, and pulmonary diseases result in improved economic outcomes, and does it promote equity in health care systems?"

METHODS

The study is a systematic review of economic and equity evaluations following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) guidelines. Eligibility criteria include articles reporting on economic evaluations or equity considerations related to AI-based diagnostic imaging for specified diseases. Data will be collected from PubMed, Embase, Scopus, Web of Science, and reference lists. Data quality and transferability will be assessed according to CHEC (Consensus on Health Economic Criteria), EPHPP (Effective Public Health Practice Project), and Welte checklists.

RESULTS

This systematic review began in March 2023. The literature search identified 9,526 publications and, after full-text screening, 9 publications were included in the study. We plan to submit a manuscript to a peer-reviewed journal once it is finalized, with an expected completion date in January 2024.

CONCLUSIONS

AI in diagnostic imaging offers potential benefits but also raises concerns about equity and economic impact. Bias in algorithms and disparities in access may hinder equitable outcomes. Evaluating the economic viability of AI applications is essential for resource allocation and affordability. Policy makers and health care stakeholders can benefit from this review's insights to make informed decisions. Limitations, including study variability and publication bias, will be considered in the analysis. This systematic review will provide valuable insights into the economic and equity implications of AI in diagnostic imaging. It aims to inform evidence-based decision-making and contribute to more efficient and equitable health care systems.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48544.

The Importance of Gender-Sensitive Health Care in the Context of Pain, Emergency and Vaccination: A Narrative Review

So far, health care has been insufficiently organized in a gender-sensitive way, which makes the promotion of care that meets the needs of women and men equally emerge as a relevant public health problem. The aim of this narrative review was to outline the need for more gender-sensitive medical care in the context of pain, emergency care and vaccinations. In this narrative review, a selective search was performed in Pubmed, and the databases of the World Health Organization (WHO), the European Institute for Gender Equality and the German Federal Ministry of Health were searched. Study data indicate that there are differences between men and women with regard to the ability to bear pain. On the other hand, socially constructed role expectations in pain and the communication of these are also relevant. Studies indicate that women receive adequate pain medication less often than men with a comparable pain score. Furthermore, study results indicate that the female gender is associated with an increased risk of inadequate emergency care. In terms of vaccine provision, women are less likely than men to utilize or gain access to vaccination services, and there are gender-sensitive differences in vaccine efficacy and safety. Sensitization in teaching, research and care is needed to mitigate gender-specific health inequalities.

Developing technology-based interventions for infectious diseases: ethical considerations for young sexual and gender minority people

Compared to their heterosexual and cisgender peers, young sexual and gender minority (YSGM) people are more likely to contract sexually transmitted infections (STIs; e.g., HIV) and to face adverse consequences of emerging infections, such as COVID-19 and mpox. To reduce these sexual health disparities, technology-based interventions (TBIs) for STIs and emerging infections among YSGM adolescents and young adults have been developed. In this Perspective, we discuss ethical issues, ethical principles, and recommendations in the development and implementation of TBIs to address STIs and emerging infections among YSGM. Our discussion covers: (1) confidentiality, privacy, and data security (e.g., if TBI use is revealed, YSGM are at increased risk of discrimination and family rejection); (2) empowerment and autonomy (e.g., designing TBIs that can still function if YSGM users opt-out of multiple features and data collection requests); (3) evidence-based and quality controlled (e.g., going above and beyond minimum FDA effectiveness standards to protect vulnerable YSGM people); (4) cultural sensitivity and tailoring (e.g., using YSGM-specific models of prevention and intervention); (5) balancing inclusivity vs. group specificity (e.g., honoring YSGM heterogeneity); (6) duty to care (e.g., providing avenues to contact affirming healthcare professionals); (7) equitable access (e.g., prioritizing YSGM people living in low-resource, high-stigma areas); and (8) digital temperance (e.g., being careful with gamification because YSGM experience substantial screen time compared to their peers). We conclude that a community-engaged, YSGM-centered approach to TBI development and implementation is paramount to ethically preventing and treating STIs and emerging infections with innovative technology.

The emerging role of digital health in the management of asthma

The most common reasons seen for lack of asthma control include misconceptions about disease control, low controller treatment adherence, poor inhaler technique, and the resulting underuse of controllers and overuse of short-acting beta2 agonists (SABAs). Narrowing these care gaps may be achieved through well-designed patient education that considers the patient's motivation, beliefs, and capabilities regarding their asthma and its management and empowers the patient to become an active participant in treatment decisions. Digital health technologies (DHTs) and digital therapeutic (DT) devices provide new opportunities to monitor treatment behaviors, improve communication between healthcare providers and patients, and generate data that inform educational interactions. DHT and DT have been proven effective in enhancing patient self-management in other chronic conditions, particularly diabetes. Accelerated integration of DHT and DT into the management of asthma patients is facilitated by the use of digital inhalers that employ sensor technology ("smart" inhalers). These devices efficiently provide real-time feedback on controller adherence, SABA use, and inhaler technique that have the strong potential to optimize asthma control.

[The transformative effect of artificial intelligence in hospitals : The focus is on the individual]

Rapid advances in digital technology and the promising potential of artificial intelligence (AI) are changing our everyday lives and have already impacted on hospital procedures. The use of AI applications, in particular, enables a wide range of possible uses and has considerable potential for improving medical and nursing care. In radiological diagnostics, for example, there are already many well-researched applications for AI-based image evaluation. In this article further AI developments are presented, which can help to relieve medical staff in order to create more time for direct patient care. In addition, essential aspects regarding the development and transfer of AI-based applications are highlighted. It is crucial that the integration of AI into medical practice is carried out with the utmost care and prudence. Data protection and ethical aspects need to be considered and respected at all times. Ensuring the reliability and integrity of AI systems is essential to earn the trust of both patients and healthcare professionals. A comprehensive inspection for possible bias within the underlying data and algorithms is indispensable. In this field of tension between promising possibilities and ethical challenges, the digital transformation in medicine and care can be designed to increase patient safety and to relieve staff.

Teaching AI Ethics in Medical Education: A Scoping Review of Current Literature and Practices

Introduction

The increasing use of Artificial Intelligence (AI) in medicine has raised ethical concerns, such as patient autonomy, bias, and transparency. Recent studies suggest a need for teaching AI ethics as part of medical curricula. This scoping review aimed to represent and synthesize the literature on teaching AI ethics as part of medical education.

Methods

The PRISMA-SCR guidelines and JBI methodology guided a literature search in four databases (PubMed, Embase, Scopus, and Web of Science) for the past 22 years (2000-2022). To account for the release of AI-based chat applications, such as ChatGPT, the literature search was updated to include publications until the end of June 2023.

Results

1384 publications were originally identified and, after screening titles and abstracts, the full text of 87 publications was assessed. Following the assessment of the full text, 10 publications were included for further analysis. The updated literature search identified two additional relevant publications from 2023 were identified and included in the analysis. All 12 publications recommended teaching AI ethics in medical curricula due to the potential implications of AI in medicine. Anticipated ethical challenges such as bias were identified as the recommended basis for teaching content in addition to basic principles of medical ethics. Case-based teaching using real-world examples in interactive seminars and small groups was recommended as a teaching modality.

Conclusion

This scoping review reveals a scarcity of literature on teaching AI ethics in medical education, with most of the available literature being recent and theoretical. These findings emphasize the importance of more empirical studies and foundational definitions of AI ethics to guide the development of teaching content and modalities. Recognizing AI's significant impact of AI on medicine, additional research on the teaching of AI ethics in medical education is needed to best prepare medical students for future ethical challenges.

A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement

Virtual Mental Health Assistants (VMHAs) continuously evolve to support the overloaded global healthcare system, which receives approximately 60 million primary care visits and 6 million emergency room visits annually. These systems, developed by clinical psychologists, psychiatrists, and AI researchers, are designed to aid in Cognitive Behavioral Therapy (CBT). The main focus of VMHAs is to provide relevant information to mental health professionals (MHPs) and engage in meaningful conversations to support individuals with mental health conditions. However, certain gaps prevent VMHAs from fully delivering on their promise during active communications. One of the gaps is their inability to explain their decisions to patients and MHPs, making conversations less trustworthy. Additionally, VMHAs can be vulnerable in providing unsafe responses to patient queries, further undermining their reliability. In this review, we assess the current state of VMHAs on the grounds of user-level explainability and safety, a set of desired properties for the broader adoption of VMHAs. This includes the examination of ChatGPT, a conversation agent developed on AI-driven models: GPT3.5 and GPT-4, that has been proposed for use in providing mental health services. By harnessing the collaborative and impactful contributions of AI, natural language processing, and the mental health professionals (MHPs) community, the review identifies opportunities for technological progress in VMHAs to ensure their capabilities include explainable and safe behaviors. It also emphasizes the importance of measures to guarantee that these advancements align with the promise of fostering trustworthy conversations.

Promoting Equity In Clinical Decision Making: Dismantling Race-Based Medicine

As the use of artificial intelligence has spread rapidly throughout the US health care system, concerns have been raised about racial and ethnic biases built into the algorithms that often guide clinical decision making. Race-based medicine, which relies on algorithms that use race as a proxy for biological differences, has led to treatment patterns that are inappropriate, unjust, and harmful to minoritized racial and ethnic groups. These patterns have contributed to persistent disparities in health and health care. To reduce these disparities, we recommend a race-aware approach to clinical decision support that considers social and environmental factors such as structural racism and social determinants of health. Recent policy changes in medical specialty societies and innovations in algorithm development represent progress on the path to dismantling race-based medicine. Success will require continued commitment and sustained efforts among stakeholders in the health care, research, and technology sectors. Increasing the diversity of clinical trial populations, broadening the focus of precision medicine, improving education about the complex factors shaping health outcomes, and developing new guidelines and policies to enable culturally responsive care are important next steps.

Current state and prospects of artificial intelligence in allergy

The field of medicine is witnessing an exponential growth of interest in artificial intelligence (AI), which enables new research questions and the analysis of larger and new types of data. Nevertheless, applications that go beyond proof of concepts and deliver clinical value remain rare, especially in the field of allergy. This narrative review provides a fundamental understanding of the core concepts of AI and critically discusses its limitations and open challenges, such as data availability and bias, along with potential directions to surmount them. We provide a conceptual framework to structure AI applications within this field and discuss forefront case examples. Most of these applications of AI and machine learning in allergy concern supervised learning and unsupervised clustering, with a strong emphasis on diagnosis and subtyping. A perspective is shared on guidelines for good AI practice to guide readers in applying it effectively and safely, along with prospects of field advancement and initiatives to increase clinical impact. We anticipate that AI can further deepen our knowledge of disease mechanisms and contribute to precision medicine in allergy.

Data drift in medical machine learning: implications and potential remedies

Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.

Leveraging Digital Health to Improve the Cardiovascular Health of Women

Purpose of Review

In this review, we present a comprehensive discussion on the population-level implications of digital health interventions (DHIs) to improve cardiovascular health (CVH) through sex- and gender-specific prevention strategies among women.

Recent Findings

Over the past 30 years, there have been significant advancements in the diagnosis and treatment of cardiovascular diseases, a leading cause of morbidity and mortality among men and women worldwide. However, women are often underdiagnosed, undertreated, and underrepresented in cardiovascular clinical trials, which all contribute to disparities within this population. One approach to address this is through DHIs, particularly among racial and ethnic minoritized groups. Implementation of telemedicine has shown promise in increasing adherence to healthcare visits, improving BP monitoring, weight control, physical activity, and the adoption of healthy behaviors. Furthermore, the use of mobile health applications facilitated by smart devices, wearables, and other eHealth (defined as electronically delivered health services) modalities has also promoted CVH among women in general, as well as during pregnancy and the postpartum period. Overall, utilizing a digital health approach for healthcare delivery, decentralized clinical trials, and incorporation into daily lifestyle activities has the potential to improve CVH among women by mitigating geographical, structural, and financial barriers to care.

Summary

Leveraging digital technologies and strategies introduces novel methods to address sex- and gender-specific health and healthcare disparities and improve the quality of care provided to women. However, it is imperative to be mindful of the digital divide in specific populations, which may hinder accessibility to these novel technologies and inadvertently widen preexisting inequities.

A translational perspective towards clinical AI fairness

Artificial intelligence (AI) has demonstrated the ability to extract insights from data, but the fairness of such data-driven insights remains a concern in high-stakes fields. Despite extensive developments, issues of AI fairness in clinical contexts have not been adequately addressed. A fair model is normally expected to perform equally across subgroups defined by sensitive variables (e.g., age, gender/sex, race/ethnicity, socio-economic status, etc.). Various fairness measurements have been developed to detect differences between subgroups as evidence of bias, and bias mitigation methods are designed to reduce the differences detected. This perspective of fairness, however, is misaligned with some key considerations in clinical contexts. The set of sensitive variables used in healthcare applications must be carefully examined for relevance and justified by clear clinical motivations. In addition, clinical AI fairness should closely investigate the ethical implications of fairness measurements (e.g., potential conflicts between group- and individual-level fairness) to select suitable and objective metrics. Generally defining AI fairness as "equality" is not necessarily reasonable in clinical settings, as differences may have clinical justifications and do not indicate biases. Instead, "equity" would be an appropriate objective of clinical AI fairness. Moreover, clinical feedback is essential to developing fair and well-performing AI models, and efforts should be made to actively involve clinicians in the process. The adaptation of AI fairness towards healthcare is not self-evident due to misalignments between technical developments and clinical considerations. Multidisciplinary collaboration between AI researchers, clinicians, and ethicists is necessary to bridge the gap and translate AI fairness into real-life benefits.

Raising awareness of sex and gender bias in artificial intelligence and health

Historically, biomedical research has been led by and focused on men. The recent introduction of Artificial Intelligence (AI) in this area has further proven this practice to be discriminatory for other sexes and genders, more noticeably for women. To move towards a fair AI development, it is essential to include sex and gender diversity both in research practices and in the workplace. In this context, the Bioinfo4women (B4W) program of the Barcelona Supercomputing Center (i) promotes the participation of women scientists by improving their visibility, (ii) fosters international collaborations between institutions and programs and (iii) advances research on sex and gender bias in AI and health. In this article, we discuss methodology and results of a series of conferences, titled “Sex and Gender Bias in Artificial Intelligence and Health, organized by B4W and La Caixa Foundation from March to June 2021 in Barcelona, Spain. The series consisted of nine hybrid events, composed of keynote sessions and seminars open to the general audience, and two working groups with invited experts from different professional backgrounds (academic fields such as biology, engineering, and sociology, as well as NGOs, journalists, lawyers, policymakers, industry). Based on this awareness-raising action, we distilled key recommendations to facilitate the inclusion of sex and gender perspective into public policies, educational programs, industry, and biomedical research, among other sectors, and help overcome sex and gender biases in AI and health.

A machine-learning based bio-psycho-social model for the prediction of non-obstructive and obstructive coronary artery disease

BACKGROUND

Mechanisms of myocardial ischemia in obstructive and non-obstructive coronary artery disease (CAD), and the interplay between clinical, functional, biological and psycho-social features, are still far to be fully elucidated.

OBJECTIVES

To develop a machine-learning (ML) model for the supervised prediction of obstructive versus non-obstructive CAD.

METHODS

From the EVA study, we analysed adults hospitalized for IHD undergoing conventional coronary angiography (CCA). Non-obstructive CAD was defined by a stenosis < 50% in one or more vessels. Baseline clinical and psycho-socio-cultural characteristics were used for computing a Rockwood and Mitnitski frailty index, and a gender score according to GENESIS-PRAXY methodology. Serum concentration of inflammatory cytokines was measured with a multiplex flow cytometry assay. Through an XGBoost classifier combined with an explainable artificial intelligence tool (SHAP), we identified the most influential features in discriminating obstructive versus non-obstructive CAD.

RESULTS

Among the overall EVA cohort (n = 509), 311 individuals (mean age 67 ± 11 years, 38% females; 67% obstructive CAD) with complete data were analysed. The ML-based model (83% accuracy and 87% precision) showed that while obstructive CAD was associated with higher frailty index, older age and a cytokine signature characterized by IL-1β, IL-12p70 and IL-33, non-obstructive CAD was associated with a higher gender score (i.e., social characteristics traditionally ascribed to women) and with a cytokine signature characterized by IL-18, IL-8, IL-23.

CONCLUSIONS

Integrating clinical, biological, and psycho-social features, we have optimized a sex- and gender-unbiased model that discriminates obstructive and non-obstructive CAD. Further mechanistic studies will shed light on the biological plausibility of these associations.

CLINICAL TRIAL REGISTRATION

NCT02737982.

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders.

Understanding Biases and Disparities in Radiology AI Datasets: A Review

Artificial intelligence (AI) continues to show great potential in disease detection and diagnosis on medical imaging with increasingly high accuracy. An important component of AI model creation is dataset development for training, validation, and testing. Diverse and high-quality datasets are critical to ensure robust and unbiased AI models that maintain validity, especially in traditionally underserved populations globally. Yet publicly available datasets demonstrate problems with quality and inclusivity. In this literature review, the authors evaluate publicly available medical imaging datasets for demographic, geographic, genetic, and disease representation or lack thereof and call for an increase emphasis on dataset development to maximize the impact of AI models.

Building Diversity, Equity, and Inclusion Within Radiology Artificial Intelligence: Representation Matters, From Data to the Workforce

Diversity, equity, and inclusion (DEI) is both a critical ingredient and moral imperative in shaping the future of radiology artificial intelligence (AI) for improved patient care, from design to deployment. At the design level: Potential biases and discrimination within data sets results in inaccurate radiology AI models, and there is an urgent need to purposefully embed DEI principles throughout the AI development and implementation process. At the deployment level: Diverse representation in radiology AI leadership, research, and career development is necessary to avoid worsening structural and historical health inequities. To create an inclusive and equitable AI-enabled future in healthcare, a DEI radiology AI leadership training program may be needed to cultivate a diverse and sustainable pipeline of leaders in the field.

Potential applications of artificial intelligence and machine learning on diagnosis, treatment, and outcome prediction to address health care disparities of chronic limb-threatening ischemia

Chronic limb-threatening ischemia (CLTI) is the most advanced form of peripheral artery disease. CLTI has an extremely poor prognosis and is associated with considerable risk of major amputation, cardiac morbidity, mortality, and poor quality of life. Early diagnosis and targeted treatment of CLTI is critical for improving patient's prognosis. However, this objective has proven elusive, time-consuming, and challenging due to existing health care disparities among patients. In this article, we reviewed how artificial intelligence (AI) and machine learning (ML) can be helpful to accurately diagnose, improve outcome prediction, and identify disparities in the treatment of CLTI. We demonstrate the importance of AI/ML approaches for management of these patients and how available data could be used for computer-guided interventions. Although AI/ML applications to mitigate health care disparities in CLTI are in their infancy, we also highlighted specific AI/ML methods that show potential for addressing health care disparities in CLTI.

Making a 'sex-difference fact': Ambien dosing at the interface of policy, regulation, women's health, and biology

The U.S. Food and Drug Administration's (FDA) 2013 decision to lower recommended Ambien dosing for women has been widely cited as a hallmark example of the importance of sex differences in biomedicine. Using regulatory documents, scientific publications, and media coverage, this article analyzes the making of this highly influential and mobile 'sex-difference fact'. As we show, the FDA's decision was a contingent outcome of the drug approval process. Attending to how a contested sex-difference fact came to anchor elite women's health advocacy, this article excavates the role of regulatory processes, advocacy groups, and the media in producing perceptions of scientific agreement while foreclosing ongoing debate, ultimately enabling the stabilization of a binary, biological sex-difference fact and the distancing of this fact from its conditions of construction.

A comparison of synthetic data generation and federated analysis for enabling international evaluations of cardiovascular health

Sharing health data for research purposes across international jurisdictions has been a challenge due to privacy concerns. Two privacy enhancing technologies that can enable such sharing are synthetic data generation (SDG) and federated analysis, but their relative strengths and weaknesses have not been evaluated thus far. In this study we compared SDG with federated analysis to enable such international comparative studies. The objective of the analysis was to assess country-level differences in the role of sex on cardiovascular health (CVH) using a pooled dataset of Canadian and Austrian individuals. The Canadian data was synthesized and sent to the Austrian team for analysis. The utility of the pooled (synthetic Canadian + real Austrian) dataset was evaluated by comparing the regression results from the two approaches. The privacy of the Canadian synthetic data was assessed using a membership disclosure test which showed an F1 score of 0.001, indicating low privacy risk. The outcome variable of interest was CVH, calculated through a modified CANHEART index. The main and interaction effect parameter estimates of the federated and pooled analyses were consistent and directionally the same. It took approximately one month to set up the synthetic data generation platform and generate the synthetic data, whereas it took over 1.5 years to set up the federated analysis system. Synthetic data generation can be an efficient and effective tool for enabling multi-jurisdictional studies while addressing privacy concerns.

An Automatically Adaptive Digital Health Intervention to Decrease Opioid-Related Risk While Conserving Counselor Time: Quantitative Analysis of Treatment Decisions Based on Artificial Intelligence and Patient-Reported Risk Measures

BACKGROUND

Some patients prescribed opioid analgesic (OA) medications for pain experience serious side effects, including dependence, sedation, and overdose. As most patients are at low risk for OA-related harms, risk reduction interventions requiring multiple counseling sessions are impractical on a large scale.

OBJECTIVE

This study evaluates whether an intervention based on reinforcement learning (RL), a field of artificial intelligence, learned through experience to personalize interactions with patients with pain discharged from the emergency department (ED) and decreased self-reported OA misuse behaviors while conserving counselors' time.

METHODS

We used data representing 2439 weekly interactions between a digital health intervention ("Prescription Opioid Wellness and Engagement Research in the ED" [PowerED]) and 228 patients with pain discharged from 2 EDs who reported recent opioid misuse. During each patient's 12 weeks of intervention, PowerED used RL to select from 3 treatment options: a brief motivational message delivered via an interactive voice response (IVR) call, a longer motivational IVR call, or a live call from a counselor. The algorithm selected session types for each patient each week, with the goal of minimizing OA risk, defined in terms of a dynamic score reflecting patient reports during IVR monitoring calls. When a live counseling call was predicted to have a similar impact on future risk as an IVR message, the algorithm favored IVR to conserve counselor time. We used logit models to estimate changes in the relative frequency of each session type as PowerED gained experience. Poisson regression was used to examine the changes in self-reported OA risk scores over calendar time, controlling for the ordinal session number (1st to 12th).

RESULTS

Participants on average were 40 (SD 12.7) years of age; 66.7% (152/228) were women and 51.3% (117/228) were unemployed. Most participants (175/228, 76.8%) reported chronic pain, and 46.2% (104/225) had moderate to severe depressive symptoms. As PowerED gained experience through interactions over a period of 142 weeks, it delivered fewer live counseling sessions than brief IVR sessions (P=.006) and extended IVR sessions (P<.001). Live counseling sessions were selected 33.5% of the time in the first 5 weeks of interactions (95% CI 27.4%-39.7%) but only for 16.4% of sessions (95% CI 12.7%-20%) after 125 weeks. Controlling for each patient's changes during the course of treatment, this adaptation of treatment-type allocation led to progressively greater improvements in self-reported OA risk scores (P<.001) over calendar time, as measured by the number of weeks since enrollment began. Improvement in risk behaviors over time was especially pronounced among patients with the highest risk at baseline (P=.02).

CONCLUSIONS

The RL-supported program learned which treatment modalities worked best to improve self-reported OA risk behaviors while conserving counselors' time. RL-supported interventions represent a scalable solution for patients with pain receiving OA prescriptions.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02990377; https://classic.clinicaltrials.gov/ct2/show/NCT02990377.

Achieving trust in health-behavior-change artificial intelligence apps (HBC-AIApp) development: a multi-perspective guide

OBJECTIVE

Trust determines the success of Health-Behavior-Change Artificial Intelligence Apps (HBC-AIApp). Developers of such apps need theory-based practical methods that can guide them in achieving such trust. Our study aimed to develop a comprehensive conceptual model and development process that can guide developers how to build HBC-AIApp in order to support trust creation among the app's users.

METHODS

We apply a multi-disciplinary approach where medical informatics, human-centered design, and holistic health methods are integrated to address the trust challenge in HBC-AIApps. The integration extends a conceptual model of trust in AI developed by Jermutus et al., whose properties guide the extension of the IDEAS (integrate, design, assess, and share) HBC-App development process.

RESULTS

The HBC-AIApp framework consists of three main blocks: (1) system development methods that study the users' complex reality, hence, their perceptions, needs, goals and environment; (2) mediators and other stakeholders who are important for developing and operating the HBC-AIApp, boundary objects that examine users' activities via the HBC-AIApp; and (3) the HBC-AIApp's structural components, AI logic, and physical implementation. These blocks come together to provide the extended conceptual model of trust in HBC-AIApps and the extended IDEAS process.

DISCUSSION

The developed HBC-AIApp framework drew from our own experience in developing trust in HBC-AIApp. Further research will focus on studying the application of the proposed comprehensive HBC-AIApp development framework and whether applying it supports trust creation in such apps.

A global perspective on data powering responsible AI solutions in health applications

Healthcare AI solutions have the potential to transform access, quality of care, and improve outcomes for patients globally. This review suggests consideration of a more global perspective, with a particular focus on marginalized communities, during the development of healthcare AI solutions. The review focuses on one aspect (medical applications) to allow technologists to build solutions in today's environment with an understanding of the challenges they face. The following sections explore and discuss the current challenges in the underlying data and AI technology design on healthcare solutions for global deployment. We highlight some of the factors that lead to gaps in data, gaps around regulations for the healthcare sector, and infrastructural challenges in power and network connectivity, as well as lack of social systems for healthcare and education, which pose challenges to the potential universal impacts of such technologies. We recommend using these considerations in developing prototype healthcare AI solutions to better capture the needs of a global population.

Solving the explainable AI conundrum by bridging clinicians' needs and developers' goals

Explainable artificial intelligence (XAI) has emerged as a promising solution for addressing the implementation challenges of AI/ML in healthcare. However, little is known about how developers and clinicians interpret XAI and what conflicting goals and requirements they may have. This paper presents the findings of a longitudinal multi-method study involving 112 developers and clinicians co-designing an XAI solution for a clinical decision support system. Our study identifies three key differences between developer and clinician mental models of XAI, including opposing goals (model interpretability vs. clinical plausibility), different sources of truth (data vs. patient), and the role of exploring new vs. exploiting old knowledge. Based on our findings, we propose design solutions that can help address the XAI conundrum in healthcare, including the use of causal inference models, personalized explanations, and ambidexterity between exploration and exploitation mindsets. Our study highlights the importance of considering the perspectives of both developers and clinicians in the design of XAI systems and provides practical recommendations for improving the effectiveness and usability of XAI in healthcare.

The ethical aspects of exposome research: a systematic review

In recent years, exposome research has been put forward as the next frontier for the study of human health and disease. Exposome research entails the analysis of the totality of environmental exposures and their corresponding biological responses within the human body. Increasingly, this is operationalized by big-data approaches to map the effects of internal as well as external exposures using smart sensors and multiomics technologies. However, the ethical implications of exposome research are still only rarely discussed in the literature. Therefore, we conducted a systematic review of the academic literature regarding both the exposome and underlying research fields and approaches, to map the ethical aspects that are relevant to exposome research. We identify five ethical themes that are prominent in ethics discussions: the goals of exposome research, its standards, its tools, how it relates to study participants, and the consequences of its products. Furthermore, we provide a number of general principles for how future ethics research can best make use of our comprehensive overview of the ethical aspects of exposome research. Lastly, we highlight three aspects of exposome research that are most in need of ethical reflection: the actionability of its findings, the epidemiological or clinical norms applicable to exposome research, and the meaning and action-implications of bias.

Gender medicine: effects of sex and gender on cardiovascular disease manifestation and outcomes

Despite a growing body of evidence, the distinct contributions of biological sex and the sociocultural dimension of gender to the manifestations and outcomes of ischaemic heart disease and heart failure remain unknown. The intertwining of sex-based differences in genetic and hormonal mechanisms with the complex dimension of gender and its different components and determinants that result in different disease phenotypes in women and men needs to be elucidated. The relative contribution of purely biological factors, such as genes and hormones, to cardiovascular phenotypes and outcomes is not yet fully understood. Increasing awareness of the effects of gender has led to efforts to measure gender in retrospective and prospective clinical studies and the development of gender scores. However, the synergistic or opposing effects of sex and gender on cardiovascular traits and on ischaemic heart disease and heart failure mechanisms have not yet been systematically described. Furthermore, specific considerations of sex-related and gender-related factors in gender dysphoria or in heart-brain interactions and their association with cardiovascular disease are still lacking. In this Review, we summarize contemporary evidence on the distinct effects of sex and gender as well as of their interactions on cardiovascular disease and how they favourably or unfavourably influence the pathogenesis, clinical manifestations and treatment responses in patients with ischaemic heart disease or heart failure.