Does "AI" stand for augmenting inequality in the era of covid-19 healthcare?

The risks of artificial intelligence: A narrative review and ethical reflection from an Oral Medicine group

As a relatively new tool, the use of artificial intelligence (AI) in medicine and dentistry has the potential to significantly transform the healthcare sector. AI has already demonstrated efficacy in medical diagnosis across several specialties, used successfully to detect breast, lung and skin cancer. In Oral Medicine, AI may be applied in a similar fashion, used in the detection and diagnosis of oral cancers and oral potentially malignant diseases. Despite its promise as a transformative diagnostic aid, the use of AI in healthcare presents significant safety, reliability and ethical concerns. There is no formal consensus on the safe and ethical implementation of AI systems in healthcare settings, but the literature converges on several key principles of ethical AI use including transparency, justice and fairness, non-maleficence, responsibility and privacy. This article provides a narrative review of the key ethical issues surrounding AI use in medicine, and reflects on these, providing view-points of a bioethicist and Oral Medicine clinicians from several units.

Towards equitable AI in oncology

Artificial intelligence (AI) stands at the threshold of revolutionizing clinical oncology, with considerable potential to improve early cancer detection and risk assessment, and to enable more accurate personalized treatment recommendations. However, a notable imbalance exists in the distribution of the benefits of AI, which disproportionately favour those living in specific geographical locations and in specific populations. In this Perspective, we discuss the need to foster the development of equitable AI tools that are both accurate in and accessible to a diverse range of patient populations, including those in low-income to middle-income countries. We also discuss some of the challenges and potential solutions in attaining equitable AI, including addressing the historically limited representation of diverse populations in existing clinical datasets and the use of inadequate clinical validation methods. Additionally, we focus on extant sources of inequity including the type of model approach (such as deep learning, and feature engineering-based methods), the implications of dataset curation strategies, the need for rigorous validation across a variety of populations and settings, and the risk of introducing contextual bias that comes with developing tools predominantly in high-income countries.

Navigating the Landscape of Digital Twins in Medicine: A Relational Bioethical Inquiry

This perspective article explores the use of digital twins (DTs) in medicine, highlighting its capacity to simulate risks and personalize treatments while examining the emerging bioethical concerns. Central concerns include power dynamics, exclusion, and misrepresentation. We propose adopting a relational bioethical approach that advocates for a comprehensive assessment of DTs in medicine, extending beyond individual interactions to consider broader structural relations and varying levels of access to power. This can be achieved through two key relational recommendations: acknowledging the impact of uneven relational structures on access to medical care and promoting social justice by evaluating resource allocation. While DTs in medicine offer promising advancements, a relational bioethical lens may provide a nuanced understanding, fostering equitable, inclusive and responsible integration of DTs into medical practice.

Large Language Models for Intraoperative Decision Support in Plastic Surgery: A Comparison between ChatGPT-4 and Gemini

Background and Objectives: Large language models (LLMs) are emerging as valuable tools in plastic surgery, potentially reducing surgeons' cognitive loads and improving patients' outcomes. This study aimed to assess and compare the current state of the two most common and readily available LLMs, Open AI's ChatGPT-4 and Google's Gemini Pro (1.0 Pro), in providing intraoperative decision support in plastic and reconstructive surgery procedures. Materials and Methods: We presented each LLM with 32 independent intraoperative scenarios spanning 5 procedures. We utilized a 5-point and a 3-point Likert scale for medical accuracy and relevance, respectively. We determined the readability of the responses using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) score. Additionally, we measured the models' response time. We compared the performance using the Mann-Whitney U test and Student's t-test. Results: ChatGPT-4 significantly outperformed Gemini in providing accurate (3.59 ± 0.84 vs. 3.13 ± 0.83, p-value = 0.022) and relevant (2.28 ± 0.77 vs. 1.88 ± 0.83, p-value = 0.032) responses. Alternatively, Gemini provided more concise and readable responses, with an average FKGL (12.80 ± 1.56) significantly lower than ChatGPT-4's (15.00 ± 1.89) (p < 0.0001). However, there was no difference in the FRE scores (p = 0.174). Moreover, Gemini's average response time was significantly faster (8.15 ± 1.42 s) than ChatGPT'-4's (13.70 ± 2.87 s) (p < 0.0001). Conclusions: Although ChatGPT-4 provided more accurate and relevant responses, both models demonstrated potential as intraoperative tools. Nevertheless, their performance inconsistency across the different procedures underscores the need for further training and optimization to ensure their reliability as intraoperative decision-support tools.

Artificial Intelligence in Postoperative Care: Assessing Large Language Models for Patient Recommendations in Plastic Surgery

Since their release, the medical community has been actively exploring large language models' (LLMs) capabilities, which show promise in providing accurate medical knowledge. One potential application is as a patient resource. This study analyzes and compares the ability of the currently available LLMs, ChatGPT-3.5, GPT-4, and Gemini, to provide postoperative care recommendations to plastic surgery patients. We presented each model with 32 questions addressing common patient concerns after surgical cosmetic procedures and evaluated the medical accuracy, readability, understandability, and actionability of the models' responses. The three LLMs provided equally accurate information, with GPT-3.5 averaging the highest on the Likert scale (LS) (4.18 ± 0.93) (p = 0.849), while Gemini provided significantly more readable (p = 0.001) and understandable responses (p = 0.014; p = 0.001). There was no difference in the actionability of the models' responses (p = 0.830). Although LLMs have shown their potential as adjunctive tools in postoperative patient care, further refinement and research are imperative to enable their evolution into comprehensive standalone resources.

Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population

Artificial Intelligence (AI) models for medical diagnosis often face challenges of generalizability and fairness. We highlighted the algorithmic unfairness in a large thyroid ultrasound dataset with significant diagnostic performance disparities across subgroups linked causally to sample size imbalances. To address this, we introduced the Quasi-Pareto Improvement (QPI) approach and a deep learning implementation (QP-Net) combining multi-task learning and domain adaptation to improve model performance among disadvantaged subgroups without compromising overall population performance. On the thyroid ultrasound dataset, our method significantly mitigated the area under curve (AUC) disparity for three less-prevalent subgroups by 0.213, 0.112, and 0.173 while maintaining the AUC for dominant subgroups; we also further confirmed the generalizability of our approach on two public datasets: the ISIC2019 skin disease dataset and the CheXpert chest radiograph dataset. Here we show the QPI approach to be widely applicable in promoting AI for equitable healthcare outcomes.

Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions

This perspective highlights the importance of addressing social determinants of health (SDOH) in patient health outcomes and health inequity, a global problem exacerbated by the COVID-19 pandemic. We provide a broad discussion on current developments in digital health and artificial intelligence (AI), including large language models (LLMs), as transformative tools in addressing SDOH factors, offering new capabilities for disease surveillance and patient care. Simultaneously, we bring attention to challenges, such as data standardization, infrastructure limitations, digital literacy, and algorithmic bias, that could hinder equitable access to AI benefits. For LLMs, we highlight potential unique challenges and risks including environmental impact, unfair labor practices, inadvertent disinformation or "hallucinations," proliferation of bias, and infringement of copyrights. We propose the need for a multitiered approach to digital inclusion as an SDOH and the development of ethical and responsible AI practice frameworks globally and provide suggestions on bridging the gap from development to implementation of equitable AI technologies.

Global health in the age of AI: Safeguarding humanity through collaboration and action

This opinion article discusses the impact of artificial intelligence (AI) on global health, addressing its potential risks and benefits to the field. It suggests that, given the existential risks of AI development, the global health community must contribute to AI-related advances, ensuring health equity and the wellbeing of vulnerable populations. Through transdisciplinary collaborations, robust AI governance, and an emphasis on equity, strategies are proposed to harness the potential of AI to reduce health inequalities and improve wellbeing at global and local levels.

Clearing the way for participatory data stewardship in artificial intelligence development: a mixed methods approach

Participatory data stewardship (PDS) empowers individuals to shape and govern their data via responsible collection and use. As artificial intelligence (AI) requires massive amounts of data, research must assess what factors predict consumers' willingness to provide their data to AI. This mixed-methods study applied the extended Technology Acceptance Model (TAM) with additional predictors of trust and subjective norms. Participants' data donation profile was also measured to assess the influence of individuals' social duty, understanding of the purpose and guilt. Participants (N = 322) completed an experimental survey. Individuals were willing to provide data to AI via PDS when they believed it was their social duty, understood the purpose and trusted AI. However, the TAM may not be a complete model for assessing user willingness. This study establishes that individuals value the importance of trusting and comprehending the broader societal impact of AI when providing their data to AI.Practitioner summary: To build responsible and representative AI, individuals are needed to participate in data stewardship. The factors driving willingness to participate in such methods were studied via an online survey. Trust, social duty and understanding the purpose significantly predicted willingness to provide data to AI via participatory data stewardship.

Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques

Introduction

Machine-assisted topic analysis (MATA) uses artificial intelligence methods to help qualitative researchers analyze large datasets. This is useful for researchers to rapidly update healthcare interventions during changing healthcare contexts, such as a pandemic. We examined the potential to support healthcare interventions by comparing MATA with "human-only" thematic analysis techniques on the same dataset (1,472 user responses from a COVID-19 behavioral intervention).

Methods

In MATA, an unsupervised topic-modeling approach identified latent topics in the text, from which researchers identified broad themes. In human-only codebook analysis, researchers developed an initial codebook based on previous research that was applied to the dataset by the team, who met regularly to discuss and refine the codes. Formal triangulation using a "convergence coding matrix" compared findings between methods, categorizing them as "agreement", "complementary", "dissonant", or "silent".

Results

Human analysis took much longer than MATA (147.5 vs. 40 h). Both methods identified key themes about what users found helpful and unhelpful. Formal triangulation showed both sets of findings were highly similar. The formal triangulation showed high similarity between the findings. All MATA codes were classified as in agreement or complementary to the human themes. When findings differed slightly, this was due to human researcher interpretations or nuance from human-only analysis.

Discussion

Results produced by MATA were similar to human-only thematic analysis, with substantial time savings. For simple analyses that do not require an in-depth or subtle understanding of the data, MATA is a useful tool that can support qualitative researchers to interpret and analyze large datasets quickly. This approach can support intervention development and implementation, such as enabling rapid optimization during public health emergencies.

Exploring patient perspectives on how they can and should be engaged in the development of artificial intelligence (AI) applications in health care

BACKGROUND

Artificial intelligence (AI) is a rapidly evolving field which will have implications on both individual patient care and the health care system. There are many benefits to the integration of AI into health care, such as predicting acute conditions and enhancing diagnostic capabilities. Despite these benefits potential harms include algorithmic bias, inadequate consent processes, and implications on the patient-provider relationship. One tool to address patients' needs and prevent the negative implications of AI is through patient engagement. As it currently stands, patients have infrequently been involved in AI application development for patient care delivery. Furthermore, we are unaware of any frameworks or recommendations specifically addressing patient engagement within the field of AI in health care.

METHODS

We conducted four virtual focus groups with thirty patient participants to understand of how patients can and should be meaningfully engaged within the field of AI development in health care. Participants completed an educational module on the fundamentals of AI prior to participating in this study. Focus groups were analyzed using qualitative content analysis.

RESULTS

We found that participants in our study wanted to be engaged at the problem-identification stages using multiple methods such as surveys and interviews. Participants preferred that recruitment methodologies for patient engagement included both in-person and social media-based approaches with an emphasis on varying language modalities of recruitment to reflect diverse demographics. Patients prioritized the inclusion of underrepresented participant populations, longitudinal relationship building, accessibility, and interdisciplinary involvement of other stakeholders in AI development. We found that AI education is a critical step to enable meaningful patient engagement within this field. We have curated recommendations into a framework for the field to learn from and implement in future development.

CONCLUSION

Given the novelty and speed at which AI innovation is progressing in health care, patient engagement should be the gold standard for application development. Our proposed recommendations seek to enable patient-centered AI application development in health care. Future research must be conducted to evaluate the effectiveness of patient engagement in AI application development to ensure that both AI application development and patient engagement are done rigorously, efficiently, and meaningfully.

Mapping and evaluating national data flows: transparency, privacy, and guiding infrastructural transformation

The importance of big health data is recognised worldwide. Most UK National Health Service (NHS) care interactions are recorded in electronic health records, resulting in an unmatched potential for population-level datasets. However, policy reviews have highlighted challenges from a complex data-sharing landscape relating to transparency, privacy, and analysis capabilities. In response, we used public information sources to map all electronic patient data flows across England, from providers to more than 460 subsequent academic, commercial, and public data consumers. Although NHS data support a global research ecosystem, we found that multistage data flow chains limit transparency and risk public trust, most data interactions do not fulfil recommended best practices for safe data access, and existing infrastructure produces aggregation of duplicate data assets, thus limiting diversity of data and added value to end users. We provide recommendations to support data infrastructure transformation and have produced a website (https://DataInsights.uk) to promote transparency and showcase NHS data assets.

Considerations for addressing bias in artificial intelligence for health equity

Health equity is a primary goal of healthcare stakeholders: patients and their advocacy groups, clinicians, other providers and their professional societies, bioethicists, payors and value based care organizations, regulatory agencies, legislators, and creators of artificial intelligence/machine learning (AI/ML)-enabled medical devices. Lack of equitable access to diagnosis and treatment may be improved through new digital health technologies, especially AI/ML, but these may also exacerbate disparities, depending on how bias is addressed. We propose an expanded Total Product Lifecycle (TPLC) framework for healthcare AI/ML, describing the sources and impacts of undesirable bias in AI/ML systems in each phase, how these can be analyzed using appropriate metrics, and how they can be potentially mitigated. The goal of these "Considerations" is to educate stakeholders on how potential AI/ML bias may impact healthcare outcomes and how to identify and mitigate inequities; to initiate a discussion between stakeholders on these issues, in order to ensure health equity along the expanded AI/ML TPLC framework, and ultimately, better health outcomes for all.

Building Diversity, Equity, and Inclusion Within Radiology Artificial Intelligence: Representation Matters, From Data to the Workforce

Diversity, equity, and inclusion (DEI) is both a critical ingredient and moral imperative in shaping the future of radiology artificial intelligence (AI) for improved patient care, from design to deployment. At the design level: Potential biases and discrimination within data sets results in inaccurate radiology AI models, and there is an urgent need to purposefully embed DEI principles throughout the AI development and implementation process. At the deployment level: Diverse representation in radiology AI leadership, research, and career development is necessary to avoid worsening structural and historical health inequities. To create an inclusive and equitable AI-enabled future in healthcare, a DEI radiology AI leadership training program may be needed to cultivate a diverse and sustainable pipeline of leaders in the field.

Detecting shortcut learning for fair medical AI using shortcut testing

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models-their tendency to perform differently across subgroups of the population-and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Diagnosing this phenomenon is difficult as sensitive attributes may be causally linked with disease. Using multitask learning, we propose a method to directly test for the presence of shortcut learning in clinical ML systems and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI.

The Quality and Utility of Artificial Intelligence in Patient Care

BACKGROUND

Artificial intelligence (AI) is increasingly being used in patient care. In the future, physicians will need to understand not only the basic functioning of AI applications, but also their quality, utility, and risks.

METHODS

This article is based on a selective review of the literature on the principles, quality, limitations, and benefits AI applications in patient care, along with examples of individual applications.

RESULTS

The number of AI applications in patient care is rising, with more than 500 approvals in the United States to date. Their quality and utility are based on a number of interdependent factors, including the real-life setting, the type and amount of data collected, the choice of variables used by the application, the algorithms used, and the goal and implementation of each application. Bias (which may be hidden) and errors can arise at all these levels. Any evaluation of the quality and utility of an AI application must, therefore, be conducted according to the scientific principles of evidence-based medicine-a requirement that is often hampered by a lack of transparency.

CONCLUSION

AI has the potential to improve patient care while meeting the challenge of dealing with an ever-increasing surfeit of information and data in medicine with limited human resources. The limitations and risks of AI applications require critical and responsible consideration. This can best be achieved through a combination of scientific.

Data for sale: trust, confidence and sharing health data with commercial companies

Powered by 'big health data' and enormous gains in computing power, artificial intelligence and related technologies are already changing the healthcare landscape. Harnessing the potential of these technologies will necessitate partnerships between health institutions and commercial companies, particularly as it relates to sharing health data. The need for commercial companies to be trustworthy users of data has been argued to be critical to the success of this endeavour. I argue that this approach is mistaken. Our interactions with commercial companies need not, and should not, be based on trust. Rather, they should be based on confidence. I begin by elucidating the differences between trust, reliability, and confidence, and argue that trust is not the appropriate attitude to adopt when it comes to sharing data with commercial companies. I argue that what we really should want is confidence in a system of data sharing. I then provide an outline of what a confidence-worthy system of data sharing with commercial companies might look like, and conclude with some remarks about the role of trust within this system.

Artificial Intelligence-Based Opportunities in Liver Pathology-A Systematic Review

BACKGROUND

Artificial Intelligence (AI)-based Deep Neural Networks (DNNs) can handle a wide range of applications in image analysis, ranging from automated segmentation to diagnostic and prediction. As such, they have revolutionized healthcare, including in the liver pathology field.

OBJECTIVE

The present study aims to provide a systematic review of applications and performances provided by DNN algorithms in liver pathology throughout the Pubmed and Embase databases up to December 2022, for tumoral, metabolic and inflammatory fields.

RESULTS

42 articles were selected and fully reviewed. Each article was evaluated through the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, highlighting their risks of bias.

CONCLUSIONS

DNN-based models are well represented in the field of liver pathology, and their applications are diverse. Most studies, however, presented at least one domain with a high risk of bias according to the QUADAS-2 tool. Hence, DNN models in liver pathology present future opportunities and persistent limitations. To our knowledge, this review is the first one solely focused on DNN-based applications in liver pathology, and to evaluate their bias through the lens of the QUADAS2 tool.

The Aspects of Running Artificial Intelligence in Emergency Care; a Scoping Review

Introduction

Artificial Inteligence (AI) application in emergency medicine is subject to ethical and legal inconsistencies. The purposes of this study were to map the extent of AI applications in emergency medicine, to identify ethical issues related to the use of AI, and to propose an ethical framework for its use.

Methods

A comprehensive literature collection was compiled through electronic databases/internet search engines (PubMed, Web of Science Platform, MEDLINE, Scopus, Google Scholar/Academia, and ERIC) and reference lists. We considered studies published between 1 January 2014 and 6 October 2022. Articles that did not self-classify as studies of an AI intervention, those that were not relevant to Emergency Departments (EDs), and articles that did not report outcomes or evaluations were excluded. Descriptive and thematic analyses of data extracted from the included articles were conducted.

Results

A total of 137 out of the 2175 citations in the original database were eligible for full-text evaluation. Of these articles, 47 were included in the scoping review and considered for theme extraction. This review covers seven main areas of AI techniques in emergency medicine: Machine Learning (ML) Algorithms (10.64%), prehospital emergency management (12.76%), triage, patient acuity and disposition of patients (19.15%), disease and condition prediction (23.40%), emergency department management (17.03%), the future impact of AI on Emergency Medical Services (EMS) (8.51%), and ethical issues (8.51%).

Conclusion

There has been a rapid increase in AI research in emergency medicine in recent years. Several studies have demonstrated the potential of AI in diverse contexts, particularly when improving patient outcomes through predictive modelling. According to the synthesis of studies in our review, AI-based decision-making lacks transparency. This feature makes AI decision-making opaque.

Conventional and Novel Diagnostic Tools for the Diagnosis of Emerging SARS-CoV-2 Variants

Accurate identification at an early stage of infection is critical for effective care of any infectious disease. The "coronavirus disease 2019 (COVID-19)" outbreak, caused by the virus "Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2)", corresponds to the current and global pandemic, characterized by several developing variants, many of which are classified as variants of concern (VOCs) by the "World Health Organization (WHO, Geneva, Switzerland)". The primary diagnosis of infection is made using either the molecular technique of RT-PCR, which detects parts of the viral genome's RNA, or immunodiagnostic procedures, which identify viral proteins or antibodies generated by the host. As the demand for the RT-PCR test grew fast, several inexperienced producers joined the market with innovative kits, and an increasing number of laboratories joined the diagnostic field, rendering the test results increasingly prone to mistakes. It is difficult to determine how the outcomes of one unnoticed result could influence decisions about patient quarantine and social isolation, particularly when the patients themselves are health care providers. The development of point-of-care testing helps in the rapid in-field diagnosis of the disease, and such testing can also be used as a bedside monitor for mapping the progression of the disease in critical patients. In this review, we have provided the readers with available molecular diagnostic techniques and their pitfalls in detecting emerging VOCs of SARS-CoV-2, and lastly, we have discussed AI-ML- and nanotechnology-based smart diagnostic techniques for SARS-CoV-2 detection.

AI revolution in healthcare and medicine and the (re-)emergence of inequalities and disadvantages for ageing population

AI systems in medicine and healthcare are being extensively explored in prevention, diagnosis, novel drug designs and after-care. The application of AI technology in healthcare systems promises impressive outcomes such as equalising healthcare, reducing mortality rate and human error, reducing medical costs, as well as reducing reliance on social services. In the light of the WHO "Decade of Healthy Ageing", AI applications are designed as digital innovations to support the quality of life for older persons. However, the emergence of evidence of different types of algorithmic bias in AI applications, ageism in the use of digital devices and platforms, as well as age bias in digital data suggests that the use of AI might have discriminatory effects on older population or even cause harm. This paper addresses the issue of age biases and age discrimination in AI applications in medicine and healthcare systems and try to identify main challenges in this area. It will reflect on the potential of AI applications to amplify the already existing health inequalities by discussing two levels where potential negative impact of AI on age inequalities might be observed. Firstly, we will address the technical level of age bias in algorithms and digital datasets (especially health data). Secondly, we will discuss the potential disparate outcomes of automatic decision-making systems (ADMs) used in healthcare on the older population. These examples will demonstrate, although only partially, how AI systems may create new structures of age inequalities and novel dimensions of exclusion in healthcare and medicine.

Machine learning approaches for electronic health records phenotyping: a methodical review

OBJECTIVE

Accurate and rapid phenotyping is a prerequisite to leveraging electronic health records for biomedical research. While early phenotyping relied on rule-based algorithms curated by experts, machine learning (ML) approaches have emerged as an alternative to improve scalability across phenotypes and healthcare settings. This study evaluates ML-based phenotyping with respect to (1) the data sources used, (2) the phenotypes considered, (3) the methods applied, and (4) the reporting and evaluation methods used.

MATERIALS AND METHODS

We searched PubMed and Web of Science for articles published between 2018 and 2022. After screening 850 articles, we recorded 37 variables on 100 studies.

RESULTS

Most studies utilized data from a single institution and included information in clinical notes. Although chronic conditions were most commonly considered, ML also enabled the characterization of nuanced phenotypes such as social determinants of health. Supervised deep learning was the most popular ML paradigm, while semi-supervised and weakly supervised learning were applied to expedite algorithm development and unsupervised learning to facilitate phenotype discovery. ML approaches did not uniformly outperform rule-based algorithms, but deep learning offered a marginal improvement over traditional ML for many conditions.

DISCUSSION

Despite the progress in ML-based phenotyping, most articles focused on binary phenotypes and few articles evaluated external validity or used multi-institution data. Study settings were infrequently reported and analytic code was rarely released.

CONCLUSION

Continued research in ML-based phenotyping is warranted, with emphasis on characterizing nuanced phenotypes, establishing reporting and evaluation standards, and developing methods to accommodate misclassified phenotypes due to algorithm errors in downstream applications.

AI-deploying organizations are key to addressing 'perfect storm' of AI risks

We argue that a perfect storm of five conditions heightens the risk of harm to society from artificial intelligence: (1) the powerful, invisible nature of AI, (2) low public awareness and AI literacy, (3) rapid scaled deployment of AI, (4) insufficient regulation, and (5) the gap between trustworthy AI principles and practices. To prevent harm, fit-for-purpose regulation and public AI literacy programs have been recommended, but education and government regulation will not be sufficient: AI-deploying organizations need to play a central role in creating and deploying trustworthy AI in line with the principles of trustworthy AI, and taking accountability to mitigate the risks.

Artificial intelligence and health inequities in primary care: a systematic scoping review and framework

OBJECTIVE

Artificial intelligence (AI) will have a significant impact on healthcare over the coming decade. At the same time, health inequity remains one of the biggest challenges. Primary care is both a driver and a mitigator of health inequities and with AI gaining traction in primary care, there is a need for a holistic understanding of how AI affect health inequities, through the act of providing care and through potential system effects. This paper presents a systematic scoping review of the ways AI implementation in primary care may impact health inequity.

DESIGN

Following a systematic scoping review approach, we searched for literature related to AI, health inequity, and implementation challenges of AI in primary care. In addition, articles from primary exploratory searches were added, and through reference screening.The results were thematically summarised and used to produce both a narrative and conceptual model for the mechanisms by which social determinants of health and AI in primary care could interact to either improve or worsen health inequities.Two public advisors were involved in the review process.

ELIGIBILITY CRITERIA

Peer-reviewed publications and grey literature in English and Scandinavian languages.

INFORMATION SOURCES

PubMed, SCOPUS and JSTOR.

RESULTS

A total of 1529 publications were identified, of which 86 met the inclusion criteria. The findings were summarised under six different domains, covering both positive and negative effects: (1) access, (2) trust, (3) dehumanisation, (4) agency for self-care, (5) algorithmic bias and (6) external effects. The five first domains cover aspects of the interface between the patient and the primary care system, while the last domain covers care system-wide and societal effects of AI in primary care. A graphical model has been produced to illustrate this. Community involvement throughout the whole process of designing and implementing of AI in primary care was a common suggestion to mitigate the potential negative effects of AI.

CONCLUSION

AI has the potential to affect health inequities through a multitude of ways, both directly in the patient consultation and through transformative system effects. This review summarises these effects from a system tive and provides a base for future research into responsible implementation.

Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging

A growing number of artificial intelligence (AI)-based systems are being proposed and developed in cardiology, driven by the increasing need to deal with the vast amount of clinical and imaging data with the ultimate aim of advancing patient care, diagnosis and prognostication. However, there is a critical gap between the development and clinical deployment of AI tools. A key consideration for implementing AI tools into real-life clinical practice is their "trustworthiness" by end-users. Namely, we must ensure that AI systems can be trusted and adopted by all parties involved, including clinicians and patients. Here we provide a summary of the concepts involved in developing a "trustworthy AI system." We describe the main risks of AI applications and potential mitigation techniques for the wider application of these promising techniques in the context of cardiovascular imaging. Finally, we show why trustworthy AI concepts are important governing forces of AI development.

Artificial intelligence applications used in the clinical response to COVID-19: A scoping review

Research into using artificial intelligence (AI) in health care is growing and several observers predicted that AI would play a key role in the clinical response to the COVID-19. Many AI models have been proposed though previous reviews have identified only a few applications used in clinical practice. In this study, we aim to (1) identify and characterize AI applications used in the clinical response to COVID-19; (2) examine the timing, location, and extent of their use; (3) examine how they relate to pre-pandemic applications and the U.S. regulatory approval process; and (4) characterize the evidence that is available to support their use. We searched academic and grey literature sources to identify 66 AI applications that performed a wide range of diagnostic, prognostic, and triage functions in the clinical response to COVID-19. Many were deployed early in the pandemic and most were used in the U.S., other high-income countries, or China. While some applications were used to care for hundreds of thousands of patients, others were used to an unknown or limited extent. We found studies supporting the use of 39 applications, though few of these were independent evaluations and we found no clinical trials evaluating any application's impact on patient health. Due to limited evidence, it is impossible to determine the extent to which the clinical use of AI in the pandemic response has benefited patients overall. Further research is needed, particularly independent evaluations on AI application performance and health impacts in real-world care settings.

Ethical considerations for precision psychiatry: A roadmap for research and clinical practice

Precision psychiatry is an emerging field with transformative opportunities for mental health. However, the use of clinical prediction models carries unprecedented ethical challenges, which must be addressed before accessing the potential benefits of precision psychiatry. This critical review covers multidisciplinary areas, including psychiatry, ethics, statistics and machine-learning, healthcare and academia, as well as input from people with lived experience of mental disorders, their family, and carers. We aimed to identify core ethical considerations for precision psychiatry and mitigate concerns by designing a roadmap for research and clinical practice. We identified priorities: learning from somatic medicine; identifying precision psychiatry use cases; enhancing transparency and generalizability; fostering implementation; promoting mental health literacy; communicating risk estimates; data protection and privacy; and fostering the equitable distribution of mental health care. We hope this blueprint will advance research and practice and enable people with mental health problems to benefit from precision psychiatry.

Data science in the intensive care unit

In this editorial, we comment on the current development and deployment of data science in intensive care units (ICUs). Data in ICUs can be classified into qualitative and quantitative data with different technologies needed to translate and interpret them. Data science, in the form of artificial intelligence (AI), should find the right interaction between physicians, data and algorithm. For individual patients and physicians, sepsis and mechanical ventilation have been two important aspects where AI has been extensively studied. However, major risks of bias, lack of generalizability and poor clinical values remain. AI deployment in the ICUs should be emphasized more to facilitate AI development. For ICU management, AI has a huge potential in transforming resource allocation. The coronavirus disease 2019 pandemic has given opportunities to establish such systems which should be investigated further. Ethical concerns must be addressed when designing such AI.

Transparent human – (non-) transparent technology? The Janus-faced call for transparency in AI-based health care technologies

The use of Artificial Intelligence and Big Data in health care opens up new opportunities for the measurement of the human. Their application aims not only at gathering more and better data points but also at doing it less invasive. With this change in health care towards its extension to almost all areas of life and its increasing invisibility and opacity, new questions of transparency arise. While the complex human-machine interactions involved in deploying and using AI tend to become non-transparent, the use of these technologies makes the patient seemingly transparent. Papers on the ethical implementation of AI plead for transparency but neglect the factor of the “transparent patient” as intertwined with AI. Transparency in this regard appears to be Janus-faced: The precondition for receiving help - e.g., treatment advice regarding the own health - is to become transparent for the digitized health care system. That is, for instance, to donate data and become visible to the AI and its operators. The paper reflects on this entanglement of transparent patients and (non-) transparent technology. It argues that transparency regarding both AI and humans is not an ethical principle per se but an infraethical concept. Further, it is no sufficient basis for avoiding harm and human dignity violations. Rather, transparency must be enriched by intelligibility following Judith Butler’s use of the term. Intelligibility is understood as an epistemological presupposition for recognition and the ensuing humane treatment. Finally, the paper highlights ways to testify intelligibility in dealing with AI in health care ex ante, ex post, and continuously.

Translational Bioinformatics to Enable Precision Medicine for All: Elevating Equity across Molecular, Clinical, and Digital Realms

OBJECTIVES

Over the past few years, challenges from the pandemic have led to an explosion of data sharing and algorithmic development efforts in the areas of molecular measurements, clinical data, and digital health. We aim to characterize and describe recent advanced computational approaches in translational bioinformatics across these domains in the context of issues or progress related to equity and inclusion.

METHODS

We conducted a literature assessment of the trends and approaches in translational bioinformatics in the past few years.

RESULTS

We present a review of recent computational approaches across molecular, clinical, and digital realms. We discuss applications of phenotyping, disease subtype characterization, predictive modeling, biomarker discovery, and treatment selection. We consider these methods and applications through the lens of equity and inclusion in biomedicine.

CONCLUSION

Equity and inclusion should be incorporated at every step of translational bioinformatics projects, including project design, data collection, model creation, and clinical implementation. These considerations, coupled with the exciting breakthroughs in big data and machine learning, are pivotal to reach the goals of precision medicine for all.

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models

OBJECTIVE

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias.

MATERIALS AND METHODS

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

RESULTS

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

DISCUSSION

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

CONCLUSION

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Combining the advantages of AlexNet convolutional deep neural network optimized with anopheles search algorithm based feature extraction and random forest classifier for COVID-19 classification

In this article, COVID-19 detection and classification framework based on anopheles search optimized AlexNet convolutional deep neural network for random forest classifier is implemented. Here, the COVID-19 dataset is taken from Joseph Paul Cohen database. Then, the input images are preprocessed with the help of fuzzy gray level difference histogram equalization technique (FGLHE) and fuzzy stacking technique for color enhancement and noise elimination in the input images. The FGLHE technique and fuzzy stacking technique are combined together and forms into stacked dataset image. This stacked dataset are trained with AlexNet convolutional deep neural network model and the feature packages acquired via the models are processed by the anopheles search algorithm. Subsequently, the efficient features are combined and delivered to random forest (RF) classifier. The proposed approach is implemented in MATLAB. The proposed ADCNN-ASA-RFC provides 91.66%, 69.13%, 34.86%, and 70.13% higher accuracy, 79.13%, 60.33%, and 63.34% higher specificity and 77.13%, 58.45%, 25.86%, and 55.33%, higher sensitivity compared with existing algorithms. At last, the simulation outcomes demonstrate that the proposed system can be able to find the optimal solutions efficiently and accurately with COVID-19 diagnosis.

Accuracy of Automated Computer-Aided Diagnosis for Stroke Imaging: A Critical Evaluation of Current Evidence

There is increasing interest in computer applications, using artificial intelligence methodologies, to perform health care tasks previously performed by humans, particularly in medical imaging for diagnosis. In stroke, there are now commercial artificial intelligence software for use with computed tomography or MR imaging to identify acute ischemic brain tissue pathology, arterial obstruction on computed tomography angiography or as hyperattenuated arteries on computed tomography, brain hemorrhage, or size of perfusion defects. A rapid, accurate diagnosis may aid treatment decisions for individual patients and could improve outcome if it leads to effective and safe treatment; or conversely, to disaster if a delayed or incorrect diagnosis results in inappropriate treatment. Despite this potential clinical impact, diagnostic tools including artificial intelligence methods are not subjected to the same clinical evaluation standards as are mandatory for drugs. Here, we provide an evidence-based review of the pros and cons of commercially available automated methods for medical imaging diagnosis, including those based on artificial intelligence, to diagnose acute brain pathology on computed tomography or magnetic resonance imaging in patients with stroke.

Standardizing, Harmonizing, and Protecting Data Collection to Broaden the Impact of COVID-19 Research: The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-up) Initiative

Objective

The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) program is a consortium of community-engaged research projects with the goal of increasing access to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) tests in underserved populations. To accelerate clinical research, common data elements (CDEs) were selected and refined to standardize data collection and enhance cross-consortium analysis.

Materials and Methods

The RADx-UP consortium began with more than 700 CDEs from the National Institutes of Health (NIH) CDE Repository, Disaster Research Response (DR2) guidelines, and the PHENotypes and eXposures (PhenX) Toolkit. Following a review of initial CDEs, we made selections and further refinements through an iterative process that included live forums, consultations, and surveys completed by the first 69 RADx-UP projects.

Results

Following a multistep CDE development process, we decreased the number of CDEs, modified the question types, and changed the CDE wording. Most research projects were willing to collect and share demographic NIH Tier 1 CDEs, with the top exception reason being a lack of CDE applicability to the project. The NIH RADx-UP Tier 1 CDE with the lowest frequency of collection and sharing was sexual orientation.

Discussion

We engaged a wide range of projects and solicited bidirectional input to create CDEs. These RADx-UP CDEs could serve as the foundation for a patient-centered informatics architecture allowing the integration of disease-specific databases to support hypothesis-driven clinical research in underserved populations.

Conclusion

A community-engaged approach using bidirectional feedback can lead to the better development and implementation of CDEs in underserved populations during public health emergencies.

Accuracy of Cloud-Based Speech Recognition Open Application Programming Interface for Medical Terms of Korean

BACKGROUND

There are limited data on the accuracy of cloud-based speech recognition (SR) open application programming interfaces (APIs) for medical terminology. This study aimed to evaluate the medical term recognition accuracy of current available cloud-based SR open APIs in Korean.

METHODS

We analyzed the SR accuracy of currently available cloud-based SR open APIs using real doctor-patient conversation recordings collected from an outpatient clinic at a large tertiary medical center in Korea. For each original and SR transcription, we analyzed the accuracy rate of each cloud-based SR open API (i.e., the number of medical terms in the SR transcription per number of medical terms in the original transcription).

RESULTS

A total of 112 doctor-patient conversation recordings were converted with three cloud-based SR open APIs (Naver Clova SR from Naver Corporation; Google Speech-to-Text from Alphabet Inc.; and Amazon Transcribe from Amazon), and each transcription was compared. Naver Clova SR (75.1%) showed the highest accuracy with the recognition of medical terms compared to the other open APIs (Google Speech-to-Text, 50.9%, P < 0.001; Amazon Transcribe, 57.9%, P < 0.001), and Amazon Transcribe demonstrated higher recognition accuracy compared to Google Speech-to-Text (P < 0.001). In the sub-analysis, Naver Clova SR showed the highest accuracy in all areas according to word classes, but the accuracy of words longer than five characters showed no statistical differences (Naver Clova SR, 52.6%; Google Speech-to-Text, 56.3%; Amazon Transcribe, 36.6%).

CONCLUSION

Among three current cloud-based SR open APIs, Naver Clova SR which manufactured by Korean company showed highest accuracy of medical terms in Korean, compared to Google Speech-to-Text and Amazon Transcribe. Although limitations are existing in the recognition of medical terminology, there is a lot of rooms for improvement of this promising technology by combining strengths of each SR engines.

Algorithmic Political Bias in Artificial Intelligence Systems

Some artificial intelligence (AI) systems can display algorithmic bias, i.e. they may produce outputs that unfairly discriminate against people based on their social identity. Much research on this topic focuses on algorithmic bias that disadvantages people based on their gender or racial identity. The related ethical problems are significant and well known. Algorithmic bias against other aspects of people’s social identity, for instance, their political orientation, remains largely unexplored. This paper argues that algorithmic bias against people’s political orientation can arise in some of the same ways in which algorithmic gender and racial biases emerge. However, it differs importantly from them because there are (in a democratic society) strong social norms against gender and racial biases. This does not hold to the same extent for political biases. Political biases can thus more powerfully influence people, which increases the chances that these biases become embedded in algorithms and makes algorithmic political biases harder to detect and eradicate than gender and racial biases even though they all can produce similar harm. Since some algorithms can now also easily identify people’s political orientations against their will, these problems are exacerbated. Algorithmic political bias thus raises substantial and distinctive risks that the AI community should be aware of and examine.

The lockdown, mobility, and spatial health disparities in COVID-19 pandemic: A case study of New York City

The world has adopted unprecedented lockdown as the key method to mitigate COVID-19; yet its effect on pandemic outcomes and health disparities remains largely unknown. Adopting a multilevel conceptual framework, this research investigates how city-level lockdown policy and public transit system shape mobility and thus intra-city health disparities, using New York City as a case study. With a spatial method and multiple sources of data, this research demonstrates the uneven impact of the lockdown policy and public transit system in shaping local pandemic outcomes. Census tracts with people spending more time at home have lower infection and death rates, while those with a higher density of transit stations have higher infection and death rates. Residential profile matters and census tracts with a higher concentration of disadvantaged population, such as Blacks, Hispanics, poor and elderly people, and people with no health insurance, have higher infection and death rates. Spatial analyses identify clusters where the lockdown policy was not effective and census tracts that share similar pandemic characteristics. Through the lens of mobility, this research advances knowledge of health disparities by focusing on institutional causes for health disparities and the role of the government through intervention policy and public transit system.

Integrating artificial intelligence in bedside care for covid-19 and future pandemics

Michael Yu and colleagues examine the challenges in developing AI tools for use at point of care

Data-Driven Analytics Leveraging Artificial Intelligence in the Era of COVID-19: An Insightful Review of Recent Developments

This paper presents the role of artificial intelligence (AI) and other latest technologies that were employed to fight the recent pandemic (i.e., novel coronavirus disease-2019 (COVID-19)). These technologies assisted the early detection/diagnosis, trends analysis, intervention planning, healthcare burden forecasting, comorbidity analysis, and mitigation and control, to name a few. The key-enablers of these technologies was data that was obtained from heterogeneous sources (i.e., social networks (SN), internet of (medical) things (IoT/IoMT), cellular networks, transport usage, epidemiological investigations, and other digital/sensing platforms). To this end, we provide an insightful overview of the role of data-driven analytics leveraging AI in the era of COVID-19. Specifically, we discuss major services that AI can provide in the context of COVID-19 pandemic based on six grounds, (i) AI role in seven different epidemic containment strategies (a.k.a non-pharmaceutical interventions (NPIs)), (ii) AI role in data life cycle phases employed to control pandemic via digital solutions, (iii) AI role in performing analytics on heterogeneous types of data stemming from the COVID-19 pandemic, (iv) AI role in the healthcare sector in the context of COVID-19 pandemic, (v) general-purpose applications of AI in COVID-19 era, and (vi) AI role in drug design and repurposing (e.g., iteratively aligning protein spikes and applying three/four-fold symmetry to yield a low-resolution candidate template) against COVID-19. Further, we discuss the challenges involved in applying AI to the available data and privacy issues that can arise from personal data transitioning into cyberspace. We also provide a concise overview of other latest technologies that were increasingly applied to limit the spread of the ongoing pandemic. Finally, we discuss the avenues of future research in the respective area. This insightful review aims to highlight existing AI-based technological developments and future research dynamics in this area.