Regulation of predictive analytics in medicine

Bioactivity and Bioavailability of Carotenoids Applied in Human Health: Technological Advances and Innovation

This article presents a groundbreaking perspective on carotenoids, focusing on their innovative applications and transformative potential in human health and medicine. Research jointly delves deeper into the bioactivity and bioavailability of carotenoids, revealing therapeutic uses and technological advances that have the potential to revolutionize medical treatments. We explore pioneering therapeutic applications in which carotenoids are used to treat chronic diseases such as cancer, cardiovascular disease, and age-related macular degeneration, offering novel protective mechanisms and innovative therapeutic benefits. Our study also shows cutting-edge technological innovations in carotenoid extraction and bioavailability, including the development of supramolecular carriers and advanced nanotechnology, which dramatically improve the absorption and efficacy of these compounds. These technological advances not only ensure consistent quality but also tailor carotenoid therapies to each patient's health needs, paving the way for personalized medicine. By integrating the latest scientific discoveries and innovative techniques, this research provides a prospective perspective on the clinical applications of carotenoids, establishing a new benchmark for future studies in this field. Our findings underscore the importance of optimizing carotenoid extraction, administration, bioactivity, and bioavailability methods to develop more effective, targeted, and personalized treatments, thus offering visionary insight into their potential in modern medical practices.

Artificial Intelligence for Smoking Cessation in Pregnancy

Artificial intelligence (AI) has emerged as a revolutionary tool in various healthcare domains, including smoking cessation among pregnant women. Smoking during pregnancy is a significant public health concern, linked to adverse maternal and fetal outcomes. Traditional cessation methods have had limited success, necessitating innovative approaches. AI offers personalized interventions, predictive analytics, and real-time support, enhancing the effectiveness of smoking cessation programs. This editorial explores the potential of AI in transforming smoking cessation efforts for pregnant women, highlighting its benefits, challenges, and future prospects. By integrating AI into healthcare strategies, we can improve maternal and fetal health outcomes and contribute to the broader public health goal of reducing smoking rates among expectant mothers.

Unveiling the Influence of AI Predictive Analytics on Patient Outcomes: A Comprehensive Narrative Review

This comprehensive literature review explores the transformative impact of artificial intelligence (AI) predictive analytics on healthcare, particularly in improving patient outcomes regarding disease progression, treatment response, and recovery rates. AI, encompassing capabilities such as learning, problem-solving, and decision-making, is leveraged to predict disease progression, optimize treatment plans, and enhance recovery rates through the analysis of vast datasets, including electronic health records (EHRs), imaging, and genetic data. The utilization of machine learning (ML) and deep learning (DL) techniques in predictive analytics enables personalized medicine by facilitating the early detection of conditions, precision in drug discovery, and the tailoring of treatment to individual patient profiles. Ethical considerations, including data privacy, bias, and accountability, emerge as vital in the responsible implementation of AI in healthcare. The findings underscore the potential of AI predictive analytics in revolutionizing clinical decision-making and healthcare delivery, emphasizing the necessity of ethical guidelines and continuous model validation to ensure its safe and effective use in augmenting human judgment in medical practice.

Prognostic Value of a Combined Nomogram Model Integrating 3-Dimensional Deep Learning and Radiomics for Head and Neck Cancer

OBJECTIVE

The preoperative prediction of the overall survival (OS) status of patients with head and neck cancer (HNC) is significant value for their individualized treatment and prognosis. This study aims to evaluate the impact of adding 3D deep learning features to radiomics models for predicting 5-year OS status.

METHODS

Two hundred twenty cases from The Cancer Imaging Archive public dataset were included in this study; 2212 radiomics features and 304 deep features were extracted from each case. The features were selected by univariate analysis and the least absolute shrinkage and selection operator, and then grouped into a radiomics model containing Positron Emission Tomography /Computed Tomography (PET/CT) radiomics features score, a deep model containing deep features score, and a combined model containing PET/CT radiomics features score +3D deep features score. TumorStage model was also constructed using initial patient tumor node metastasis stage to compare the performance of the combined model. A nomogram was constructed to analyze the influence of deep features on the performance of the model. The 10-fold cross-validation of the average area under the receiver operating characteristic curve and calibration curve were used to evaluate performance, and Shapley Additive exPlanations (SHAP) was developed for interpretation.

RESULTS

The TumorStage model, radiomics model, deep model, and the combined model achieved areas under the receiver operating characteristic curve of 0.604, 0.851, 0.840, and 0.895 on the train set and 0.571, 0.849, 0.832, and 0.900 on the test set. The combined model showed better performance of predicting the 5-year OS status of HNC patients than the radiomics model and deep model. The combined model was shown to provide a favorable fit in calibration curves and be clinically useful in decision curve analysis. SHAP summary plot and SHAP The SHAP summary plot and SHAP force plot visually interpreted the influence of deep features and radiomics features on the model results.

CONCLUSIONS

In predicting 5-year OS status in patients with HNC, 3D deep features could provide richer features for combined model, which showed outperformance compared with the radiomics model and deep model.

Clinical data mining: challenges, opportunities, and recommendations for translational applications

Clinical data mining of predictive models offers significant advantages for re-evaluating and leveraging large amounts of complex clinical real-world data and experimental comparison data for tasks such as risk stratification, diagnosis, classification, and survival prediction. However, its translational application is still limited. One challenge is that the proposed clinical requirements and data mining are not synchronized. Additionally, the exotic predictions of data mining are difficult to apply directly in local medical institutions. Hence, it is necessary to incisively review the translational application of clinical data mining, providing an analytical workflow for developing and validating prediction models to ensure the scientific validity of analytic workflows in response to clinical questions. This review systematically revisits the purpose, process, and principles of clinical data mining and discusses the key causes contributing to the detachment from practice and the misuse of model verification in developing predictive models for research. Based on this, we propose a niche-targeting framework of four principles: Clinical Contextual, Subgroup-Oriented, Confounder- and False Positive-Controlled (CSCF), to provide guidance for clinical data mining prior to the model's development in clinical settings. Eventually, it is hoped that this review can help guide future research and develop personalized predictive models to achieve the goal of discovering subgroups with varied remedial benefits or risks and ensuring that precision medicine can deliver its full potential.

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Over the past two decades, advances in computational power and data availability combined with increased accessibility to pre-trained models have led to an exponential rise in machine learning (ML) publications. While ML may have the potential to transform healthcare, this sharp increase in ML research output without focus on methodological rigor and standard reporting guidelines has fueled a reproducibility crisis. In addition, the rapidly growing complexity of these models compromises their interpretability, which currently impedes their successful and widespread clinical adoption. In medicine, where failure of such models may have severe implications for patients' health, the high requirements for accuracy, robustness, and interpretability confront ML researchers with a unique set of challenges. In this review, we discuss the semantics of reproducibility and interpretability, as well as related issues and challenges, and outline possible solutions to counteracting the "black box". To foster reproducibility, standard reporting guidelines need to be further developed and data or code sharing encouraged. Editors and reviewers may equally play a critical role by establishing high methodological standards and thus preventing the dissemination of low-quality ML publications. To foster interpretable learning, the use of simpler models more suitable for medical data can inform the clinician how results are generated based on input data. Model-agnostic explanation tools, sensitivity analysis, and hidden layer representations constitute further promising approaches to increase interpretability. Balancing model performance and interpretability are important to ensure clinical applicability. We have now reached a critical moment for ML in medicine, where addressing these issues and implementing appropriate solutions will be vital for the future evolution of the field.

Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare

In 2020, the U.S. Department of Defense officially disclosed a set of ethical principles to guide the use of Artificial Intelligence (AI) technologies on future battlefields. Despite stark differences, there are core similarities between the military and medical service. Warriors on battlefields often face life-altering circumstances that require quick decision-making. Medical providers experience similar challenges in a rapidly changing healthcare environment, such as in the emergency department or during surgery treating a life-threatening condition. Generative AI, an emerging technology designed to efficiently generate valuable information, holds great promise. As computing power becomes more accessible and the abundance of health data, such as electronic health records, electrocardiograms, and medical images, increases, it is inevitable that healthcare will be revolutionized by this technology. Recently, generative AI has garnered a lot of attention in the medical research community, leading to debates about its application in the healthcare sector, mainly due to concerns about transparency and related issues. Meanwhile, questions around the potential exacerbation of health disparities due to modeling biases have raised notable ethical concerns regarding the use of this technology in healthcare. However, the ethical principles for generative AI in healthcare have been understudied. As a result, there are no clear solutions to address ethical concerns, and decision-makers often neglect to consider the significance of ethical principles before implementing generative AI in clinical practice. In an attempt to address these issues, we explore ethical principles from the military perspective and propose the "GREAT PLEA" ethical principles, namely Governability, Reliability, Equity, Accountability, Traceability, Privacy, Lawfulness, Empathy, and Eutonomy, for generative AI in healthcare. Furthermore, we introduce a framework for adopting and expanding these ethical principles in a practical way that has been useful in the military and can be applied to healthcare for generative AI, based on contrasting their ethical concerns and risks. Ultimately, we aim to proactively address the ethical dilemmas and challenges posed by the integration of generative AI into healthcare practice.

Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care

Importance

Health care algorithms are used for diagnosis, treatment, prognosis, risk stratification, and allocation of resources. Bias in the development and use of algorithms can lead to worse outcomes for racial and ethnic minoritized groups and other historically marginalized populations such as individuals with lower income.

Objective

To provide a conceptual framework and guiding principles for mitigating and preventing bias in health care algorithms to promote health and health care equity.

Evidence Review

The Agency for Healthcare Research and Quality and the National Institute for Minority Health and Health Disparities convened a diverse panel of experts to review evidence, hear from stakeholders, and receive community feedback.

Findings

The panel developed a conceptual framework to apply guiding principles across an algorithm's life cycle, centering health and health care equity for patients and communities as the goal, within the wider context of structural racism and discrimination. Multiple stakeholders can mitigate and prevent bias at each phase of the algorithm life cycle, including problem formulation (phase 1); data selection, assessment, and management (phase 2); algorithm development, training, and validation (phase 3); deployment and integration of algorithms in intended settings (phase 4); and algorithm monitoring, maintenance, updating, or deimplementation (phase 5). Five principles should guide these efforts: (1) promote health and health care equity during all phases of the health care algorithm life cycle; (2) ensure health care algorithms and their use are transparent and explainable; (3) authentically engage patients and communities during all phases of the health care algorithm life cycle and earn trustworthiness; (4) explicitly identify health care algorithmic fairness issues and trade-offs; and (5) establish accountability for equity and fairness in outcomes from health care algorithms.

Conclusions and Relevance

Multiple stakeholders must partner to create systems, processes, regulations, incentives, standards, and policies to mitigate and prevent algorithmic bias. Reforms should implement guiding principles that support promotion of health and health care equity in all phases of the algorithm life cycle as well as transparency and explainability, authentic community engagement and ethical partnerships, explicit identification of fairness issues and trade-offs, and accountability for equity and fairness.

The use of individual and multilevel data in the development of a risk prediction model to predict patients' likelihood of completing colorectal cancer screening

Promotion of colorectal cancer (CRC) screening can be expensive and unnecessary for many patients. The use of predictive analytics promises to help health systems target the right services to the right patients at the right time while improving population health. Multilevel data at the interpersonal, organizational, community, and policy levels, is rarely considered in clinical decision making but may be used to improve CRC screening risk prediction. We compared the effectiveness of a CRC screening risk prediction model that uses multilevel data with a more conventional model that uses only individual patient data. We used a retrospective cohort to ascertain the one-year occurrence of CRC screening. The cohort was determined from a Health Maintenance Organization, in Oregon. Eligible patients were 50-75 years old, health plan members for at least one year before their birthday in 2018 and were due for screening. We created a risk model using logistic regression first with data available in the electronic health record (EHR), and then added multilevel data. In a cohort of 59,249 patients, 36.1% completed CRC screening. The individual level model included 14 demographic, clinical and encounter based characteristics, had a bootstrap-corrected C-statistic of 0.722 and sufficient calibration. The multilevel model added 9 variables from clinical setting and community characteristics, and the bootstrap-corrected C-statistic remained the same with continued sufficient calibration. The predictive power of the CRC screening model did not improve after adding multilevel data. Our findings suggest that multilevel data added no improvement to the prediction of the likelihood of CRC screening.

Diagnostic accuracy of point-of-care ultrasound with artificial intelligence-assisted assessment of left ventricular ejection fraction

Focused cardiac ultrasound (FoCUS) is becoming standard practice in a wide spectrum of clinical settings. There is limited data evaluating the real-world use of FoCUS with artificial intelligence (AI). Our objective was to determine the accuracy of FoCUS AI-assisted left ventricular ejection fraction (LVEF) assessment and compare its accuracy between novice and experienced users. In this prospective, multicentre study, participants requiring a transthoracic echocardiogram (TTE) were recruited to have a FoCUS done by a novice or experienced user. The AI-assisted device calculated LVEF at the bedside, which was subsequently compared to TTE. 449 participants were enrolled with 424 studies included in the final analysis. The overall intraclass coefficient was 0.904, and 0.921 in the novice (n = 208) and 0.845 in the experienced (n = 216) cohorts. There was a significant bias of 0.73% towards TTE (p = 0.005) with a level of agreement of 11.2%. Categorical grading of LVEF severity had excellent agreement to TTE (weighted kappa = 0.83). The area under the curve (AUC) was 0.98 for identifying an abnormal LVEF (<50%) with a sensitivity of 92.8%, specificity of 92.3%, negative predictive value (NPV) of 0.97 and a positive predictive value (PPV) of 0.83. In identifying severe dysfunction (<30%) the AUC was 0.99 with a sensitivity of 78.1%, specificity of 98.0%, NPV of 0.98 and PPV of 0.76. Here we report that FoCUS AI-assisted LVEF assessments provide highly reproducible LVEF estimations in comparison to formal TTE. This finding was consistent among senior and novice echocardiographers suggesting applicability in a variety of clinical settings.

Fair and equitable AI in biomedical research and healthcare: Social science perspectives

Artificial intelligence (AI) offers opportunities but also challenges for biomedical research and healthcare. This position paper shares the results of the international conference "Fair medicine and AI" (online 3-5 March 2021). Scholars from science and technology studies (STS), gender studies, and ethics of science and technology formulated opportunities, challenges, and research and development desiderata for AI in healthcare. AI systems and solutions, which are being rapidly developed and applied, may have undesirable and unintended consequences including the risk of perpetuating health inequalities for marginalized groups. Socially robust development and implications of AI in healthcare require urgent investigation. There is a particular dearth of studies in human-AI interaction and how this may best be configured to dependably deliver safe, effective and equitable healthcare. To address these challenges, we need to establish diverse and interdisciplinary teams equipped to develop and apply medical AI in a fair, accountable and transparent manner. We formulate the importance of including social science perspectives in the development of intersectionally beneficent and equitable AI for biomedical research and healthcare, in part by strengthening AI health evaluation.

Interpretable artificial intelligence in radiology and radiation oncology

Artificial intelligence has been introduced to clinical practice, especially radiology and radiation oncology, from image segmentation, diagnosis, treatment planning and prognosis. It is not only crucial to have an accurate artificial intelligence model, but also to understand the internal logic and gain the trust of the experts. This review is intended to provide some insights into core concepts of the interpretability, the state-of-the-art methods for understanding the machine learning models, the evaluation of these methods, identifying some challenges and limits of them, and gives some examples of medical applications.

Computer Science meets Vascular Surgery: Keeping a pulse on artificial intelligence

Artificial intelligence (AI)-based technologies have garnered interest across a range of disciplines in the past several years, with an even more recent interest in various health care fields, including Vascular Surgery. AI offers a unique ability to analyze health data more quickly and efficiently than could be done by humans alone and can be used for clinical applications such as diagnosis, risk stratification, and follow-up, as well as patient-used applications to improve both patient and provider experiences, mitigate health care disparities, and individualize treatment. As with all novel technologies, AI is not without its risks and carries with it unique ethical considerations that will need to be addressed before its broad integration into health care systems. AI has the potential to revolutionize the way care is provided to patients, including those requiring vascular care.

Clinical use of artificial intelligence requires AI-capable organizations

Artificial intelligence-based algorithms are being widely implemented in health care, even as evidence is emerging of bias in their design, problems with implementation, and potential harm to patients. To achieve the promise of using of AI-based tools to improve health, healthcare organizations will need to be AI-capable, with internal and external systems functioning in tandem to ensure the safe, ethical, and effective use of AI-based tools. Ideas are starting to emerge about the organizational routines, competencies, resources, and infrastructures that will be required for safe and effective deployment of AI in health care, but there has been little empirical research. Infrastructures that provide legal and regulatory guidance for managers, clinician competencies for the safe and effective use of AI-based tools, and learner-centric resources such as clear AI documentation and local health ecosystem impact reviews can help drive continuous improvement.

A model-based approach for historical borrowing, with an application to neovascular age-related macular degeneration

Bayesian historical borrowing has recently attracted growing interest due to the increasing availability of historical control data, as well as improved computational methodology and software. In this article, we argue that the statistical models used for borrowing may be suboptimal when they do not adjust for differing factors across historical studies such as covariates, dosing regimen, etc. We propose an alternative approach to address these shortcomings. We start by constructing a historical model based on subject-level historical data to accurately characterize the control treatment by adjusting for known between trials differences. This model is subsequently used to predict the control arm response in the current trial, enabling the derivation of a model-informed prior for the treatment effect parameter of another (potentially simpler) model used to analyze the trial efficacy (i.e. the trial model). Our approach is applied to neovascular age-related macular degeneration trials, employing a cross-sectional regression trial model, and a longitudinal non-linear mixed-effects drug-disease-trial historical model. The latter model characterizes the relationship between clinical response, drug exposure and baseline covariates so that the derived model-informed prior seamlessly adapts to the trial population and can be extrapolated to a different dosing regimen. This approach can yield a more accurate prior for borrowing, thus optimizing gains in efficiency (e.g. increasing power or reducing the sample size) in future trials.

Methods for Clinical Evaluation of Artificial Intelligence Algorithms for Medical Diagnosis

Adequate clinical evaluation of artificial intelligence (AI) algorithms before adoption in practice is critical. Clinical evaluation aims to confirm acceptable AI performance through adequate external testing and confirm the benefits of AI-assisted care compared with conventional care through appropriately designed and conducted studies, for which prospective studies are desirable. This article explains some of the fundamental methodological points that should be considered when designing and appraising the clinical evaluation of AI algorithms for medical diagnosis. The specific topics addressed include the following: (a) the importance of external testing of AI algorithms and strategies for conducting the external testing effectively, (b) the various metrics and graphical methods for evaluating the AI performance as well as essential methodological points to note in using and interpreting them, (c) paired study designs primarily for comparative performance evaluation of conventional and AI-assisted diagnoses, (d) parallel study designs primarily for evaluating the effect of AI intervention with an emphasis on randomized clinical trials, and (e) up-to-date guidelines for reporting clinical studies on AI, with an emphasis on guidelines registered in the EQUATOR Network library. Sound methodological knowledge of these topics will aid the design, execution, reporting, and appraisal of clinical evaluation of AI.

Public Trust in Artificial Intelligence Applications in Mental Health Care: Topic Modeling Analysis

Background

Mental disorders (MDs) impose heavy burdens on health care (HC) systems and affect a growing number of people worldwide. The use of mobile health (mHealth) apps empowered by artificial intelligence (AI) is increasingly being resorted to as a possible solution.

Objective

This study adopted a topic modeling (TM) approach to investigate the public trust in AI apps in mental health care (MHC) by identifying the dominant topics and themes in user reviews of the 8 most relevant mental health (MH) apps with the largest numbers of reviewers.

Methods

We searched Google Play for the top MH apps with the largest numbers of reviewers, from which we selected the most relevant apps. Subsequently, we extracted data from user reviews posted from January 1, 2020, to April 2, 2022. After cleaning the extracted data using the Python text processing tool spaCy, we ascertained the optimal number of topics, drawing on the coherence scores and used latent Dirichlet allocation (LDA) TM to generate the most salient topics and related terms. We then classified the ascertained topics into different theme categories by plotting them onto a 2D plane via multidimensional scaling using the pyLDAvis visualization tool. Finally, we analyzed these topics and themes qualitatively to better understand the status of public trust in AI apps in MHC.

Results

From the top 20 MH apps with the largest numbers of reviewers retrieved, we chose the 8 (40%) most relevant apps: (1) Wysa: Anxiety Therapy Chatbot; (2) Youper Therapy; (3) MindDoc: Your Companion; (4) TalkLife for Anxiety, Depression & Stress; (5) 7 Cups: Online Therapy for Mental Health & Anxiety; (6) BetterHelp-Therapy; (7) Sanvello; and (8) InnerHour. These apps provided 14.2% (n=559), 11.0% (n=431), 13.7% (n=538), 8.8% (n=356), 14.1% (n=554), 11.9% (n=468), 9.2% (n=362), and 16.9% (n=663) of the collected 3931 reviews, respectively. The 4 dominant topics were topic 4 (cheering people up; n=1069, 27%), topic 3 (calming people down; n=1029, 26%), topic 2 (helping figure out the inner world; n=963, 25%), and topic 1 (being an alternative or complement to a therapist; n=870, 22%). Based on topic coherence and intertopic distance, topics 3 and 4 were combined into theme 3 (dispelling negative emotions), while topics 2 and 1 remained 2 separate themes: theme 2 (helping figure out the inner world) and theme 1 (being an alternative or complement to a therapist), respectively. These themes and topics, though involving some dissenting voices, reflected an overall high status of trust in AI apps.

Conclusions

This is the first study to investigate the public trust in AI apps in MHC from the perspective of user reviews using the TM technique. The automatic text analysis and complementary manual interpretation of the collected data allowed us to discover the dominant topics hidden in a data set and categorize these topics into different themes to reveal an overall high degree of public trust. The dissenting voices from users, though only a few, can serve as indicators for health providers and app developers to jointly improve these apps, which will ultimately facilitate the treatment of prevalent MDs and alleviate the overburdened HC systems worldwide.

Factors influencing clinicians' willingness to use an AI-based clinical decision support system

Background

Given the opportunities created by artificial intelligence (AI) based decision support systems in healthcare, the vital question is whether clinicians are willing to use this technology as an integral part of clinical workflow.

Purpose

This study leverages validated questions to formulate an online survey and consequently explore cognitive human factors influencing clinicians' intention to use an AI-based Blood Utilization Calculator (BUC), an AI system embedded in the electronic health record that delivers data-driven personalized recommendations for the number of packed red blood cells to transfuse for a given patient.

Method

A purposeful sampling strategy was used to exclusively include BUC users who are clinicians in a university hospital in Wisconsin. We recruited 119 BUC users who completed the entire survey. We leveraged structural equation modeling to capture the direct and indirect effects of “AI Perception” and “Expectancy” on clinicians' Intention to use the technology when mediated by “Perceived Risk”.

Results

The findings indicate a significant negative relationship concerning the direct impact of AI's perception on BUC Risk (ß = −0.23, p < 0.001). Similarly, Expectancy had a significant negative effect on Risk (ß = −0.49, p < 0.001). We also noted a significant negative impact of Risk on the Intent to use BUC (ß = −0.34, p < 0.001). Regarding the indirect effect of Expectancy on the Intent to Use BUC, the findings show a significant positive impact mediated by Risk (ß = 0.17, p = 0.004). The study noted a significant positive and indirect effect of AI Perception on the Intent to Use BUC when mediated by risk (ß = 0.08, p = 0.027). Overall, this study demonstrated the influences of expectancy, perceived risk, and perception of AI on clinicians' intent to use BUC (an AI system). AI developers need to emphasize the benefits of AI technology, ensure ease of use (effort expectancy), clarify the system's potential (performance expectancy), and minimize the risk perceptions by improving the overall design.

Conclusion

Identifying the factors that determine clinicians' intent to use AI-based decision support systems can help improve technology adoption and use in the healthcare domain. Enhanced and safe adoption of AI can uplift the overall care process and help standardize clinical decisions and procedures. An improved AI adoption in healthcare will help clinicians share their everyday clinical workload and make critical decisions.

Digital Health Applications in Oncology: An Opportunity to Seize

Digital health advances have transformed many clinical areas including psychiatric and cardiovascular care. However, digital health innovation is relatively nascent in cancer care, which represents the fastest growing area of health-care spending. Opportunities for digital health innovation in oncology include patient-facing technologies that improve patient experience, safety, and patient-clinician interactions; clinician-facing technologies that improve their ability to diagnose pathology and predict adverse events; and quality of care and research infrastructure to improve clinical workflows, documentation, decision support, and clinical trial monitoring. The COVID-19 pandemic and associated shifts of care to the home and community dramatically accelerated the integration of digital health technologies into virtually every aspect of oncology care. However, the pandemic has also exposed potential flaws in the digital health ecosystem, namely in clinical integration strategies; data access, quality, and security; and regulatory oversight and reimbursement for digital health technologies. Stemming from the proceedings of a 2020 workshop convened by the National Cancer Policy Forum of the National Academies of Sciences, Engineering, and Medicine, this article summarizes the current state of digital health technologies in medical practice and strategies to improve clinical utility and integration. These recommendations, with calls to action for clinicians, health systems, technology innovators, and policy makers, will facilitate efficient yet safe integration of digital health technologies into cancer care.

Generalizability of Cardiovascular Disease Clinical Prediction Models: 158 Independent External Validations of 104 Unique Models

Background:

While clinical prediction models (CPMs) are used increasingly commonly to guide patient care, the performance and clinical utility of these CPMs in new patient cohorts is poorly understood.

Methods:

We performed 158 external validations of 104 unique CPMs across 3 domains of cardiovascular disease (primary prevention, acute coronary syndrome, and heart failure). Validations were performed in publicly available clinical trial cohorts and model performance was assessed using measures of discrimination, calibration, and net benefit. To explore potential reasons for poor model performance, CPM-clinical trial cohort pairs were stratified based on relatedness, a domain-specific set of characteristics to qualitatively grade the similarity of derivation and validation patient populations. We also examined the model-based C-statistic to assess whether changes in discrimination were because of differences in case-mix between the derivation and validation samples. The impact of model updating on model performance was also assessed.

Results:

Discrimination decreased significantly between model derivation (0.76 [interquartile range 0.73–0.78]) and validation (0.64 [interquartile range 0.60–0.67], P<0.001), but approximately half of this decrease was because of narrower case-mix in the validation samples. CPMs had better discrimination when tested in related compared with distantly related trial cohorts. Calibration slope was also significantly higher in related trial cohorts (0.77 [interquartile range, 0.59–0.90]) than distantly related cohorts (0.59 [interquartile range 0.43–0.73], P=0.001). When considering the full range of possible decision thresholds between half and twice the outcome incidence, 91% of models had a risk of harm (net benefit below default strategy) at some threshold; this risk could be reduced substantially via updating model intercept, calibration slope, or complete re-estimation.

Conclusions:

There are significant decreases in model performance when applying cardiovascular disease CPMs to new patient populations, resulting in substantial risk of harm. Model updating can mitigate these risks. Care should be taken when using CPMs to guide clinical decision-making.

Concordance rate of radiologists and a commercialized deep-learning solution for chest X-ray: Real-world experience with a multicenter health screening cohort

Purpose

Lunit INSIGHT CXR (Lunit) is a commercially available deep-learning algorithm-based decision support system for chest radiography (CXR). This retrospective study aimed to evaluate the concordance rate of radiologists and Lunit for thoracic abnormalities in a multicenter health screening cohort.

Methods and materials

We retrospectively evaluated the radiology reports and Lunit results for CXR at several health screening centers in August 2020. Lunit was adopted as a clinical decision support system (CDSS) in routine clinical practice. Subsequently, radiologists completed their reports after reviewing the Lunit results. The DLA result was provided as a color map with an abnormality score (%) for thoracic lesions when the score was greater than the predefined cutoff value of 15%. Concordance was achieved when (a) the radiology reports were consistent with the DLA results (“accept”), (b) the radiology reports were partially consistent with the DLA results (“edit”) or had additional lesions compared with the DLA results (“add”). There was discordance when the DLA results were rejected in the radiology report. In addition, we compared the reading times before and after Lunit was introduced. Finally, we evaluated systemic usability scale questionnaire for radiologists and physicians who had experienced Lunit.

Results

Among 3,113 participants (1,157 men; mean age, 49 years), thoracic abnormalities were found in 343 (11.0%) based on the CXR radiology reports and 621 (20.1%) based on the Lunit results. The concordance rate was 86.8% (accept: 85.3%, edit: 0.9%, and add: 0.6%), and the discordance rate was 13.2%. Except for 479 cases (7.5%) for whom reading time data were unavailable (n = 5) or unreliable (n = 474), the median reading time increased after the clinical integration of Lunit (median, 19s vs. 14s, P < 0.001).

Conclusion

The real-world multicenter health screening cohort showed a high concordance of the chest X-ray report and the Lunit result under the clinical integration of the deep-learning solution. The reading time slight increased with the Lunit assistance.

Building a Precision Medicine Delivery Platform for Clinics: The University of California, San Francisco, BRIDGE Experience

Despite an ever-expanding number of analytics with the potential to impact clinical care, the field currently lacks point-of-care technological tools that allow clinicians to efficiently select disease-relevant data about their patients, algorithmically derive clinical indices (eg, risk scores), and view these data in straightforward graphical formats to inform real-time clinical decisions. Thus far, solutions to this problem have relied on either bottom-up approaches that are limited to a single clinic or generic top-down approaches that do not address clinical users’ specific setting-relevant or disease-relevant needs. As a road map for developing similar platforms, we describe our experience with building a custom but institution-wide platform that enables economies of time, cost, and expertise. The BRIDGE platform was designed to be modular and scalable and was customized to data types relevant to given clinical contexts within a major university medical center. The development process occurred by using a series of human-centered design phases with extensive, consistent stakeholder input. This institution-wide approach yielded a unified, carefully regulated, cross-specialty clinical research platform that can be launched during a patient’s electronic health record encounter. The platform pulls clinical data from the electronic health record (Epic; Epic Systems) as well as other clinical and research sources in real time; analyzes the combined data to derive clinical indices; and displays them in simple, clinician-designed visual formats specific to each disorder and clinic. By integrating an application into the clinical workflow and allowing clinicians to access data sources that would otherwise be cumbersome to assemble, view, and manipulate, institution-wide platforms represent an alternative approach to achieving the vision of true personalized medicine.

Evaluation of Prognostic and Predictive Models in the Oncology Clinic

Predictive and prognostic models hold great potential to support clinical decision making in oncology and could ultimately facilitate a paradigm shift to a more personalised form of treatment. While a large number of models relevant to the field of oncology have been developed, few have been translated into clinical use and assessment of clinical utility is not currently considered a routine part of model development. In this narrative review of the clinical evaluation of prediction models in oncology, we propose a high-level process diagram for the life cycle of a clinical model, encompassing model commissioning, clinical implementation and ongoing quality assurance, which aims to bridge the gap between model development and clinical implementation.

Point-of-care screening for heart failure with reduced ejection fraction using artificial intelligence during ECG-enabled stethoscope examination in London, UK: a prospective, observational, multicentre study

Background

Most patients who have heart failure with a reduced ejection fraction, when left ventricular ejection fraction (LVEF) is 40% or lower, are diagnosed in hospital. This is despite previous presentations to primary care with symptoms. We aimed to test an artificial intelligence (AI) algorithm applied to a single-lead ECG, recorded during ECG-enabled stethoscope examination, to validate a potential point-of-care screening tool for LVEF of 40% or lower.

Methods

We conducted an observational, prospective, multicentre study of a convolutional neural network (known as AI-ECG) that was previously validated for the detection of reduced LVEF using 12-lead ECG as input. We used AI-ECG retrained to interpret single-lead ECG input alone. Patients (aged ≥18 years) attending for transthoracic echocardiogram in London (UK) were recruited. All participants had 15 s of supine, single-lead ECG recorded at the four standard anatomical positions for cardiac auscultation, plus one handheld position, using an ECG-enabled stethoscope. Transthoracic echocardiogram-derived percentage LVEF was used as ground truth. The primary outcome was performance of AI-ECG at classifying reduced LVEF (LVEF ≤40%), measured using metrics including the area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity, with two-sided 95% CIs. The primary outcome was reported for each position individually and with an optimal combination of AI-ECG outputs (interval range 0–1) from two positions using a rule-based approach and several classification models. This study is registered with ClinicalTrials.gov, NCT04601415.

Findings

Between Feb 6 and May 27, 2021, we recruited 1050 patients (mean age 62 years [SD 17·4], 535 [51%] male, 432 [41%] non-White). 945 (90%) had an ejection fraction of at least 40%, and 105 (10%) had an ejection fraction of 40% or lower. Across all positions, ECGs were most frequently of adequate quality for AI-ECG interpretation at the pulmonary position (979 [93·3%] of 1050). Quality was lowest for the aortic position (846 [80·6%]). AI-ECG performed best at the pulmonary valve position (p=0·02), with an AUROC of 0·85 (95% CI 0·81–0·89), sensitivity of 84·8% (76·2–91·3), and specificity of 69·5% (66·4–72·6). Diagnostic odds ratios did not differ by age, sex, or non-White ethnicity. Taking the optimal combination of two positions (pulmonary and handheld positions), the rule-based approach resulted in an AUROC of 0·85 (0·81–0·89), sensitivity of 82·7% (72·7–90·2), and specificity of 79·9% (77·0–82·6). Using AI-ECG outputs from these two positions, a weighted logistic regression with l2 regularisation resulted in an AUROC of 0·91 (0·88–0·95), sensitivity of 91·9% (78·1–98·3), and specificity of 80·2% (75·5–84·3).

Interpretation

A deep learning system applied to single-lead ECGs acquired during a routine examination with an ECG-enabled stethoscope can detect LVEF of 40% or lower. These findings highlight the potential for inexpensive, non-invasive, workflow-adapted, point-of-care screening, for earlier diagnosis and prognostically beneficial treatment.

Funding

NHS Accelerated Access Collaborative, NHSX, and the National Institute for Health Research.

Artificial Intelligence to Improve Health Outcomes in the NICU and PICU: A Systematic Review

CONTEXT

Artificial intelligence (AI) technologies are increasingly used in pediatrics and have the potential to help inpatient physicians provide high-quality care for critically ill children.

OBJECTIVE

We aimed to describe the use of AI to improve any health outcome(s) in neonatal and pediatric intensive care.

DATA SOURCE

PubMed, IEEE Xplore, Cochrane, and Web of Science databases.

STUDY SELECTION

We used peer-reviewed studies published between June 1, 2010, and May 31, 2020, in which researchers described (1) AI, (2) pediatrics, and (3) intensive care. Studies were included if researchers assessed AI use to improve at least 1 health outcome (eg, mortality).

DATA EXTRACTION

Data extraction was conducted independently by 2 researchers. Articles were categorized by direct or indirect impact of AI, defined by the European Institute of Innovation and Technology Health joint report.

RESULTS

Of the 287 publications screened, 32 met inclusion criteria. Approximately 22% (n = 7) of studies revealed a direct impact and improvement in health outcomes after AI implementation. Majority were in prototype testing, and few were deployed into an ICU setting. Among the remaining 78% (n = 25) AI models outperformed standard clinical modalities and may have indirectly influenced patient outcomes. Quantitative assessment of health outcomes using statistical measures, such as area under the receiver operating curve (56%; n = 18) and specificity (38%; n = 12), revealed marked heterogeneity in metrics and standardization.

CONCLUSIONS

Few studies have revealed that AI has directly improved health outcomes for pediatric critical care patients. Further prospective, experimental studies are needed to assess AI's impact by using established implementation frameworks, standardized metrics, and validated outcome measures.

Predictive Analytics in Clinical Practice: Advantages and Disadvantages

Predictive analytics are increasingly reported by clinicians. These tools aim to improve patient outcomes in terms of quality, safety, and efficiency. However, deploying predictive analytics in clinical practice remains challenging today. We highlight several advantages and disadvantages of the application of predictive analytics in clinical practice. To flourish and reach its potential, predictive analytics need data that is of adequate quantity and quality, ideally tailored to clinical scenarios in equipoise regarding optimal management. Adequate reporting of predictive analytic tools is incumbent for uptake into clinical workflows. At least for now, the clinicians' knowledge, experience, and vigilance remain imperative for applying predictive analytics in clinical practice.

What Makes Artificial Intelligence Exceptional in Health Technology Assessment?

The application of artificial intelligence (AI) may revolutionize the healthcare system, leading to enhance efficiency by automatizing routine tasks and decreasing health-related costs, broadening access to healthcare delivery, targeting more precisely patient needs, and assisting clinicians in their decision-making. For these benefits to materialize, governments and health authorities must regulate AI, and conduct appropriate health technology assessment (HTA). Many authors have highlighted that AI health technologies (AIHT) challenge traditional evaluation and regulatory processes. To inform and support HTA organizations and regulators in adapting their processes to AIHTs, we conducted a systematic review of the literature on the challenges posed by AIHTs in HTA and health regulation. Our research question was: What makes artificial intelligence exceptional in HTA? The current body of literature appears to portray AIHTs as being exceptional to HTA. This exceptionalism is expressed along 5 dimensions: 1) AIHT's distinctive features; 2) their systemic impacts on health care and the health sector; 3) the increased expectations towards AI in health; 4) the new ethical, social and legal challenges that arise from deploying AI in the health sector; and 5) the new evaluative constraints that AI poses to HTA. Thus, AIHTs are perceived as exceptional because of their technological characteristics and potential impacts on society at large. As AI implementation by governments and health organizations carries risks of generating new, and amplifying existing, challenges, there are strong arguments for taking into consideration the exceptional aspects of AIHTs, especially as their impacts on the healthcare system will be far greater than that of drugs and medical devices. As AIHTs begin to be increasingly introduced into the health care sector, there is a window of opportunity for HTA agencies and scholars to consider AIHTs' exceptionalism and to work towards only deploying clinically, economically, socially acceptable AIHTs in the health care system.

Artificial Intelligence and Mapping a New Direction in Laboratory Medicine: A Review

BACKGROUND

Modern artificial intelligence (AI) and machine learning (ML) methods are now capable of completing tasks with performance characteristics that are comparable to those of expert human operators. As a result, many areas throughout healthcare are incorporating these technologies, including in vitro diagnostics and, more broadly, laboratory medicine. However, there are limited literature reviews of the landscape, likely future, and challenges of the application of AI/ML in laboratory medicine.

CONTENT

In this review, we begin with a brief introduction to AI and its subfield of ML. The ensuing sections describe ML systems that are currently in clinical laboratory practice or are being proposed for such use in recent literature, ML systems that use laboratory data outside the clinical laboratory, challenges to the adoption of ML, and future opportunities for ML in laboratory medicine.

SUMMARY

AI and ML have and will continue to influence the practice and scope of laboratory medicine dramatically. This has been made possible by advancements in modern computing and the widespread digitization of health information. These technologies are being rapidly developed and described, but in comparison, their implementation thus far has been modest. To spur the implementation of reliable and sophisticated ML-based technologies, we need to establish best practices further and improve our information system and communication infrastructure. The participation of the clinical laboratory community is essential to ensure that laboratory data are sufficiently available and incorporated conscientiously into robust, safe, and clinically effective ML-supported clinical diagnostics.

Artificial Intelligence and Liability in Medicine: Balancing Safety and Innovation

Policy Points With increasing integration of artificial intelligence and machine learning in medicine, there are concerns that algorithm inaccuracy could lead to patient injury and medical liability. While prior work has focused on medical malpractice, the artificial intelligence ecosystem consists of multiple stakeholders beyond clinicians. Current liability frameworks are inadequate to encourage both safe clinical implementation and disruptive innovation of artificial intelligence. Several policy options could ensure a more balanced liability system, including altering the standard of care, insurance, indemnification, special/no-fault adjudication systems, and regulation. Such liability frameworks could facilitate safe and expedient implementation of artificial intelligence and machine learning in clinical care.

An explainable supervised machine learning predictor of acute kidney injury after adult deceased donor liver transplantation

Background

Early prediction of acute kidney injury (AKI) after liver transplantation (LT) facilitates timely recognition and intervention. We aimed to build a risk predictor of post-LT AKI via supervised machine learning and visualize the mechanism driving within to assist clinical decision-making.

Methods

Data of 894 cases that underwent liver transplantation from January 2015 to September 2019 were collected, covering demographics, donor characteristics, etiology, peri-operative laboratory results, co-morbidities and medications. The primary outcome was new-onset AKI after LT according to Kidney Disease Improving Global Outcomes guidelines. Predicting performance of five classifiers including logistic regression, support vector machine, random forest, gradient boosting machine (GBM) and adaptive boosting were respectively evaluated by the area under the receiver-operating characteristic curve (AUC), accuracy, F1-score, sensitivity and specificity. Model with the best performance was validated in an independent dataset involving 195 adult LT cases from October 2019 to March 2021. SHapley Additive exPlanations (SHAP) method was applied to evaluate feature importance and explain the predictions made by ML algorithms.

Results

430 AKI cases (55.1%) were diagnosed out of 780 included cases. The GBM model achieved the highest AUC (0.76, CI 0.70 to 0.82), F1-score (0.73, CI 0.66 to 0.79) and sensitivity (0.74, CI 0.66 to 0.8) in the internal validation set, and a comparable AUC (0.75, CI 0.67 to 0.81) in the external validation set. High preoperative indirect bilirubin, low intraoperative urine output, long anesthesia time, low preoperative platelets, and graft steatosis graded NASH CRN 1 and above were revealed by SHAP method the top 5 important variables contributing to the diagnosis of post-LT AKI made by GBM model.

Conclusions

Our GBM-based predictor of post-LT AKI provides a highly interoperable tool across institutions to assist decision-making after LT.

Graphic abstract

Supplementary Information

The online version contains supplementary material available at 10.1186/s12967-021-02990-4.

Development of a risk prediction nomogram for disposition of acute toxic exposure patients to intensive care unit

Early risk stratification of acutely poisoned patients is essential to identify patients at high risk of intensive care unit (ICU) admission. We aimed to develop a prognostic model and risk-stratification nomogram based on the readily accessible clinical and laboratory predictors on admission for the probability of ICU admission in acutely poisoned patients. This retrospective cohort study included adult patients with acute toxic exposure to a drug or a chemical substance. Patients' demographic, toxicologic, clinical and laboratory data were collected. Among the 1260 eligible patients, 180 (14.3%) were admitted to the ICU. We developed a generalized prognostic model for predicting ICU admission in patients with acute poisoning. The predictors included the Glasgow coma scale, oxygen saturation, diastolic blood pressure, respiratory rate and blood bicarbonate concentration. The model displayed excellent discrimination and calibration (optimistic-adjusted area under the curve = 0.924 and optimistic-adjusted Hosmer and Lemeshow test = 0.922, respectively) when internally validated. Additionally, we developed prognostic models that determine ICU admission in patients with specific poisonings. Furthermore, we constructed risk-stratification nomograms that rank the probability of ICU admission in these patients. The developed risk-stratification nomograms help decision-making regarding ICU admission in acute poisonings. Future external validation in independent cohorts is necessary before clinical application.

Ethical Machine Learning in Healthcare

The use of machine learning (ML) in healthcare raises numerous ethical concerns, especially as models can amplify existing health inequities. Here, we outline ethical considerations for equitable ML in the advancement of healthcare. Specifically, we frame ethics of ML in healthcare through the lens of social justice. We describe ongoing efforts and outline challenges in a proposed pipeline of ethical ML in health, ranging from problem selection to postdeployment considerations. We close by summarizing recommendations to address these challenges.

Artificial intelligence in hospitals: providing a status quo of ethical considerations in academia to guide future research

The application of artificial intelligence (AI) in hospitals yields many advantages but also confronts healthcare with ethical questions and challenges. While various disciplines have conducted specific research on the ethical considerations of AI in hospitals, the literature still requires a holistic overview. By conducting a systematic discourse approach highlighted by expert interviews with healthcare specialists, we identified the status quo of interdisciplinary research in academia on ethical considerations and dimensions of AI in hospitals. We found 15 fundamental manuscripts by constructing a citation network for the ethical discourse, and we extracted actionable principles and their relationships. We provide an agenda to guide academia, framed under the principles of biomedical ethics. We provide an understanding of the current ethical discourse of AI in clinical environments, identify where further research is pressingly needed, and discuss additional research questions that should be addressed. We also guide practitioners to acknowledge AI-related benefits in hospitals and to understand the related ethical concerns.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00146-021-01239-4.

Application of miRNA-seq in neuropsychiatry: A methodological perspective

MiRNAs are emerging as key molecules to study neuropsychiatric diseases. However, despite the large number of methodologies and software for miRNA-seq analyses, there is little supporting literature for researchers in this area. This review focuses on evaluating how miRNA-seq has been used to study neuropsychiatric diseases to date, analyzing both the main findings discovered and the bioinformatics workflows and tools used from a methodological perspective. The objective of this review is two-fold: first, to evaluate current miRNA-seq procedures used in neuropsychiatry; and second, to offer comprehensive information that can serve as a guide to new researchers in bioinformatics. After conducting a systematic search (from 2016 to June 30, 2020) of articles using miRNA-seq in neuropsychiatry, we have seen that it has already been used for different types of studies in three main categories: diagnosis, prognosis, and mechanism. We carefully analyzed the bioinformatics workflows of each study, observing a high degree of variability with respect to the tools and methods used and several methodological complexities that are identified and discussed in this review.

Effect of Having, but Not Consulting, a Computerized Diagnostic Aid

Previous research has described physicians' reluctance to use computerized diagnostic aids (CDAs) but has never experimentally examined the effects of not consulting an aid that was readily available. Experiment 1. Participants read about a diagnosis made either by a physician or an auto mechanic (to control for perceived expertise). Half read that a CDA was available but never actually consulted; no mention of a CDA was made for the remaining half. For the physician, failure to consult the CDA had no significant effect on competence ratings for either the positive or negative outcome. For the auto mechanic, failure to consult the CDA actually increased competence ratings following a negative but not a positive outcome. Negligence judgments were greater for the mechanic than for the physician overall. Experiment 2. Using only a negative outcome, we included 2 different reasons for not consulting the aid and provided accuracy information highlighting the superiority of the CDA over the physician. In neither condition was the physician rated lower than when no aid was mentioned. Ratings were lower when the physician did not trust the CDA and, surprisingly, higher when the physician believed he or she already knew what the CDA would say. Finally, consistent with our previous research, ratings were also high when the physician consulted and then followed the advice of a CDA and low when the CDA was consulted but ignored. Individual differences in numeracy did not qualify these results. Implications for the literature on algorithm aversion and clinical practice are discussed.

A COVID-19 Test Triage Tool, Predicting Negative Results and Reducing the Testing Burden on Healthcare Systems During a Pandemic

Background: Detecting and isolating cases of COVID-19 are amongst the key elements listed by the WHO to reduce transmission. This approach has been reported to reduce those symptomatic with COVID-19 in the population by over 90%. Testing is part of a strategy that will save lives. Testing everyone maybe ideal, but it is not practical. A risk tool based on patient demographics and clinical parameters has the potential to help identify patients most likely to test negative for SARS-CoV-2. If effective it could be used to aide clinical decision making and reduce the testing burden. Methods: At the time of this analysis, a total of 9,516 patients with symptoms suggestive of Covid-19, were assessed and tested at Mount Sinai Institutions in New York. Patient demographics, clinical parameters and test results were collected. A robust prediction pipeline was used to develop a risk tool to predict the likelihood of a positive test for Covid-19. The risk tool was analyzed in a holdout dataset from the cohort and its discriminative ability, calibration and net benefit assessed. Results: Over 48% of those tested in this cohort, had a positive result. The derived model had an AUC of 0.77, provided reliable risk prediction, and demonstrated a superior net benefit than a strategy of testing everybody. When a risk cut-off of 70% was applied, the model had a negative predictive value of 96%. Conclusion: Such a tool could be used to help aide but not replace clinical decision making and conserve vital resources needed to effectively tackle this pandemic.

Integrated Analytical Framework for the Development of Artificial Intelligence-Based Medical Devices

INTRODUCTION

A persistent fundamental problem in applying artificial intelligence (AI) to medical devices is achieving high performance while guaranteeing the safety and reliability of AI solutions. However, the regulation of medical devices and that of AI implementations both differ among countries, resulting in "double-layered" differences in the regulations for medical devices employing AI.

METHODS

To overcome these differences and examine the universal requirements for AI-based medical devices, we clarify legal and ethical issues, regulatory requirements, and terms in representative guidances related to AI-based medical devices from different countries. Then, we propose an integrated framework for analyzing the requirements of AI-based medical devices for a comprehensive comparison.

RESULTS

We demonstrate that no guidance completely covers all the evaluation requirements, and the differences in coverage per country may direct the international harmonization of these requirements.

DISCUSSION

Consequently, we show two development strategies available to manufacturers, namely adherence to local regulations with tedious efforts or compliance with universal requirements using our integrated framework even under the low level of international harmonization; the latter is recommended for efficiency and convergence.

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence

Artificial intelligence (AI) will likely affect various fields of medicine. This article aims to explain the fundamental principles of clinical validation, device approval, and insurance coverage decisions of AI algorithms for medical diagnosis and prediction. Discrimination accuracy of AI algorithms is often evaluated with the Dice similarity coefficient, sensitivity, specificity, and traditional or free-response receiver operating characteristic curves. Calibration accuracy should also be assessed, especially for algorithms that provide probabilities to users. As current AI algorithms have limited generalizability to real-world practice, clinical validation of AI should put it to proper external testing and assisting roles. External testing could adopt diagnostic case-control or diagnostic cohort designs. A diagnostic case-control study evaluates the technical validity/accuracy of AI while the latter tests the clinical validity/accuracy of AI in samples representing target patients in real-world clinical scenarios. Ultimate clinical validation of AI requires evaluations of its impact on patient outcomes, referred to as clinical utility, and for which randomized clinical trials are ideal. Device approval of AI is typically granted with proof of technical validity/accuracy and thus does not intend to directly indicate if AI is beneficial for patient care or if it improves patient outcomes. Neither can it categorically address the issue of limited generalizability of AI. After achieving device approval, it is up to medical professionals to determine if the approved AI algorithms are beneficial for real-world patient care. Insurance coverage decisions generally require a demonstration of clinical utility that the use of AI has improved patient outcomes.

High Throughput Multi-Omics Approaches for Clinical Trial Evaluation and Drug Discovery

High throughput single cell multi-omics platforms, such as mass cytometry (cytometry by time-of-flight; CyTOF), high dimensional imaging (>6 marker; Hyperion, MIBIscope, CODEX, MACSima) and the recently evolved genomic cytometry (Citeseq or REAPseq) have enabled unprecedented insights into many biological and clinical questions, such as hematopoiesis, transplantation, cancer, and autoimmunity. In synergy with constantly adapting new single-cell analysis approaches and subsequent accumulating big data collections from these platforms, whole atlases of cell types and cellular and sub-cellular interaction networks are created. These atlases build an ideal scientific discovery environment for reference and data mining approaches, which often times reveals new cellular disease networks. In this review we will discuss how combinations and fusions of different -omic workflows on a single cell level can be used to examine cellular phenotypes, immune effector functions, and even dynamic changes, such as metabolomic state of different cells in a sample or even in a defined tissue location. We will touch on how pre-print platforms help in optimization and reproducibility of workflows, as well as community outreach. We will also shortly discuss how leveraging single cell multi-omic approaches can be used to accelerate cellular biomarker discovery during clinical trials to predict response to therapy, follow responsive cell types, and define novel druggable target pathways. Single cell proteome approaches already have changed how we explore cellular mechanism in disease and during therapy. Current challenges in the field are how we share these disruptive technologies to the scientific communities while still including new approaches, such as genomic cytometry and single cell metabolomics.

Differential diagnostic value of rheumatic symptoms in patients with Whipple's disease

Most patients with Whipple’s disease have rheumatic symptoms. The aim of our prospective, questionnaire-based, non-interventional clinical study was to assess whether these symptoms are useful in guiding the differential diagnosis to the rheumatic disorders. Forty patients with Whipple’s disease, followed by 20 patients for validation and 30 patients with rheumatoid-, 21 with psoriatic-, 15 with palindromic- and 25 with axial spondyloarthritis were recruited for the present investigation. Patients with Whipple’s disease and patients with rheumatic disorders were asked to record rheumatic symptoms on pseudonymized questionnaires. The data obtained were subjected to multiple logistic regression analysis. Episodic pain with rapid onset, springing from joint to joint was most common in patients with palindromic arthritis and second most common and somewhat less conspicuous in Whipple’s disease. Continuous pain in the same joints predominated in patients with rheumatoid-, psoriatic-, and axial spondyloarthritis. Multiple logistic equations resulted in a predicted probability for the diagnosis of Whipple’s disease of 43.4 ± 0.19% (M ± SD) versus a significantly lower probability of 23.8 ± 0.19% (M ± SD) in the aggregate of patients with rheumatic disorders. Mean area under the curve (AUC) ± SD was 0.781 ± 0.044, 95% CI 0.695–0.867, asymptotic significance p < 0.001. The logistic equations predicted probability for the diagnosis of Whipple’s disease in the initial series of 40 patients of 43.4 ± 0.19% was not significantly different in the subsequent 20 patients of 38.2 ± 0.28% (M ± SD) (p = 0.376). The data may be useful in a predictive algorithm for diagnosing Whipple’s disease. The project is registered as clinical study DRK S0001566.

Imaging Beyond Seeing: Early Prognosis of Cancer Treatment

Assessing cancer response to therapeutic interventions has been realized as an important course to early predict curative efficacy and treatment outcomes due to tumor heterogeneity. Compared to the traditional invasive tissue biopsy method, molecular imaging techniques have fundamentally revolutionized the ability to evaluate cancer response in a spatiotemporal manner. The past few years has witnessed a paradigm shift on the efforts from manufacturing functional molecular imaging probes for seeing a tumor to a vantage stage of interpreting the tumor response during different treatments. This review is to stand by the current development of advanced imaging technologies aiming to predict the treatment response in cancer therapy. Special interest is placed on the systems that are able to provide rapid and noninvasive assessment of pharmacokinetic drug fates (e.g., drug distribution, release, and activation) and tumor microenvironment heterogeneity (e.g., tumor cells, macrophages, dendritic cells (DCs), T cells, and inflammatory cells). The current status, practical significance, and future challenges of the emerging artificial intelligence (AI) technology and machine learning in the applications of medical imaging fields is overviewed. Ultimately, the authors hope that this review is timely to spur research interest in molecular imaging and precision medicine.

Do as AI say: susceptibility in deployment of clinical decision-aids

Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.

Comparison of an Artificial Intelligence-Enabled Patient Decision Aid vs Educational Material on Decision Quality, Shared Decision-Making, Patient Experience, and Functional Outcomes in Adults With Knee Osteoarthritis: A Randomized Clinical Trial

IMPORTANCE

Decision aids can help inform appropriate selection of total knee replacement (TKR) for advanced knee osteoarthritis (OA). However, few decision aids combine patient education, preference assessment, and artificial intelligence (AI) using patient-reported outcome measurement data to generate personalized estimations of outcomes to augment shared decision-making (SDM).

OBJECTIVE

To assess the effect of an AI-enabled patient decision aid that includes education, preference assessment, and personalized outcome estimations (using patient-reported outcome measurements) on decision quality, patient experience, functional outcomes, and process-level outcomes among individuals with advanced knee OA considering TKR in comparison with education only.

DESIGN, SETTING, AND PARTICIPANTS

This randomized clinical trial at a single US academic orthopedic practice included 129 new adult patients presenting for OA-related knee pain from March 2019 to January 2020. Data were analyzed from April to May 2020.

INTERVENTION

Patients were randomized into a group that received a decision aid including patient education, preference assessment, and personalized outcome estimations (intervention group) or a group receiving educational material only (control group) alongside usual care.

MAIN OUTCOMES AND MEASURES

The primary outcome was decision quality, measured using the Knee OA Decision Quality Instrument (K-DQI). Secondary outcomes were collaborative decision-making (assessed using the CollaboRATE survey), patient satisfaction with consultation (using a numerical rating scale), Knee Injury and Osteoarthritis Outcome Score Joint Replacement (KOOS JR) score, consultation time, TKR rate, and treatment concordance.

RESULTS

A total of 69 patients in the intervention group (46 [67%] women) and 60 patients in the control group (37 [62%] women) were included in the analysis. The intervention group showed better decisional quality (K-DQI mean difference, 20.0%; SE, 3.02; 95% CI, 14.2%-26.1%; P < .001), collaborative decision-making (CollaboRATE, 8 of 69 [12%] vs 28 of 60 [47%] patients below median; P < .001), satisfaction (numerical rating scale, 9 of 65 [14%] vs 19 of 58 [33%] patients below median; P = .01), and improved functional outcomes at 4 to 6 months (mean [SE] KOOS JR, 4.9 [2.24] points higher in intervention group; 95% CI, 0.8-9.0 points; P = .02). The intervention did not significantly affect consultation time (mean [SE] difference, 2.23 [2.18] minutes; P = .31), TKR rates (16 of 69 [23%] vs 7 of 60 [12%] patients; P = .11), or treatment concordance (58 of 69 [84%] vs 44 of 60 [73%] patients; P = .19).

CONCLUSIONS AND RELEVANCE

In this randomized clinical trial, an AI-enabled decision aid significantly improved decision quality, level of SDM, satisfaction, and physical limitations without significantly impacting consultation times, TKR rates, or treatment concordance in patients with knee OA considering TKR. Decision aids using a personalized, data-driven approach can enhance SDM in the management of knee OA.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03956004.

Clinical Application of Computational Methods in Precision Oncology: A Review

Importance

There is an enormous and growing amount of data available from individual cancer cases, which makes the work of clinical oncologists more demanding. This data challenge has attracted engineers to create software that aims to improve cancer diagnosis or treatment. However, the move to use computers in the oncology clinic for diagnosis or treatment has led to instances of premature or inappropriate use of computational predictive systems.

Objective

To evaluate best practices for developing and assessing the clinical utility of predictive computational methods in oncology.

Evidence Review

The National Cancer Policy Forum and the Board on Mathematical Sciences and Analytics at the National Academies of Sciences, Engineering, and Medicine hosted a workshop to examine the use of multidimensional data derived from patients with cancer and the computational methods used to analyze these data. The workshop convened diverse stakeholders and experts, including computer scientists, oncology clinicians, statisticians, patient advocates, industry leaders, ethicists, leaders of health systems (academic and community based), private and public health insurance carriers, federal agencies, and regulatory authorities. Key characteristics for successful computational oncology were considered in 3 thematic areas: (1) data quality, completeness, sharing, and privacy; (2) computational methods for analysis, interpretation, and use of oncology data; and (3) clinical infrastructure and expertise for best use of computational precision oncology.

Findings

Quality control was found to be essential across all stages, from data collection to data processing, management, and use. Collecting a standardized parsimonious data set at every cancer diagnosis and restaging could enhance reliability and completeness of clinical data for precision oncology. Data completeness refers to key data elements such as information about cancer diagnosis, treatment, and outcomes, while data quality depends on whether appropriate variables have been measured in valid and reliable ways. Collecting data from diverse populations can reduce the risk of creating invalid and biased algorithms. Computational systems that aid clinicians should be classified as software as a medical device and thus regulated according to the potential risk posed. To facilitate appropriate use of computational methods that interpret high-dimensional data in oncology, treating physicians need access to multidisciplinary teams with broad expertise and deep training among a subset of clinical oncology fellows in clinical informatics.

Conclusions and Relevance

Workshop discussions suggested best practices in demonstrating the clinical utility of predictive computational methods for diagnosing or treating cancer.