Multimodal biomedical AI

Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review

This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

PURPOSE

This study aimed to perform multimodal analysis by vision transformer (vViT) in predicting O6-methylguanine-DNA methyl transferase (MGMT) promoter status among adult patients with diffuse glioma using demographics (sex and age), radiomic features, and MRI.

METHODS

The training and test datasets contained 122 patients with 1,570 images and 30 patients with 484 images, respectively. The radiomic features were extracted from enhancing tumors (ET), necrotic tumor cores (NCR), and the peritumoral edematous/infiltrated tissues (ED) using contrast-enhanced T1-weighted images (CE-T1WI) and T2-weighted images (T2WI). The vViT had 9 sectors; 1 demographic sector, 6 radiomic sectors (CE-T1WI ET, CE-T1WI NCR, CE-T1WI ED, T2WI ET, T2WI NCR, and T2WI ED), 2 image sectors (CE-T1WI, and T2WI). Accuracy and area under the curve of receiver-operating characteristics (AUC-ROC) were calculated for the test dataset. The performance of vViT was compared with AlexNet, GoogleNet, VGG16, and ResNet by McNemar and Delong test. Permutation importance (PI) analysis with the Mann-Whitney U test was performed.

RESULTS

The accuracy was 0.833 (95% confidence interval [95%CI]: 0.714-0.877) and the area under the curve of receiver-operating characteristics was 0.840 (0.650-0.995) in the patient-based analysis. The vViT had higher accuracy than VGG16 and ResNet, and had higher AUC-ROC than GoogleNet (p<0.05). The ED radiomic features extracted from the T2-weighted image demonstrated the highest importance (PI=0.239, 95%CI: 0.237-0.240) among all other sectors (p<0.0001).

CONCLUSION

The vViT is a competent deep learning model in predicting MGMT status. The ED radiomic features of the T2-weighted image demonstrated the most dominant contribution.

PARKINSON'S DISEASE CLASSIFICATION USING CONTRASTIVE GRAPH CROSS-VIEW LEARNING WITH MULTIMODAL FUSION OF SPECT IMAGES AND CLINICAL FEATURES

Parkinson's Disease (PD) affects millions globally, impacting movement. Prior research utilized deep learning for PD prediction, primarily focusing on medical images, neglecting the data's underlying manifold structure. This work proposes a multimodal approach encompassing both image and non-image features, leveraging contrastive cross-view graph fusion for PD classification. We introduce a novel multimodal co-attention module, integrating embeddings from separate graph views derived from low-dimensional representations of images and clinical features. This enables more robust and structured feature extraction for improved multi-view data analysis. Additionally, a simplified contrastive loss-based fusion method is devised to enhance cross-view fusion learning. Our graph-view multimodal approach achieves an accuracy of 91% and an area under the receiver operating characteristic curve (AUC) of 92.8% in five-fold cross-validation. It also demonstrates superior predictive capabilities on non-image data compared to solely machine learning-based methods.

Deep Learning Approaches for Medical Image Analysis and Diagnosis

In addition to enhancing diagnostic accuracy, deep learning techniques offer the potential to streamline workflows, reduce interpretation time, and ultimately improve patient outcomes. The scalability and adaptability of deep learning algorithms enable their deployment across diverse clinical settings, ranging from radiology departments to point-of-care facilities. Furthermore, ongoing research efforts focus on addressing the challenges of data heterogeneity, model interpretability, and regulatory compliance, paving the way for seamless integration of deep learning solutions into routine clinical practice. As the field continues to evolve, collaborations between clinicians, data scientists, and industry stakeholders will be paramount in harnessing the full potential of deep learning for advancing medical image analysis and diagnosis. Furthermore, the integration of deep learning algorithms with other technologies, including natural language processing and computer vision, may foster multimodal medical data analysis and clinical decision support systems to improve patient care. The future of deep learning in medical image analysis and diagnosis is promising. With each success and advancement, this technology is getting closer to being leveraged for medical purposes. Beyond medical image analysis, patient care pathways like multimodal imaging, imaging genomics, and intelligent operating rooms or intensive care units can benefit from deep learning models.

Artificial intelligence in neurology: opportunities, challenges, and policy implications

Neurological conditions are the leading cause of disability and mortality combined, demanding innovative, scalable, and sustainable solutions. Brain health has become a global priority with adoption of the World Health Organization's Intersectoral Global Action Plan in 2022. Simultaneously, rapid advancements in artificial intelligence (AI) are revolutionizing neurological research and practice. This scoping review of 66 original articles explores the value of AI in neurology and brain health, systematizing the landscape for emergent clinical opportunities and future trends across the care trajectory: prevention, risk stratification, early detection, diagnosis, management, and rehabilitation. AI's potential to advance personalized precision neurology and global brain health directives hinges on resolving core challenges across four pillars-models, data, feasibility/equity, and regulation/innovation-through concerted pursuit of targeted recommendations. Paramount actions include swift, ethical, equity-focused integration of novel technologies into clinical workflows, mitigating data-related issues, counteracting digital inequity gaps, and establishing robust governance frameworks balancing safety and innovation.

Ultra-secure storage and analysis of genetic data for the advancement of precision medicine

Cloud computing provides the opportunity to store the ever-growing genotype-phenotype data sets needed to achieve the full potential of precision medicine. However, due to the sensitive nature of this data and the patchwork of data privacy laws across states and countries, additional security protections are proving necessary to ensure data privacy and security. Here we present SQUiD, a secure queryable database for storing and analyzing genotype-phenotype data. With SQUiD, genotype-phenotype data can be stored in a low-security, low-cost public cloud in the encrypted form, which researchers can securely query without the public cloud ever being able to decrypt the data. We demonstrate the usability of SQUiD by replicating various commonly used calculations such as polygenic risk scores, cohort creation for GWAS, MAF filtering, and patient similarity analysis both on synthetic and UK Biobank data. Our work represents a new and scalable platform enabling the realization of precision medicine without security and privacy concerns.

Prioritizing Patient Experiences in the Management of Diabetes and Its Complications: An Endocrine Society Position Statement

Diabetes can be an arduous journey both for people with diabetes (PWD) and their caregivers. While the journey of every person with diabetes is unique, common themes emerge in managing this disease. To date, the experiences of PWD have not been fully considered to successfully implement the recommended standards of diabetes care in practice. It is critical for health-care providers (HCPs) to recognize perspectives of PWD to achieve optimal health outcomes. Further, existing tools are available to facilitate patient-centered care but are often underused. This statement summarizes findings from multistakeholder expert roundtable discussions hosted by the Endocrine Society that aimed to identify existing gaps in the management of diabetes and its complications and to identify tools needed to empower HCPs and PWD to address their many challenges. The roundtables included delegates from professional societies, governmental organizations, patient advocacy organizations, and social enterprises committed to making life better for PWD. Each section begins with a clinical scenario that serves as a framework to achieve desired health outcomes and includes a discussion of resources for HCPs to deliver patient-centered care in clinical practice. As diabetes management evolves, achieving this goal will also require the development of new tools to help guide HCPs in supporting PWD, as well as concrete strategies for the efficient uptake of these tools in clinical practice to minimize provider burden. Importantly, coordination among various stakeholders including PWD, HCPs, caregivers, policymakers, and payers is critical at all stages of the patient journey.

Evaluating ChatGPT-4's Diagnostic Accuracy: Impact of Visual Data Integration

BACKGROUND

In the evolving field of health care, multimodal generative artificial intelligence (AI) systems, such as ChatGPT-4 with vision (ChatGPT-4V), represent a significant advancement, as they integrate visual data with text data. This integration has the potential to revolutionize clinical diagnostics by offering more comprehensive analysis capabilities. However, the impact on diagnostic accuracy of using image data to augment ChatGPT-4 remains unclear.

OBJECTIVE

This study aims to assess the impact of adding image data on ChatGPT-4's diagnostic accuracy and provide insights into how image data integration can enhance the accuracy of multimodal AI in medical diagnostics. Specifically, this study endeavored to compare the diagnostic accuracy between ChatGPT-4V, which processed both text and image data, and its counterpart, ChatGPT-4, which only uses text data.

METHODS

We identified a total of 557 case reports published in the American Journal of Case Reports from January 2022 to March 2023. After excluding cases that were nondiagnostic, pediatric, and lacking image data, we included 363 case descriptions with their final diagnoses and associated images. We compared the diagnostic accuracy of ChatGPT-4V and ChatGPT-4 without vision based on their ability to include the final diagnoses within differential diagnosis lists. Two independent physicians evaluated their accuracy, with a third resolving any discrepancies, ensuring a rigorous and objective analysis.

RESULTS

The integration of image data into ChatGPT-4V did not significantly enhance diagnostic accuracy, showing that final diagnoses were included in the top 10 differential diagnosis lists at a rate of 85.1% (n=309), comparable to the rate of 87.9% (n=319) for the text-only version (P=.33). Notably, ChatGPT-4V's performance in correctly identifying the top diagnosis was inferior, at 44.4% (n=161), compared with 55.9% (n=203) for the text-only version (P=.002, χ2 test). Additionally, ChatGPT-4's self-reports showed that image data accounted for 30% of the weight in developing the differential diagnosis lists in more than half of cases.

CONCLUSIONS

Our findings reveal that currently, ChatGPT-4V predominantly relies on textual data, limiting its ability to fully use the diagnostic potential of visual information. This study underscores the need for further development of multimodal generative AI systems to effectively integrate and use clinical image data. Enhancing the diagnostic performance of such AI systems through improved multimodal data integration could significantly benefit patient care by providing more accurate and comprehensive diagnostic insights. Future research should focus on overcoming these limitations, paving the way for the practical application of advanced AI in medicine.

AI Applications to Breast MRI: Today and Tomorrow

In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.

A Vision for Democratizing Next-Generation Oncology Clinical Trials

SUMMARY

Revolutionary advancements in oncology have transformed lives, but the clinical trials ecosystem encounters challenges, including restricted access to innovative therapies and a lack of diversity in participant representation. A vision emerges for democratized, globally accessible oncology trials, necessitating collaboration among researchers, clinicians, patients, and policymakers to shift from converting complex, exclusive trials into a dynamic, inclusive force against cancer.

Transforming the cardiometabolic disease landscape: Multimodal AI-powered approaches in prevention and management

The rise of artificial intelligence (AI) has revolutionized various scientific fields, particularly in medicine, where it has enabled the modeling of complex relationships from massive datasets. Initially, AI algorithms focused on improved interpretation of diagnostic studies such as chest X-rays and electrocardiograms in addition to predicting patient outcomes and future disease onset. However, AI has evolved with the introduction of transformer models, allowing analysis of the diverse, multimodal data sources existing in medicine today. Multimodal AI holds great promise in more accurate disease risk assessment and stratification as well as optimizing the key driving factors in cardiometabolic disease: blood pressure, sleep, stress, glucose control, weight, nutrition, and physical activity. In this article we outline the current state of medical AI in cardiometabolic disease, highlighting the potential of multimodal AI to augment personalized prevention and treatment strategies in cardiometabolic disease.

Current Status and Future Directions: The Application of Artificial Intelligence/Machine Learning for Precision Medicine

Technological innovations, such as artificial intelligence (AI) and machine learning (ML), have the potential to expedite the goal of precision medicine, especially when combined with increased capacity for voluminous data from multiple sources and expanded therapeutic modalities; however, they also present several challenges. In this communication, we first discuss the goals of precision medicine, and contextualize the use of AI in precision medicine by showcasing innovative applications (e.g., prediction of tumor growth and overall survival, biomarker identification using biomedical images, and identification of patient population for clinical practice) which were presented during the February 2023 virtual public workshop entitled "Application of Artificial Intelligence and Machine Learning for Precision Medicine," hosted by the US Food and Drug Administration (FDA) and University of Maryland Center of Excellence in Regulatory Science and Innovation (M-CERSI). Next, we put forward challenges brought about by the multidisciplinary nature of AI, particularly highlighting the need for AI to be trustworthy. To address such challenges, we subsequently note practical approaches, viz., differential privacy, synthetic data generation, and federated learning. The proposed strategies - some of which are highlighted presentations from the workshop - are for the protection of personal information and intellectual property. In addition, methods such as the risk-based management approach and the need for an agile regulatory ecosystem are discussed. Finally, we lay out a call for action that includes sharing of data and algorithms, development of regulatory guidance documents, and pooling of expertise from a broad-spectrum of stakeholders to enhance the application of AI in precision medicine.

Critical review of self-diagnosis of mental health conditions using artificial intelligence

The advent of artificial intelligence (AI) has revolutionised various aspects of our lives, including mental health nursing. AI-driven tools and applications have provided a convenient and accessible means for individuals to assess their mental well-being within the confines of their homes. Nonetheless, the widespread trend of self-diagnosing mental health conditions through AI poses considerable risks. This review article examines the perils associated with relying on AI for self-diagnosis in mental health, highlighting the constraints and possible adverse outcomes that can arise from such practices. It delves into the ethical, psychological, and social implications, underscoring the vital role of mental health professionals, including psychologists, psychiatrists, and nursing specialists, in providing professional assistance and guidance. This article aims to highlight the importance of seeking professional assistance and guidance in addressing mental health concerns, especially in the era of AI-driven self-diagnosis.

A Multimodal Omics Framework to Empower Target Discovery for Cardiovascular Regeneration

Ischaemic heart disease is a global healthcare challenge with high morbidity and mortality. Early revascularisation in acute myocardial infarction has improved survival; however, limited regenerative capacity and microvascular dysfunction often lead to impaired function and the development of heart failure. New mechanistic insights are required to identify robust targets for the development of novel strategies to promote regeneration. Single-cell RNA sequencing (scRNA-seq) has enabled profiling and analysis of the transcriptomes of individual cells at high resolution. Applications of scRNA-seq have generated single-cell atlases for multiple species, revealed distinct cellular compositions for different regions of the heart, and defined multiple mechanisms involved in myocardial injury-induced regeneration. In this review, we summarise findings from studies of healthy and injured hearts in multiple species and spanning different developmental stages. Based on this transformative technology, we propose a multi-species, multi-omics, meta-analysis framework to drive the discovery of new targets to promote cardiovascular regeneration.

The wedish initiative for the stdy of ary sclerosing cholangitis (SUPRIM)

Background

Despite more than 50 years of research and parallel improvements in hepatology and oncology, there is still today neither a treatment to prevent disease progression in primary sclerosing cholangitis (PSC), nor reliable early diagnostic tools for the associated hepatobiliary cancers. Importantly, the limited understanding of the underlying biological mechanisms in PSC and its natural history not only affects the identification of new drug targets but implies a lack of surrogate markers that hampers the design of clinical trials and the evaluation of drug efficacy. The lack of easy access to large representative well-characterised prospective resources is an important contributing factor to the current situation.

Methods

We here present the SUPRIM cohort, a national multicentre prospective longitudinal study of unselected PSC patients capturing the representative diversity of PSC phenotypes. We describe the 10-year effort of inclusion and follow-up, an intermediate analysis report including original results, and the associated research resource. All included patients gave written informed consent (recruitment: November 2011-April 2016).

Findings

Out of 512 included patients, 452 patients completed the five-year follow-up without endpoint outcomes. Liver transplantation was performed in 54 patients (10%) and hepatobiliary malignancy was diagnosed in 15 patients (3%). We draw a comprehensive landscape of the multidimensional clinical and biological heterogeneity of PSC illustrating the diversity of PSC phenotypes. Performances of available predictive scores are compared and perspectives on the continuation of the SUPRIM cohort are provided.

Interpretation

We envision the SUPRIM cohort as an open-access collaborative resource to accelerate the generation of new knowledge and independent validations of promising ones with the aim to uncover reliable diagnostics, prognostic tools, surrogate markers, and new treatment targets by 2040.

Funding

This work was supported by the Swedish Cancer Society, Stockholm County Council, and the Cancer Research Funds of Radiumhemmet.

Machine Learning-Based Quantification of Patient Factors Impacting Remission in Patients With Ulcerative Colitis: Insights from Etrolizumab Phase III Clinical Trials

Etrolizumab, an investigational anti-β7 integrin monoclonal antibody, has undergone evaluation for safety and efficacy in phase III clinical trials on patients with moderate to severe ulcerative colitis (UC). Etrolizumab was terminated because mixed efficacy results were shown in the induction and maintenance phase in patients with UC. In this post hoc analysis, we characterized the impact of explanatory variables on the probability of remission using XGBoost machine learning (ML) models alongside with the SHapley Additive exPlanations framework for explainability. We used patient-level data encompassing demographics, physiology, disease history, clinical questionnaires, histology, serum biomarkers, and etrolizumab drug exposure to develop ML models aimed at predicting remission. Baseline covariates and early etrolizumab exposure at week 4 in the induction phase were utilized to develop an induction ML model, whereas covariates from the end of the induction phase and early etrolizumab exposure at week 4 in the maintenance phase were used to develop a maintenance ML model. Both the induction and maintenance ML models exhibited good predictive performance, achieving an area under the receiver operating characteristic curve (AUROC) of 0.74 ± 0.03 and 0.75 ± 0.06 (mean ± SD), respectively. Compared with placebo, the highest tertile of etrolizumab exposure contributed to 15.0% (95% confidence interval (CI): 9.7-19.9) and 17.0% (95% CI: 8.1-26.4) increases in remission probability in the induction and maintenance phases, respectively. Additionally, the key covariates that predicted remission were CRP, MAdCAM-1, and stool frequency for the induction phase and white blood cells, fecal calprotectin and age for the maintenance phase. These findings hold significant implications for establishing stratification factors in the design of future clinical trials.

Artificial Intelligence for Quantitative Modeling in Drug Discovery and Development: An Innovation and Quality Consortium Perspective on Use Cases and Best Practices

Recent breakthroughs in artificial intelligence (AI) and machine learning (ML) have ushered in a new era of possibilities across various scientific domains. One area where these advancements hold significant promise is model-informed drug discovery and development (MID3). To foster a wider adoption and acceptance of these advanced algorithms, the Innovation and Quality (IQ) Consortium initiated the AI/ML working group in 2021 with the aim of promoting their acceptance among the broader scientific community as well as by regulatory agencies. By drawing insights from workshops organized by the working group and attended by key stakeholders across the biopharma industry, academia, and regulatory agencies, this white paper provides a perspective from the IQ Consortium. The range of applications covered in this white paper encompass the following thematic topics: (i) AI/ML-enabled Analytics for Pharmacometrics and Quantitative Systems Pharmacology (QSP) Workflows; (ii) Explainable Artificial Intelligence and its Applications in Disease Progression Modeling; (iii) Natural Language Processing (NLP) in Quantitative Pharmacology Modeling; and (iv) AI/ML Utilization in Drug Discovery. Additionally, the paper offers a set of best practices to ensure an effective and responsible use of AI, including considering the context of use, explainability and generalizability of models, and having human-in-the-loop. We believe that embracing the transformative power of AI in quantitative modeling while adopting a set of good practices can unlock new opportunities for innovation, increase efficiency, and ultimately bring benefits to patients.

Best practices for multimodal clinical data management and integration: An atopic dermatitis research case

BACKGROUND

In clinical research on multifactorial diseases such as atopic dermatitis, data-driven medical research has become more widely used as means to clarify diverse pathological conditions and to realize precision medicine. However, modern clinical data, characterized as large-scale, multimodal, and multi-center, causes difficulties in data integration and management, which limits productivity in clinical data science.

METHODS

We designed a generic data management flow to collect, cleanse, and integrate data to handle different types of data generated at multiple institutions by 10 types of clinical studies. We developed MeDIA (Medical Data Integration Assistant), a software to browse the data in an integrated manner and extract subsets for analysis.

RESULTS

MeDIA integrates and visualizes data and information on research participants obtained from multiple studies. It then provides a sophisticated interface that supports data management and helps data scientists retrieve the data sets they need. Furthermore, the system promotes the use of unified terms such as identifiers or sampling dates to reduce the cost of pre-processing by data analysts. We also propose best practices in clinical data management flow, which we learned from the development and implementation of MeDIA.

CONCLUSIONS

The MeDIA system solves the problem of multimodal clinical data integration, from complex text data such as medical records to big data such as omics data from a large number of patients. The system and the proposed best practices can be applied not only to allergic diseases but also to other diseases to promote data-driven medical research.

Computational model for drug research

This special issue focuses on computational model for drug research regarding drug bioactivity prediction, drug-related interaction prediction, modelling for immunotherapy and modelling for treatment of a specific disease, as conveyed by the following six research and four review articles. Notably, these 10 papers described a wide variety of in-depth drug research from the computational perspective and may represent a snapshot of the wide research landscape.

Utilizing multimodal AI to improve genetic analyses of cardiovascular traits

Electronic health records, biobanks, and wearable biosensors contain multiple high-dimensional clinical data (HDCD) modalities (e.g., ECG, Photoplethysmography (PPG), and MRI) for each individual. Access to multimodal HDCD provides a unique opportunity for genetic studies of complex traits because different modalities relevant to a single physiological system (e.g., circulatory system) encode complementary and overlapping information. We propose a novel multimodal deep learning method, M-REGLE, for discovering genetic associations from a joint representation of multiple complementary HDCD modalities. We showcase the effectiveness of this model by applying it to several cardiovascular modalities. M-REGLE jointly learns a lower representation (i.e., latent factors) of multimodal HDCD using a convolutional variational autoencoder, performs genome wide association studies (GWAS) on each latent factor, then combines the results to study the genetics of the underlying system. To validate the advantages of M-REGLE and multimodal learning, we apply it to common cardiovascular modalities (PPG and ECG), and compare its results to unimodal learning methods in which representations are learned from each data modality separately, but the downstream genetic analyses are performed on the combined unimodal representations. M-REGLE identifies 19.3% more loci on the 12-lead ECG dataset, 13.0% more loci on the ECG lead I + PPG dataset, and its genetic risk score significantly outperforms the unimodal risk score at predicting cardiac phenotypes, such as atrial fibrillation (Afib), in multiple biobanks.

Investigation of deep learning model for predicting immune checkpoint inhibitor treatment efficacy on contrast-enhanced computed tomography images of hepatocellular carcinoma

Although the use of immune checkpoint inhibitors (ICIs)-targeted agents for unresectable hepatocellular carcinoma (HCC) is promising, individual response variability exists. Therefore, we developed an artificial intelligence (AI)-based model to predict treatment efficacy using pre-ICIs contrast-enhanced computed tomography (CT) imaging characteristics. We evaluated the efficacy of atezolizumab and bevacizumab in 43 patients at the Nagasaki University Hospital from 2020 to 2022 using the modified Response Evaluation Criteria in Solid Tumors. A total of 197 Progressive Disease (PD), 271 Partial Response (PR), and 342 Stable Disease (SD) contrast CT images of HCC were used for training. We used ResNet-18 as the Convolutional Neural Network (CNN) model and YOLOv5, YOLOv7, YOLOv8 as the You Only Look Once (YOLO) model with precision-recall curves and class activation maps (CAMs) for diagnostic performance evaluation and model interpretation, respectively. The 3D t-distributed Stochastic Neighbor Embedding was used for image feature analysis. The YOLOv7 model demonstrated Precision 53.7%, Recall 100%, F1 score 69.8%, mAP@0.5 99.5% for PD, providing accurate and clinically versatile predictions by identifying decisive points. The ResNet-18 model had Precision 100% and Recall 100% for PD. However, the CAMs sites did not align with the tumors, suggesting the CNN model is not predicting that a given CT slice is PD, PR, or SD, but that it accurately predicts Individual Patient's CT slices. Preparing substantial training data for tumor drug effect prediction models is challenging compared to general tumor diagnosis models; hence, large-scale validation using an efficient YOLO model is warranted.

Wireless Battery-free and Fully Implantable Organ Interfaces

Advances in soft materials, miniaturized electronics, sensors, stimulators, radios, and battery-free power supplies are resulting in a new generation of fully implantable organ interfaces that leverage volumetric reduction and soft mechanics by eliminating electrochemical power storage. This device class offers the ability to provide high-fidelity readouts of physiological processes, enables stimulation, and allows control over organs to realize new therapeutic and diagnostic paradigms. Driven by seamless integration with connected infrastructure, these devices enable personalized digital medicine. Key to advances are carefully designed material, electrophysical, electrochemical, and electromagnetic systems that form implantables with mechanical properties closely matched to the target organ to deliver functionality that supports high-fidelity sensors and stimulators. The elimination of electrochemical power supplies enables control over device operation, anywhere from acute, to lifetimes matching the target subject with physical dimensions that supports imperceptible operation. This review provides a comprehensive overview of the basic building blocks of battery-free organ interfaces and related topics such as implantation, delivery, sterilization, and user acceptance. State of the art examples categorized by organ system and an outlook of interconnection and advanced strategies for computation leveraging the consistent power influx to elevate functionality of this device class over current battery-powered strategies is highlighted.

Digital Microscopy Augmented by Artificial Intelligence to Interpret Bone Marrow Samples for Hematological Diseases

Analysis of bone marrow aspirates (BMAs) is an essential step in the diagnosis of hematological disorders. This analysis is usually performed based on a visual examination of samples under a conventional optical microscope, which involves a labor-intensive process, limited by clinical experience and subject to high observer variability. In this work, we present a comprehensive digital microscopy system that enables BMA analysis for cell type counting and differentiation in an efficient and objective manner. This system not only provides an accessible and simple method to digitize, store, and analyze BMA samples remotely but is also supported by an Artificial Intelligence (AI) pipeline that accelerates the differential cell counting process and reduces interobserver variability. It has been designed to integrate AI algorithms with the daily clinical routine and can be used in any regular hospital workflow.

Data science opportunities of large language models for neuroscience and biomedicine

Large language models (LLMs) are a new asset class in the machine-learning landscape. Here we offer a primer on defining properties of these modeling techniques. We then reflect on new modes of investigation in which LLMs can be used to reframe classic neuroscience questions to deliver fresh answers. We reason that LLMs have the potential to (1) enrich neuroscience datasets by adding valuable meta-information, such as advanced text sentiment, (2) summarize vast information sources to overcome divides between siloed neuroscience communities, (3) enable previously unthinkable fusion of disparate information sources relevant to the brain, (4) help deconvolve which cognitive concepts most usefully grasp phenomena in the brain, and much more.

Neuroscience meets behavior: A systematic literature review on magnetic resonance imaging of the brain combined with real-world digital phenotyping

A primary goal of neuroscience is to understand the relationship between the brain and behavior. While magnetic resonance imaging (MRI) examines brain structure and function under controlled conditions, digital phenotyping via portable automatic devices (PAD) quantifies behavior in real-world settings. Combining these two technologies may bridge the gap between brain imaging, physiology, and real-time behavior, enhancing the generalizability of laboratory and clinical findings. However, the use of MRI and data from PADs outside the MRI scanner remains underexplored. Herein, we present a Preferred Reporting Items for Systematic Reviews and Meta-Analysis systematic literature review that identifies and analyzes the current state of research on the integration of brain MRI and PADs. PubMed and Scopus were automatically searched using keywords covering various MRI techniques and PADs. Abstracts were screened to only include articles that collected MRI brain data and PAD data outside the laboratory environment. Full-text screening was then conducted to ensure included articles combined quantitative data from MRI with data from PADs, yielding 94 selected papers for a total of N = 14,778 subjects. Results were reported as cross-frequency tables between brain imaging and behavior sampling methods and patterns were identified through network analysis. Furthermore, brain maps reported in the studies were synthesized according to the measurement modalities that were used. Results demonstrate the feasibility of integrating MRI and PADs across various study designs, patient and control populations, and age groups. The majority of published literature combines functional, T1-weighted, and diffusion weighted MRI with physical activity sensors, ecological momentary assessment via PADs, and sleep. The literature further highlights specific brain regions frequently correlated with distinct MRI-PAD combinations. These combinations enable in-depth studies on how physiology, brain function and behavior influence each other. Our review highlights the potential for constructing brain-behavior models that extend beyond the scanner and into real-world contexts.

Expanding from unilateral to bilateral: A robust deep learning-based approach for predicting radiographic osteoarthritis progression

OBJECTIVE

To develop and validate a deep learning (DL) model for predicting osteoarthritis (OA) progression based on bilateral knee joint views.

METHODS

In this retrospective study, knee joints from bilateral posteroanterior knee radiographs of participants in the Osteoarthritis Initiative were analyzed. At baseline, participants were divided into testing set 1 and development set according to the different enrolled sites. The development set was further divided into a training set and a validation set in an 8:2 ratio for model development. At 48-month follow-up, eligible patients were formed testing set 2. The Bilateral Knee Neural Network (BikNet) was developed using bilateral views, with the knee to be predicted as the main view and the contralateral knee as the auxiliary view. DenseNet and ResNext were also trained and compared as the unilateral model. Two reader tests were conducted to evaluate the model's value in predicting incident OA.

RESULTS

Totally 3583 participants were evaluated. The BikNet we proposed outperformed ResNext and DenseNet (all area under the curve [AUC] < 0.71, P < 0.001) with AUC values of 0.761 and 0.745 in testing sets 1 and 2, respectively. With assistance of the BikNet increased clinicians' sensitivity (from 28.1-63.2% to 42.1-68.4%) and specificity (from 57.4-83.4% to 64.1-87.5%) of incident OA prediction and improved inter-observer reliability.

CONCLUSION

The DL model, constructed based on bilateral knee views, holds promise for enhancing the assessment of OA and demonstrates greater robustness during subsequent follow-up evaluations as compared with unilateral models. BikNet represents a potential tool or imaging biomarker for predicting OA progression.

Towards Automation in Radiotherapy Planning: A Deep Learning Approach for the Delineation of Parotid Glands in Head and Neck Cancer

The delineation of parotid glands in head and neck (HN) carcinoma is critical to assess radiotherapy (RT) planning. Segmentation processes ensure precise target position and treatment precision, facilitate monitoring of anatomical changes, enable plan adaptation, and enhance overall patient safety. In this context, artificial intelligence (AI) and deep learning (DL) have proven exceedingly effective in precisely outlining tumor tissues and, by extension, the organs at risk. This paper introduces a DL framework using the AttentionUNet neural network for automatic parotid gland segmentation in HN cancer. Extensive evaluation of the model is performed in two public and one private dataset, while segmentation accuracy is compared with other state-of-the-art DL segmentation schemas. To assess replanning necessity during treatment, an additional registration method is implemented on the segmentation output, aligning images of different modalities (Computed Tomography (CT) and Cone Beam CT (CBCT)). AttentionUNet outperforms similar DL methods (Dice Similarity Coefficient: 82.65% ± 1.03, Hausdorff Distance: 6.24 mm ± 2.47), confirming its effectiveness. Moreover, the subsequent registration procedure displays increased similarity, providing insights into the effects of RT procedures for treatment planning adaptations. The implementation of the proposed methods indicates the effectiveness of DL not only for automatic delineation of the anatomical structures, but also for the provision of information for adaptive RT support.

Building digital patient pathways for the management and treatment of multiple sclerosis

Recent advances in the field of artificial intelligence (AI) could yield new insights into the potential causes of multiple sclerosis (MS) and factors influencing its course as the use of AI opens new possibilities regarding the interpretation and use of big data from not only a cross-sectional, but also a longitudinal perspective. For each patient with MS, there is a vast amount of multimodal data being accumulated over time. But for the application of AI and related technologies, these data need to be available in a machine-readable format and need to be collected in a standardized and structured manner. Through the use of mobile electronic devices and the internet it has also become possible to provide healthcare services from remote and collect information on a patient's state of health outside of regular check-ups on site. Against this background, we argue that the concept of pathways in healthcare now could be applied to structure the collection of information across multiple devices and stakeholders in the virtual sphere, enabling us to exploit the full potential of AI technology by e.g., building digital twins. By going digital and using pathways, we can virtually link patients and their caregivers. Stakeholders then could rely on digital pathways for evidence-based guidance in the sequence of procedures and selection of therapy options based on advanced analytics supported by AI as well as for communication and education purposes. As far as we aware of, however, pathway modelling with respect to MS management and treatment has not been thoroughly investigated yet and still needs to be discussed. In this paper, we thus present our ideas for a modular-integrative framework for the development of digital patient pathways for MS treatment.

Machine Learning and Deep Learning Applications in Magnetic Particle Imaging

In recent years, magnetic particle imaging (MPI) has emerged as a promising imaging technique depicting high sensitivity and spatial resolution. It originated in the early 2000s where it proposed a new approach to challenge the low spatial resolution achieved by using relaxometry in order to measure the magnetic fields. MPI presents 2D and 3D images with high temporal resolution, non-ionizing radiation, and optimal visual contrast due to its lack of background tissue signal. Traditionally, the images were reconstructed by the conversion of signal from the induced voltage by generating system matrix and X-space based methods. Because image reconstruction and analyses play an integral role in obtaining precise information from MPI signals, newer artificial intelligence-based methods are continuously being researched and developed upon. In this work, we summarize and review the significance and employment of machine learning and deep learning models for applications with MPI and the potential they hold for the future. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 1.

Multimodal machine learning models identify chemotherapy drugs with prospective clinical efficacy in dogs with relapsed B-cell lymphoma

Dogs with B-cell lymphoma typically respond well to first-line CHOP-based chemotherapy, but there is no standard of care for relapsed patients. To help veterinary oncologists select effective drugs for dogs with lymphoid malignancies such as B-cell lymphoma, we have developed multimodal machine learning models that integrate data from multiple tumor profiling modalities and predict the likelihood of a positive clinical response for 10 commonly used chemotherapy drugs. Here we report on clinical outcomes that occurred after oncologists received a prediction report generated by our models. Remarkably, we found that dogs that received drugs predicted to be effective by the models experienced better clinical outcomes by every metric we analyzed (overall response rate, complete response rate, duration of complete response, patient survival times) relative to other dogs in the study and relative to historical controls.

Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview

The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.

A physicochemical-sensing electronic skin for stress response monitoring

Approaches to quantify stress responses typically rely on subjective surveys and questionnaires. Wearable sensors can potentially be used to continuously monitor stress-relevant biomarkers. However, the biological stress response is spread across the nervous, endocrine, and immune systems, and the capabilities of current sensors are not sufficient for condition-specific stress response evaluation. Here we report an electronic skin for stress response assessment that non-invasively monitors three vital signs (pulse waveform, galvanic skin response and skin temperature) and six molecular biomarkers in human sweat (glucose, lactate, uric acid, sodium ions, potassium ions and ammonium). We develop a general approach to prepare electrochemical sensors that relies on analogous composite materials for stabilizing and conserving sensor interfaces. The resulting sensors offer long-term sweat biomarker analysis of over 100 hours with high stability. We show that the electronic skin can provide continuous multimodal physicochemical monitoring over a 24-hour period and during different daily activities. With the help of a machine learning pipeline, we also show that the platform can differentiate three stressors with an accuracy of 98.0%, and quantify psychological stress responses with a confidence level of 98.7%.

Mechanical Ventilation, Past, Present, and Future

Mechanical ventilation (MV) has played a crucial role in the medical field, particularly in anesthesia and in critical care medicine (CCM) settings. MV has evolved significantly since its inception over 70 years ago and the future promises even more advanced technology. In the past, ventilation was provided manually, intermittently, and it was primarily used for resuscitation or as a last resort for patients with severe respiratory or cardiovascular failure. The earliest MV machines for prolonged ventilatory support and oxygenation were large and cumbersome. They required a significant amount of skills and expertise to operate. These early devices had limited capabilities, battery, power, safety features, alarms, and therefore these often caused harm to patients. Moreover, the physiology of MV was modified when mechanical ventilators moved from negative pressure to positive pressure mechanisms. Monitoring systems were also very limited and therefore the risks related to MV support were difficult to quantify, predict and timely detect for individual patients who were necessarily young with few comorbidities. Technology and devices designed to use tracheostomies versus endotracheal intubation evolved in the last century too and these are currently much more reliable. In the present, positive pressure MV is more sophisticated and widely used for extensive period of time. Modern ventilators use mostly positive pressure systems and are much smaller, more portable than their predecessors, and they are much easier to operate. They can also be programmed to provide different levels of support based on evolving physiological concepts allowing lung-protective ventilation. Monitoring systems are more sophisticated and knowledge related to the physiology of MV is improved. Patients are also more complex and elderly compared to the past. MV experts are informed about risks related to prolonged or aggressive ventilation modalities and settings. One of the most significant advances in MV has been protective lung ventilation, diaphragm protective ventilation including noninvasive ventilation (NIV). Health care professionals are familiar with the use of MV and in many countries, respiratory therapists have been trained for the exclusive purpose of providing safe and professional respiratory support to critically ill patients. Analgo-sedation drugs and techniques are improved, and more sedative drugs are available and this has an impact on recovery, weaning, and overall patients' outcome. Looking toward the future, MV is likely to continue to evolve and improve alongside monitoring techniques and sedatives. There is increasing precision in monitoring global "patient-ventilator" interactions: structure and analysis (asynchrony, desynchrony, etc). One area of development is the use of artificial intelligence (AI) in ventilator technology. AI can be used to monitor patients in real-time, and it can predict when a patient is likely to experience respiratory distress. This allows medical professionals to intervene before a crisis occurs, improving patient outcomes and reducing the need for emergency intervention. This specific area of development is intended as "personalized ventilation." It involves tailoring the ventilator settings to the individual patient, based on their physiology and the specific condition they are being treated for. This approach has the potential to improve patient outcomes by optimizing ventilation and reducing the risk of harm. In conclusion, MV has come a long way since its inception, and it continues to play a critical role in anesthesia and in CCM settings. Advances in technology have made MV safer, more effective, affordable, and more widely available. As technology continues to improve, more advanced and personalized MV will become available, leading to better patients' outcomes and quality of life for those in need.

Machine Learning Approaches to Predict Asthma Exacerbations: A Narrative Review

The implementation of artificial intelligence (AI) and machine learning (ML) techniques in healthcare has garnered significant attention in recent years, especially as a result of their potential to revolutionize personalized medicine. Despite advances in the treatment and management of asthma, a significant proportion of patients continue to suffer acute exacerbations, irrespective of disease severity and therapeutic regimen. The situation is further complicated by the constellation of factors that influence disease activity in a patient with asthma, such as medical history, biomarker phenotype, pulmonary function, level of healthcare access, treatment compliance, comorbidities, personal habits, and environmental conditions. A growing body of work has demonstrated the potential for AI and ML to accurately predict asthma exacerbations while also capturing the entirety of the patient experience. However, application in the clinical setting remains mostly unexplored, and important questions on the strengths and limitations of this technology remain. This review presents an overview of the rapidly evolving landscape of AI and ML integration into asthma management by providing a snapshot of the existing scientific evidence and proposing potential avenues for future applications.

Proactive Decision Support for Glaucoma Treatment: Predicting Surgical Interventions with Clinically Available Data

A longitudinal ophthalmic dataset was used to investigate multi-modal machine learning (ML) models incorporating patient demographics and history, clinical measurements, optical coherence tomography (OCT), and visual field (VF) testing in predicting glaucoma surgical interventions. The cohort included 369 patients who underwent glaucoma surgery and 592 patients who did not undergo surgery. The data types used for prediction included patient demographics, history of systemic conditions, medication history, ophthalmic measurements, 24-2 VF results, and thickness measurements from OCT imaging. The ML models were trained to predict surgical interventions and evaluated on independent data collected at a separate study site. The models were evaluated based on their ability to predict surgeries at varying lengths of time prior to surgical intervention. The highest performing predictions achieved an AUC of 0.93, 0.92, and 0.93 in predicting surgical intervention at 1 year, 2 years, and 3 years, respectively. The models were also able to achieve high sensitivity (0.89, 0.77, 0.86 at 1, 2, and 3 years, respectively) and specificity (0.85, 0.90, and 0.91 at 1, 2, and 3 years, respectively) at an 0.80 level of precision. The multi-modal models trained on a combination of data types predicted surgical interventions with high accuracy up to three years prior to surgery and could provide an important tool to predict the need for glaucoma intervention.

Health Digital Twins in Life Science and Health Care Innovation

Health digital twins (HDTs) are virtual representations of real individuals that can be used to simulate human physiology, disease, and drug effects. HDTs can be used to improve drug discovery and development by providing a data-driven approach to inform target selection, drug delivery, and design of clinical trials. HDTs also offer new applications into precision therapies and clinical decision making. The deployment of HDTs at scale could bring a precision approach to public health monitoring and intervention. Next steps include challenges such as addressing socioeconomic barriers and ensuring the representativeness of the technology based on the training and validation data sets. Governance and regulation of HDT technology are still in the early stages.

Topical drug delivery strategies for enhancing drug effectiveness by skin barriers, drug delivery systems and individualized dosing

Topical drug delivery is widely used in various diseases because of the advantages of not passing through the gastrointestinal tract, avoiding gastrointestinal irritation and hepatic first-pass effect, and reaching the lesion directly to reduce unnecessary adverse reactions. The skin helps the organism to defend itself against a huge majority of external aggressions and is one of the most important lines of defense of the body. However, the skin's strong barrier ability is also a huge obstacle to the effectiveness of topical medications. Allowing the bioactive, composition in a drug to pass through the stratum corneum barrier as needed to reach the target site is the most essential need for the bioactive, composition to exert its therapeutic effect. The state of the skin barrier, the choice of delivery system for the bioactive, composition, and individualized disease detection and dosing planning influence the effectiveness of topical medications. Nowadays, enhancing transdermal absorption of topically applied drugs is the hottest research area. However, enhancing transdermal absorption of drugs is not the first choice to improve the effectiveness of all drugs. Excessive transdermal absorption enhances topical drug accumulation at non-target sites and the occurrence of adverse reactions. This paper introduces topical drug delivery strategies to improve drug effectiveness from three perspectives: skin barrier, drug delivery system and individualized drug delivery, describes the current status and shortcomings of topical drug research, and provides new directions and ideas for topical drug research.

Assessing the risk of prenatal depressive symptoms in Chinese women: an integrated evaluation of serum metabolome, multivitamin supplement intake, and clinical blood indicators

Background

Prenatal depressive symptoms (PDS) is a serious public health problem. This study aimed to develop an integrated panel and nomogram to assess at-risk populations by examining the association of PDS with the serum metabolome, multivitamin supplement intake, and clinical blood indicators.

Methods

This study comprised 221 pregnant women, categorized into PDS and non-PDS groups based on the Edinburgh postnatal depression scale. The participants were divided into training and test sets according to their enrollment time. We conducted logistic regression analysis to identify risk factors, and employed liquid chromatography/high resolution mass spectrometry-based serum metabolome analysis to identify metabolic biomarkers. Multiple factor analysis was used to combine risk factors, clinical blood indicators and key metabolites, and then a nomogram was developed to estimate the probability of PDS.

Results

We identified 36 important differential serum metabolites as PDS biomarkers, mainly involved in amino acid metabolism and lipid metabolism. Multivitamin intake works as a protective factor for PDS. The nomogram model, including multivitamin intake, HDL-C and three key metabolites (histidine, estrone and valylasparagine), exhibited an AUC of 0.855 in the training set and 0.774 in the test set, and the calibration curves showed good agreement, indicating that the model had good stability.

Conclusion

Our approach integrates multiple models to identify metabolic biomarkers for PDS, ensuring their robustness. Furthermore, the inclusion of dietary factors and clinical blood indicators allows for a comprehensive characterization of each participant. The analysis culminated in an intuitive nomogram based on multimodal data, displaying potential performance in initial PDS risk assessment.

Artificial Intelligence in the Differential Diagnosis of Cardiomyopathy Phenotypes

Artificial intelligence (AI) is rapidly being applied to the medical field, especially in the cardiovascular domain. AI approaches have demonstrated their applicability in the detection, diagnosis, and management of several cardiovascular diseases, enhancing disease stratification and typing. Cardiomyopathies are a leading cause of heart failure and life-threatening ventricular arrhythmias. Identifying the etiologies is fundamental for the management and diagnostic pathway of these heart muscle diseases, requiring the integration of various data, including personal and family history, clinical examination, electrocardiography, and laboratory investigations, as well as multimodality imaging, making the clinical diagnosis challenging. In this scenario, AI has demonstrated its capability to capture subtle connections from a multitude of multiparametric datasets, enabling the discovery of hidden relationships in data and handling more complex tasks than traditional methods. This review aims to present a comprehensive overview of the main concepts related to AI and its subset. Additionally, we review the existing literature on AI-based models in the differential diagnosis of cardiomyopathy phenotypes, and we finally examine the advantages and limitations of these AI approaches.

Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders

BACKGROUND

The integration of the information encoded in multiparametric MRI images can enhance the performance of machine-learning classifiers. In this study, we investigate whether the combination of structural and functional MRI might improve the performances of a deep learning (DL) model trained to discriminate subjects with Autism Spectrum Disorders (ASD) with respect to typically developing controls (TD).

MATERIAL AND METHODS

We analyzed both structural and functional MRI brain scans publicly available within the ABIDE I and II data collections. We considered 1383 male subjects with age between 5 and 40 years, including 680 subjects with ASD and 703 TD from 35 different acquisition sites. We extracted morphometric and functional brain features from MRI scans with the Freesurfer and the CPAC analysis packages, respectively. Then, due to the multisite nature of the dataset, we implemented a data harmonization protocol. The ASD vs. TD classification was carried out with a multiple-input DL model, consisting in a neural network which generates a fixed-length feature representation of the data of each modality (FR-NN), and a Dense Neural Network for classification (C-NN). Specifically, we implemented a joint fusion approach to multiple source data integration. The main advantage of the latter is that the loss is propagated back to the FR-NN during the training, thus creating informative feature representations for each data modality. Then, a C-NN, with a number of layers and neurons per layer to be optimized during the model training, performs the ASD-TD discrimination. The performance was evaluated by computing the Area under the Receiver Operating Characteristic curve within a nested 10-fold cross-validation. The brain features that drive the DL classification were identified by the SHAP explainability framework.

RESULTS

The AUC values of 0.66±0.05 and of 0.76±0.04 were obtained in the ASD vs. TD discrimination when only structural or functional features are considered, respectively. The joint fusion approach led to an AUC of 0.78±0.04. The set of structural and functional connectivity features identified as the most important for the two-class discrimination supports the idea that brain changes tend to occur in individuals with ASD in regions belonging to the Default Mode Network and to the Social Brain.

CONCLUSIONS

Our results demonstrate that the multimodal joint fusion approach outperforms the classification results obtained with data acquired by a single MRI modality as it efficiently exploits the complementarity of structural and functional brain information.

Developing Cardiac Digital Twins at Scale: Insights from Personalised Myocardial Conduction Velocity

Large-cohort studies using cardiovascular imaging and diagnostic datasets have assessed cardiac anatomy, function, and outcomes, but typically do not reveal underlying biological mechanisms. Cardiac digital twins (CDTs) provide personalized physics- and physiology-constrained in-silico representations, enabling inference of multi-scale properties tied to these mechanisms. We constructed 3464 anatomically-accurate CDTs using cardiac magnetic resonance images from UK biobank and personalised their myocardial conduction velocities (CVs) from electrocardiograms (ECG), through an automated framework. We found well-known sex-specific differences in QRS duration were fully explained by myocardial anatomy, as CV remained consistent across sexes. Conversely, significant associations of CV with ageing and increased BMI suggest myocardial tissue remodelling. Novel associations were observed with left ventricular ejection fraction and mental-health phenotypes, through a phenome-wide association study, and CV was also linked with adverse clinical outcomes. Our study highlights the utility of population-based CDTs in assessing intersubject variability and uncovering strong links with mental health.

Genetics of human brain development

Brain development in humans is achieved through precise spatiotemporal genetic control, the mechanisms of which remain largely elusive. Recently, integration of technological advances in human stem cell-based modelling with genome editing has emerged as a powerful platform to establish causative links between genotypes and phenotypes directly in the human system. Here, we review our current knowledge of complex genetic regulation of each key step of human brain development through the lens of evolutionary specialization and neurodevelopmental disorders and highlight the use of human stem cell-derived 2D cultures and 3D brain organoids to investigate human-enriched features and disease mechanisms. We also discuss opportunities and challenges of integrating new technologies to reveal the genetic architecture of human brain development and disorders.

Mapping medical image-text to a joint space via masked modeling

Recently, masked autoencoders have demonstrated their feasibility in extracting effective image and text features (e.g., BERT for natural language processing (NLP) and MAE in computer vision (CV)). This study investigates the potential of applying these techniques to vision-and-language representation learning in the medical domain. To this end, we introduce a self-supervised learning paradigm, multi-modal masked autoencoders (M3AE). It learns to map medical images and texts to a joint space by reconstructing pixels and tokens from randomly masked images and texts. Specifically, we design this approach from three aspects: First, taking into account the varying information densities of vision and language, we employ distinct masking ratios for input images and text, with a notably higher masking ratio for images; Second, we utilize visual and textual features from different layers for reconstruction to address varying levels of abstraction in vision and language; Third, we develop different designs for vision and language decoders. We establish a medical vision-and-language benchmark to conduct an extensive evaluation. Our experimental results exhibit the effectiveness of the proposed method, achieving state-of-the-art results on all downstream tasks. Further analyses validate the effectiveness of the various components and discuss the limitations of the proposed approach. The source code is available at https://github.com/zhjohnchan/M3AE.

Artificial intelligence for predictive biomarker discovery in immuno-oncology: a systematic review

BACKGROUND

The widespread use of immune checkpoint inhibitors (ICIs) has revolutionised treatment of multiple cancer types. However, selecting patients who may benefit from ICI remains challenging. Artificial intelligence (AI) approaches allow exploitation of high-dimension oncological data in research and development of precision immuno-oncology.

MATERIALS AND METHODS

We conducted a systematic literature review of peer-reviewed original articles studying the ICI efficacy prediction in cancer patients across five data modalities: genomics (including genomics, transcriptomics, and epigenomics), radiomics, digital pathology (pathomics), and real-world and multimodality data.

RESULTS

A total of 90 studies were included in this systematic review, with 80% published in 2021-2022. Among them, 37 studies included genomic, 20 radiomic, 8 pathomic, 20 real-world, and 5 multimodal data. Standard machine learning (ML) methods were used in 72% of studies, deep learning (DL) methods in 22%, and both in 6%. The most frequently studied cancer type was non-small-cell lung cancer (36%), followed by melanoma (16%), while 25% included pan-cancer studies. No prospective study design incorporated AI-based methodologies from the outset; rather, all implemented AI as a post hoc analysis. Novel biomarkers for ICI in radiomics and pathomics were identified using AI approaches, and molecular biomarkers have expanded past genomics into transcriptomics and epigenomics. Finally, complex algorithms and new types of AI-based markers, such as meta-biomarkers, are emerging by integrating multimodal/multi-omics data.

CONCLUSION

AI-based methods have expanded the horizon for biomarker discovery, demonstrating the power of integrating multimodal data from existing datasets to discover new meta-biomarkers. While most of the included studies showed promise for AI-based prediction of benefit from immunotherapy, none provided high-level evidence for immediate practice change. A priori planned prospective trial designs are needed to cover all lifecycle steps of these software biomarkers, from development and validation to integration into clinical practice.

Optimizing artificial intelligence in sepsis management: Opportunities in the present and looking closely to the future

Sepsis remains a major challenge internationally for healthcare systems. Its incidence is rising due to poor public awareness and delays in its recognition and subsequent management. In sepsis, mortality increases with every hour left untreated. Artificial intelligence (AI) is transforming worldwide healthcare delivery at present. This review has outlined how AI can augment strategies to address this global disease burden. AI and machine learning (ML) algorithms can analyze vast quantities of increasingly complex clinical datasets from electronic medical records to assist clinicians in diagnosing and treating sepsis earlier than traditional methods. Our review highlights how these models can predict the risk of sepsis and organ failure even before it occurs. This gives providers additional time to plan and execute treatment plans, thereby avoiding increasing complications associated with delayed diagnosis of sepsis. The potential for cost savings with AI implementation is also discussed, including improving workflow efficiencies, reducing administrative costs, and improving healthcare outcomes. Despite these advantages, clinicians have been slow to adopt AI into clinical practice. Some of the limitations posed by AI solutions include the lack of diverse data sets for model building so that they are widely applicable for routine clinical use. Furthermore, the subsequent algorithms are often based on complex mathematics leading to clinician hesitancy to embrace such technologies. Finally, we highlight the need for robust political and regulatory frameworks in this area to achieve the trust and approval of clinicians and patients to implement this transformational technology.

Longitudinal machine learning uncouples healthy aging factors from chronic disease risks

To understand human longevity, inherent aging processes must be distinguished from known etiologies leading to age-related chronic diseases. Such deconvolution is difficult to achieve because it requires tracking patients throughout their entire lives. Here, we used machine learning to infer health trajectories over the entire adulthood age range using extrapolation from electronic medical records with partial longitudinal coverage. Using this approach, our model tracked the state of patients who were healthy and free from known chronic disease risk and distinguished individuals with higher or lower longevity potential using a multivariate score. We showed that the model and the markers it uses performed consistently on data from Israeli, British and US populations. For example, mildly low neutrophil counts and alkaline phosphatase levels serve as early indicators of healthy aging that are independent of risk for major chronic diseases. We characterize the heritability and genetic associations of our longevity score and demonstrate at least 1 year of extended lifespan for parents of high-scoring patients compared to matched controls. Longitudinal modeling of healthy individuals is thereby established as a tool for understanding healthy aging and longevity.