Multimodal machine learning in precision health: A scoping review

The Emergence of AI-Based Wearable Sensors for Digital Health Technology: A Review

Disease diagnosis and monitoring using conventional healthcare services is typically expensive and has limited accuracy. Wearable health technology based on flexible electronics has gained tremendous attention in recent years for monitoring patient health owing to attractive features, such as lower medical costs, quick access to patient health data, ability to operate and transmit data in harsh environments, storage at room temperature, non-invasive implementation, mass scaling, etc. This technology provides an opportunity for disease pre-diagnosis and immediate therapy. Wearable sensors have opened a new area of personalized health monitoring by accurately measuring physical states and biochemical signals. Despite the progress to date in the development of wearable sensors, there are still several limitations in the accuracy of the data collected, precise disease diagnosis, and early treatment. This necessitates advances in applied materials and structures and using artificial intelligence (AI)-enabled wearable sensors to extract target signals for accurate clinical decision-making and efficient medical care. In this paper, we review two significant aspects of smart wearable sensors. First, we offer an overview of the most recent progress in improving wearable sensor performance for physical, chemical, and biosensors, focusing on materials, structural configurations, and transduction mechanisms. Next, we review the use of AI technology in combination with wearable technology for big data processing, self-learning, power-efficiency, real-time data acquisition and processing, and personalized health for an intelligent sensing platform. Finally, we present the challenges and future opportunities associated with smart wearable sensors.

Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

A scoping review of artificial intelligence-based methods for diabetes risk prediction

The increasing prevalence of type 2 diabetes mellitus (T2DM) and its associated health complications highlight the need to develop predictive models for early diagnosis and intervention. While many artificial intelligence (AI) models for T2DM risk prediction have emerged, a comprehensive review of their advancements and challenges is currently lacking. This scoping review maps out the existing literature on AI-based models for T2DM prediction, adhering to the PRISMA extension for Scoping Reviews guidelines. A systematic search of longitudinal studies was conducted across four databases, including PubMed, Scopus, IEEE-Xplore, and Google Scholar. Forty studies that met our inclusion criteria were reviewed. Classical machine learning (ML) models dominated these studies, with electronic health records (EHR) being the predominant data modality, followed by multi-omics, while medical imaging was the least utilized. Most studies employed unimodal AI models, with only ten adopting multimodal approaches. Both unimodal and multimodal models showed promising results, with the latter being superior. Almost all studies performed internal validation, but only five conducted external validation. Most studies utilized the area under the curve (AUC) for discrimination measures. Notably, only five studies provided insights into the calibration of their models. Half of the studies used interpretability methods to identify key risk predictors revealed by their models. Although a minority highlighted novel risk predictors, the majority reported commonly known ones. Our review provides valuable insights into the current state and limitations of AI-based models for T2DM prediction and highlights the challenges associated with their development and clinical integration.

Advancing Diabetic Retinopathy Diagnosis: Leveraging Optical Coherence Tomography Imaging with Convolutional Neural Networks

Diabetic retinopathy (DR) is a vision-threatening complication of diabetes, necessitating early and accurate diagnosis. The combination of optical coherence tomography (OCT) imaging with convolutional neural networks (CNNs) has emerged as a promising approach for enhancing DR diagnosis. OCT provides detailed retinal morphology information, while CNNs analyze OCT images for automated detection and classification of DR. This paper reviews the current research on OCT imaging and CNNs for DR diagnosis, discussing their technical aspects and suitability. It explores CNN applications in detecting lesions, segmenting microaneurysms, and assessing disease severity, showing high sensitivity and accuracy. CNN models outperform traditional methods and rival expert ophthalmologists' results. However, challenges such as dataset availability and model interpretability remain. Future directions include multimodal imaging integration and real-time, point-of-care CNN systems for DR screening. The integration of OCT imaging with CNNs has transformative potential in DR diagnosis, facilitating early intervention, personalized treatments, and improved patient outcomes. Abbreviations: DR = Diabetic Retinopathy, OCT = Optical Coherence Tomography, CNN = Convolutional Neural Network, CMV = Cytomegalovirus, PDR = Proliferative Diabetic Retinopathy, AMD = Age-Related Macular Degeneration, VEGF = vascular endothelial growth factor, RAP = Retinal Angiomatous Proliferation, OCTA = OCT Angiography, AI = Artificial Intelligence.

A scoping review on multimodal deep learning in biomedical images and texts

OBJECTIVE

Computer-assisted diagnostic and prognostic systems of the future should be capable of simultaneously processing multimodal data. Multimodal deep learning (MDL), which involves the integration of multiple sources of data, such as images and text, has the potential to revolutionize the analysis and interpretation of biomedical data. However, it only caught researchers' attention recently. To this end, there is a critical need to conduct a systematic review on this topic, identify the limitations of current work, and explore future directions.

METHODS

In this scoping review, we aim to provide a comprehensive overview of the current state of the field and identify key concepts, types of studies, and research gaps with a focus on biomedical images and texts joint learning, mainly because these two were the most commonly available data types in MDL research.

RESULT

This study reviewed the current uses of multimodal deep learning on five tasks: (1) Report generation, (2) Visual question answering, (3) Cross-modal retrieval, (4) Computer-aided diagnosis, and (5) Semantic segmentation.

CONCLUSION

Our results highlight the diverse applications and potential of MDL and suggest directions for future research in the field. We hope our review will facilitate the collaboration of natural language processing (NLP) and medical imaging communities and support the next generation of decision-making and computer-assisted diagnostic system development.

Artificial intelligence in psychiatry research, diagnosis, and therapy

Psychiatric disorders are now responsible for the largest proportion of the global burden of disease, and even more challenges have been seen during the COVID-19 pandemic. Artificial intelligence (AI) is commonly used to facilitate the early detection of disease, understand disease progression, and discover new treatments in the fields of both physical and mental health. The present review provides a broad overview of AI methodology and its applications in data acquisition and processing, feature extraction and characterization, psychiatric disorder classification, potential biomarker detection, real-time monitoring, and interventions in psychiatric disorders. We also comprehensively summarize AI applications with regard to the early warning, diagnosis, prognosis, and treatment of specific psychiatric disorders, including depression, schizophrenia, autism spectrum disorder, attention-deficit/hyperactivity disorder, addiction, sleep disorders, and Alzheimer's disease. The advantages and disadvantages of AI in psychiatry are clarified. We foresee a new wave of research opportunities to facilitate and improve AI technology and its long-term implications in psychiatry during and after the COVID-19 era.

AI and machine learning in resuscitation: Ongoing research, new concepts, and key challenges

Aim

Artificial intelligence (AI) and machine learning (ML) are important areas of computer science that have recently attracted attention for their application to medicine. However, as techniques continue to advance and become more complex, it is increasingly challenging for clinicians to stay abreast of the latest research. This overview aims to translate research concepts and potential concerns to healthcare professionals interested in applying AI and ML to resuscitation research but who are not experts in the field.

Main text

We present various research including prediction models using structured and unstructured data, exploring treatment heterogeneity, reinforcement learning, language processing, and large-scale language models. These studies potentially offer valuable insights for optimizing treatment strategies and clinical workflows. However, implementing AI and ML in clinical settings presents its own set of challenges. The availability of high-quality and reliable data is crucial for developing accurate ML models. A rigorous validation process and the integration of ML into clinical practice is essential for practical implementation. We furthermore highlight the potential risks associated with self-fulfilling prophecies and feedback loops, emphasizing the importance of transparency, interpretability, and trustworthiness in AI and ML models. These issues need to be addressed in order to establish reliable and trustworthy AI and ML models.

Conclusion

In this article, we overview concepts and examples of AI and ML research in the resuscitation field. Moving forward, appropriate understanding of ML and collaboration with relevant experts will be essential for researchers and clinicians to overcome the challenges and harness the full potential of AI and ML in resuscitation.

Multi-modal deep learning for automated assembly of periapical radiographs

OBJECTIVES

Periapical radiographs are oftentimes taken in series to display all teeth present in the oral cavity. Our aim was to automatically assemble such a series of periapical radiographs into an anatomically correct status using a multi-modal deep learning model.

METHODS

4,707 periapical images from 387 patients (on average, 12 images per patient) were used. Radiographs were labeled according to their field of view and the dataset split into a training, validation, and test set, stratified by patient. In addition to the radiograph the timestamp of image generation was extracted and abstracted as follows: A matrix, containing the normalized timestamps of all images of a patient was constructed, representing the order in which images were taken, providing temporal context information to the deep learning model. Using the image data together with the time sequence data a multi-modal deep learning model consisting of two residual convolutional neural networks (ResNet-152 for image data, ResNet-50 for time data) was trained. Additionally, two uni-modal models were trained on image data and time data, respectively. A custom scoring technique was used to measure model performance.

RESULTS

Multi-modal deep learning outperformed both uni-modal image-based learning (p<0.001) and time-based learning (p<0.05). The multi-modal deep learning model predicted tooth labels with an F1-score, sensitivity and precision of 0.79, respectively, and an accuracy of 0.99. 37 out of 77 patient datasets were fully correctly assembled by multi-modal learning; in the remaining ones, usually only one image was incorrectly labeled.

CONCLUSIONS

Multi-modal modeling allowed automated assembly of periapical radiographs and outperformed both uni-modal models. Dental machine learning models can benefit from additional data modalities.

CLINICAL SIGNIFICANCE

Like humans, deep learning models may profit from multiple data sources for decision-making. We demonstrate how multi-modal learning can assist assembling periapical radiographs into an anatomically correct status. Multi-modal learning should be considered for more complex tasks, as clinically a wealth of data is usually available and could be leveraged.

The shaky foundations of large language models and foundation models for electronic health records

The success of foundation models such as ChatGPT and AlphaFold has spurred significant interest in building similar models for electronic medical records (EMRs) to improve patient care and hospital operations. However, recent hype has obscured critical gaps in our understanding of these models' capabilities. In this narrative review, we examine 84 foundation models trained on non-imaging EMR data (i.e., clinical text and/or structured data) and create a taxonomy delineating their architectures, training data, and potential use cases. We find that most models are trained on small, narrowly-scoped clinical datasets (e.g., MIMIC-III) or broad, public biomedical corpora (e.g., PubMed) and are evaluated on tasks that do not provide meaningful insights on their usefulness to health systems. Considering these findings, we propose an improved evaluation framework for measuring the benefits of clinical foundation models that is more closely grounded to metrics that matter in healthcare.

Artificial Intelligence: Deciphering the Links between Psychiatric Disorders and Neurodegenerative Disease

Artificial Intelligence (AI), which is the general term used to describe technology that simulates human cognition [...].

The Emerging Role of Artificial Intelligence in Valvular Heart Disease

Valvular heart disease (VHD) is a morbid condition in which timely identification and evidence-based treatments can lead to improved outcomes. Artificial intelligence broadly refers to the ability for computers to perform tasks and problem solve like the human mind. Studies applying AI to VHD have used a variety of structured (eg, sociodemographic, clinical) and unstructured (eg, electrocardiogram, phonocardiogram, and echocardiograms) and machine learning modeling approaches. Additional researches in diverse populations, including prospective clinical trials, are needed to evaluate the effectiveness and value of AI-enabled medical technologies in clinical care for patients with VHD.

Assessment of brain cancer atlas maps with multimodal imaging features

BACKGROUND

Glioblastoma Multiforme (GBM) is a fast-growing and highly aggressive brain tumor that invades the nearby brain tissue and presents secondary nodular lesions across the whole brain but generally does not spread to distant organs. Without treatment, GBM can result in death in about 6 months. The challenges are known to depend on multiple factors: brain localization, resistance to conventional therapy, disrupted tumor blood supply inhibiting effective drug delivery, complications from peritumoral edema, intracranial hypertension, seizures, and neurotoxicity.

MAIN TEXT

Imaging techniques are routinely used to obtain accurate detections of lesions that localize brain tumors. Especially magnetic resonance imaging (MRI) delivers multimodal images both before and after the administration of contrast, which results in displaying enhancement and describing physiological features as hemodynamic processes. This review considers one possible extension of the use of radiomics in GBM studies, one that recalibrates the analysis of targeted segmentations to the whole organ scale. After identifying critical areas of research, the focus is on illustrating the potential utility of an integrated approach with multimodal imaging, radiomic data processing and brain atlases as the main components. The templates associated with the outcome of straightforward analyses represent promising inference tools able to spatio-temporally inform on the GBM evolution while being generalizable also to other cancers.

CONCLUSIONS

The focus on novel inference strategies applicable to complex cancer systems and based on building radiomic models from multimodal imaging data can be well supported by machine learning and other computational tools potentially able to translate suitably processed information into more accurate patient stratifications and evaluations of treatment efficacy.

Wearable-based accelerometer activity profile as digital biomarker of inflammation, biological age, and mortality using hierarchical clustering analysis in NHANES 2011-2014

Repeated disruptions in circadian rhythms are associated with implications for health outcomes and longevity. The utilization of wearable devices in quantifying circadian rhythm to elucidate its connection to longevity, through continuously collected data remains largely unstudied. In this work, we investigate a data-driven segmentation of the 24-h accelerometer activity profiles from wearables as a novel digital biomarker for longevity in 7,297 U.S. adults from the 2011-2014 National Health and Nutrition Examination Survey. Using hierarchical clustering, we identified five clusters and described them as follows: "High activity", "Low activity", "Mild circadian rhythm (CR) disruption", "Severe CR disruption", and "Very low activity". Young adults with extreme CR disturbance are seemingly healthy with few comorbid conditions, but in fact associated with higher white blood cell, neutrophils, and lymphocyte counts (0.05-0.07 log-unit, all p < 0.05) and accelerated biological aging (1.42 years, p < 0.001). Older adults with CR disruption are significantly associated with increased systemic inflammation indexes (0.09-0.12 log-unit, all p < 0.05), biological aging advance (1.28 years, p = 0.021), and all-cause mortality risk (HR = 1.58, p = 0.042). Our findings highlight the importance of circadian alignment on longevity across all ages and suggest that data from wearable accelerometers can help in identifying at-risk populations and personalize treatments for healthier aging.

Searching for the Predictors of Response to BoNT-A in Migraine Using Machine Learning Approaches

OnabotulinumtoxinA (BonT-A) reduces migraine frequency in a considerable portion of patients with migraine. So far, predictive characteristics of response are lacking. Here, we applied machine learning (ML) algorithms to identify clinical characteristics able to predict treatment response. We collected demographic and clinical data of patients with chronic migraine (CM) or high-frequency episodic migraine (HFEM) treated with BoNT-A at our clinic in the last 5 years. Patients received BoNT-A according to the PREEMPT (Phase III Research Evaluating Migraine Prophylaxis Therapy) paradigm and were classified according to the monthly migraine days reduction in the 12 weeks after the fourth BoNT-A cycle, as compared to baseline. Data were used as input features to run ML algorithms. Of the 212 patients enrolled, 35 qualified as excellent responders to BoNT-A administration and 38 as nonresponders. None of the anamnestic characteristics were able to discriminate responders from nonresponders in the CM group. Nevertheless, a pattern of four features (age at onset of migraine, opioid use, anxiety subscore at the hospital anxiety and depression scale (HADS-a) and Migraine Disability Assessment (MIDAS) score correctly predicted response in HFEM. Our findings suggest that routine anamnestic features acquired in real-life settings cannot accurately predict BoNT-A response in migraine and call for a more complex modality of patient profiling.

Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes

The broad adoption of electronic health records (EHRs) provides great opportunities to conduct healthcare research and solve various clinical problems in medicine. With recent advances and success, methods based on machine learning and deep learning have become increasingly popular in medical informatics. Combining data from multiple modalities may help in predictive tasks. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in Electronic Health Record (EHR) for enhanced performance in downstream predictive tasks. Early, joint, and late fusion strategies were employed to effectively combine data from various modalities. Model performance and contribution scores show that multimodal models outperform uni-modal models in various tasks. Additionally, temporal signs contain more information than CXR images and clinical notes in three explored predictive tasks. Therefore, models integrating different data modalities can work better in predictive tasks.

Prediction of disease comorbidity using explainable artificial intelligence and machine learning techniques: A systematic review

OBJECTIVE

Disease comorbidity is a major challenge in healthcare affecting the patient's quality of life and costs. AI-based prediction of comorbidities can overcome this issue by improving precision medicine and providing holistic care. The objective of this systematic literature review was to identify and summarise existing machine learning (ML) methods for comorbidity prediction and evaluate the interpretability and explainability of the models.

MATERIALS AND METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework was used to identify articles in three databases: Ovid Medline, Web of Science and PubMed. The literature search covered a broad range of terms for the prediction of disease comorbidity and ML, including traditional predictive modelling.

RESULTS

Of 829 unique articles, 58 full-text papers were assessed for eligibility. A final set of 22 articles with 61 ML models was included in this review. Of the identified ML models, 33 models achieved relatively high accuracy (80-95%) and AUC (0.80-0.89). Overall, 72% of studies had high or unclear concerns regarding the risk of bias.

DISCUSSION

This systematic review is the first to examine the use of ML and explainable artificial intelligence (XAI) methods for comorbidity prediction. The chosen studies focused on a limited scope of comorbidities ranging from 1 to 34 (mean = 6), and no novel comorbidities were found due to limited phenotypic and genetic data. The lack of standard evaluation for XAI hinders fair comparisons.

CONCLUSION

A broad range of ML methods has been used to predict the comorbidities of various disorders. With further development of explainable ML capacity in the field of comorbidity prediction, there is a significant possibility of identifying unmet health needs by highlighting comorbidities in patient groups that were not previously recognised to be at risk for particular comorbidities.

Multimodal deep learning methods enhance genomic prediction of wheat breeding

While several statistical machine learning methods have been developed and studied for assessing the genomic prediction (GP) accuracy of unobserved phenotypes in plant breeding research, few methods have linked genomics and phenomics (imaging). Deep learning (DL) neural networks have been developed to increase the GP accuracy of unobserved phenotypes while simultaneously accounting for the complexity of genotype-environment interaction (GE); however, unlike conventional GP models, DL has not been investigated for when genomics is linked with phenomics. In this study we used 2 wheat data sets (DS1 and DS2) to compare a novel DL method with conventional GP models. Models fitted for DS1 were GBLUP, gradient boosting machine (GBM), support vector regression (SVR) and the DL method. Results indicated that for 1 year, DL provided better GP accuracy than results obtained by the other models. However, GP accuracy obtained for other years indicated that the GBLUP model was slightly superior to the DL. DS2 is comprised only of genomic data from wheat lines tested for 3 years, 2 environments (drought and irrigated) and 2-4 traits. DS2 results showed that when predicting the irrigated environment with the drought environment, DL had higher accuracy than the GBLUP model in all analyzed traits and years. When predicting drought environment with information on the irrigated environment, the DL model and GBLUP model had similar accuracy. The DL method used in this study is novel and presents a strong degree of generalization as several modules can potentially be incorporated and concatenated to produce an output for a multi-input data structure.

Circadian rhythm biomarker from wearable device data is related to concurrent antidepressant treatment response

Major depressive disorder (MDD) is associated with circadian rhythm disruption. Yet, no circadian rhythm biomarkers have been clinically validated for assessing antidepressant response. In this study, 40 participants with MDD provided actigraphy data using wearable devices for one week after initiating antidepressant treatment in a randomized, double-blind, placebo-controlled trial. Their depression severity was calculated pretreatment, after one week and eight weeks of treatment. This study assesses the relationship between parametric and nonparametric measures of circadian rhythm and change in depression. Results show significant association between a lower circadian quotient (reflecting less robust rhythmicity) and improvement in depression from baseline following first week of treatment (estimate = 0.11, F = 7.01, P = 0.01). There is insufficient evidence of an association between circadian rhythm measures acquired during the first week of treatment and outcomes after eight weeks of treatment. Despite this lack of association with future treatment outcome, this scalable, cost-effective biomarker may be useful for timely mental health care through remote monitoring of real-time changes in current depression.

Use of Computation Ecosystems to Analyze the Kidney-Heart Crosstalk

The identification of mediators for physiologic processes, correlation of molecular processes, or even pathophysiological processes within a single organ such as the kidney or heart has been extensively studied to answer specific research questions using organ-centered approaches in the past 50 years. However, it has become evident that these approaches do not adequately complement each other and display a distorted single-disease progression, lacking holistic multilevel/multidimensional correlations. Holistic approaches have become increasingly significant in understanding and uncovering high dimensional interactions and molecular overlaps between different organ systems in the pathophysiology of multimorbid and systemic diseases like cardiorenal syndrome because of pathological heart-kidney crosstalk. Holistic approaches to unraveling multimorbid diseases are based on the integration, merging, and correlation of extensive, heterogeneous, and multidimensional data from different data sources, both -omics and nonomics databases. These approaches aimed at generating viable and translatable disease models using mathematical, statistical, and computational tools, thereby creating first computational ecosystems. As part of these computational ecosystems, systems medicine solutions focus on the analysis of -omics data in single-organ diseases. However, the data-scientific requirements to address the complexity of multimodality and multimorbidity reach far beyond what is currently available and require multiphased and cross-sectional approaches. These approaches break down complexity into small and comprehensible challenges. Such holistic computational ecosystems encompass data, methods, processes, and interdisciplinary knowledge to manage the complexity of multiorgan crosstalk. Therefore, this review summarizes the current knowledge of kidney-heart crosstalk, along with methods and opportunities that arise from the novel application of computational ecosystems providing a holistic analysis on the example of kidney-heart crosstalk.

Diagnostic quality model (DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML

BACKGROUND

Laboratory medicine has reached the era where promises of artificial intelligence and machine learning (AI/ML) seem palpable. Currently, the primary responsibility for risk-benefit assessment in clinical practice resides with the medical director. Unfortunately, there is no tool or concept that enables diagnostic quality assessment for the various potential AI/ML applications. Specifically, we noted that an operational definition of laboratory diagnostic quality - for the specific purpose of assessing AI/ML improvements - is currently missing.

METHODS

A session at the 3rd Strategic Conference of the European Federation of Laboratory Medicine in 2022 on "AI in the Laboratory of the Future" prompted an expert roundtable discussion. Here we present a conceptual diagnostic quality framework for the specific purpose of assessing AI/ML implementations.

RESULTS

The presented framework is termed diagnostic quality model (DQM) and distinguishes AI/ML improvements at the test, procedure, laboratory, or healthcare ecosystem level. The operational definition illustrates the nested relationship among these levels. The model can help to define relevant objectives for implementation and how levels come together to form coherent diagnostics. The affected levels are referred to as scope and we provide a rubric to quantify AI/ML improvements while complying with existing, mandated regulatory standards. We present 4 relevant clinical scenarios including multi-modal diagnostics and compare the model to existing quality management systems.

CONCLUSIONS

A diagnostic quality model is essential to navigate the complexities of clinical AI/ML implementations. The presented diagnostic quality framework can help to specify and communicate the key implications of AI/ML solutions in laboratory diagnostics.

Leveraging transcriptomics for precision diagnosis: Lessons learned from cancer and sepsis

Diagnostics require precision and predictive ability to be clinically useful. Integration of multi-omic with clinical data is crucial to our understanding of disease pathogenesis and diagnosis. However, interpretation of overwhelming amounts of information at the individual level requires sophisticated computational tools for extraction of clinically meaningful outputs. Moreover, evolution of technical and analytical methods often outpaces standardisation strategies. RNA is the most dynamic component of all -omics technologies carrying an abundance of regulatory information that is least harnessed for use in clinical diagnostics. Gene expression-based tests capture genetic and non-genetic heterogeneity and have been implemented in certain diseases. For example patients with early breast cancer are spared toxic unnecessary treatments with scores based on the expression of a set of genes (e.g., Oncotype DX). The ability of transcriptomics to portray the transcriptional status at a moment in time has also been used in diagnosis of dynamic diseases such as sepsis. Gene expression profiles identify endotypes in sepsis patients with prognostic value and a potential to discriminate between viral and bacterial infection. The application of transcriptomics for patient stratification in clinical environments and clinical trials thus holds promise. In this review, we discuss the current clinical application in the fields of cancer and infection. We use these paradigms to highlight the impediments in identifying useful diagnostic and prognostic biomarkers and propose approaches to overcome them and aid efforts towards clinical implementation.

A review on multimodal machine learning in medical diagnostics

Nowadays, the increasing number of medical diagnostic data and clinical data provide more complementary references for doctors to make diagnosis to patients. For example, with medical data, such as electrocardiography (ECG), machine learning algorithms can be used to identify and diagnose heart disease to reduce the workload of doctors. However, ECG data is always exposed to various kinds of noise and interference in reality, and medical diagnostics only based on one-dimensional ECG data is not trustable enough. By extracting new features from other types of medical data, we can implement enhanced recognition methods, called multimodal learning. Multimodal learning helps models to process data from a range of different sources, eliminate the requirement for training each single learning modality, and improve the robustness of models with the diversity of data. Growing number of articles in recent years have been devoted to investigating how to extract data from different sources and build accurate multimodal machine learning models, or deep learning models for medical diagnostics. This paper reviews and summarizes several recent papers that dealing with multimodal machine learning in disease detection, and identify topics for future research.

Predicting psychotic relapse following randomised discontinuation of paliperidone in individuals with schizophrenia or schizoaffective disorder: an individual participant data analysis

BACKGROUND

Predicting relapse for individuals with psychotic disorders is not well established, especially after discontinuation of antipsychotic treatment. We aimed to identify general prognostic factors of relapse for all participants (irrespective of treatment continuation or discontinuation) and specific predictors of relapse for treatment discontinuation, using machine learning.

METHODS

For this individual participant data analysis, we searched the Yale University Open Data Access Project's database for placebo-controlled, randomised antipsychotic discontinuation trials with participants with schizophrenia or schizoaffective disorder (aged ≥18 years). We included studies in which participants were treated with any antipsychotic study drug and randomly assigned to continue the same antipsychotic drug or to discontinue it and receive placebo. We assessed 36 prespecified baseline variables at randomisation to predict time to relapse, using univariate and multivariate proportional hazard regression models (including multivariate treatment group by variable interactions) with machine learning to categorise the variables as general prognostic factors of relapse, specific predictors of relapse, or both.

FINDINGS

We identified 414 trials, of which five trials with 700 participants (304 [43%] women and 396 [57%] men) were eligible for the continuation group and 692 participants (292 [42%] women and 400 [58%] men) were eligible for the discontinuation group (median age 37 [IQR 28-47] years for continuation group and 38 [28-47] years for discontinuation group). Out of the 36 baseline variables, general prognostic factors of increased risk of relapse for all participants were drug-positive urine; paranoid, disorganised, and undifferentiated types of schizophrenia (lower risk for schizoaffective disorder); psychiatric and neurological adverse events; higher severity of akathisia (ie, difficulty or inability to sit still); antipsychotic discontinuation; lower social performance; younger age; lower glomerular filtration rate; benzodiazepine comedication (lower risk for anti-epileptic comedication). Out of the 36 baseline variables, predictors of increased risk specifically after antipsychotic discontinuation were increased prolactin concentration, higher number of hospitalisations, and smoking. Both prognostic factors and predictors with increased risk after discontinuation were oral antipsychotic treatment (lower risk for long-acting injectables), higher last dosage of the antipsychotic study drug, shorter duration of antipsychotic treatment, and higher score on the Clinical Global Impression (CGI) severity scale The predictive performance (concordance index) for participants who were not used to train the model was 0·707 (chance level is 0·5).

INTERPRETATION

Routinely available general prognostic factors of psychotic relapse and predictors specific for treatment discontinuation could be used to support personalised treatment. Abrupt discontinuation of higher dosages of oral antipsychotics, especially for individuals with recurring hospitalisations, higher scores on the CGI severity scale, and increased prolactin concentrations, should be avoided to reduce the risk of relapse.

FUNDING

German Research Foundation and Berlin Institute of Health.

Breaking Bias: The Role of Artificial Intelligence in Improving Clinical Decision-Making

This case report reflects on a delayed diagnosis for a 27-year-old woman who reported chest pain and shortness of breath to the emergency department. The treating clinician reflects upon how cognitive biases influenced their diagnostic process and how multiple missed opportunities resulted in missteps. Using artificial intelligence (AI) tools for clinical decision-making, we suggest how AI could augment the clinician, and in this case, delayed diagnosis avoided. Incorporating AI tools into clinical decision-making brings potential benefits, including improved diagnostic accuracy and addressing human factors contributing to medical errors. For example, they may support a real-time interpretation of medical imaging and assist clinicians in generating a differential diagnosis in ensuring that critical diagnoses are considered. However, it is vital to be aware of the potential pitfalls associated with the use of AI, such as automation bias, input data quality issues, limited clinician training in interpreting AI methods, and the legal and ethical considerations associated with their use. The report draws attention to the utility of AI clinical decision-support tools in overcoming human cognitive biases. It also emphasizes the importance of clinicians developing skills needed to steward the adoption of AI tools in healthcare and serve as patient advocates, ensuring safe and effective use of health data.

A Scalable Framework for Closed-Loop Neuromodulation with Deep Learning

Closed-loop neuromodulation measures dynamic neural or physiological activity to optimize interventions for clinical and nonclinical behavioral, cognitive, wellness, attentional, or general task performance enhancement. Conventional closed-loop stimulation approaches can contain biased biomarker detection (decoders and error-based triggering) and stimulation-type application. We present and verify a novel deep learning framework for designing and deploying flexible, data-driven, automated closed-loop neuromodulation that is scalable using diverse datasets, agnostic to stimulation technology (supporting multi-modal stimulation: tACS, tDCS, tFUS, TMS), and without the need for personalized ground-truth performance data. Our approach is based on identified periods of responsiveness - detected states that result in a change in performance when stimulation is applied compared to no stimulation. To demonstrate our framework, we acquire, analyze, and apply a data-driven approach to our open sourced GX dataset, which includes concurrent physiological (ECG, EOG) and neuronal (EEG) measures, paired with continuous vigilance/attention-fatigue tracking, and High-Definition transcranial electrical stimulation (HD-tES). Our framework's decision process for intervention application identified 88.26% of trials as correct applications, showed potential improvement with varying stimulation types, or missed opportunities to stimulate, whereas 11.25% of trials were predicted to stimulate at inopportune times. With emerging datasets and stimulation technologies, our unifying and integrative framework; leveraging deep learning (Convolutional Neural Networks - CNNs); demonstrates the adaptability and feasibility of automated multimodal neuromodulation for both clinical and nonclinical applications.

Smart Wearables for the Detection of Cardiovascular Diseases: A Systematic Literature Review

Background: The advancement of information and communication technologies and the growing power of artificial intelligence are successfully transforming a number of concepts that are important to our daily lives. Many sectors, including education, healthcare, industry, and others, are benefiting greatly from the use of such resources. The healthcare sector, for example, was an early adopter of smart wearables, which primarily serve as diagnostic tools. In this context, smart wearables have demonstrated their effectiveness in detecting and predicting cardiovascular diseases (CVDs), the leading cause of death worldwide. Objective: In this study, a systematic literature review of smart wearable applications for cardiovascular disease detection and prediction is presented. After conducting the required search, the documents that met the criteria were analyzed to extract key criteria such as the publication year, vital signs recorded, diseases studied, hardware used, smart models used, datasets used, and performance metrics. Methods: This study followed the PRISMA guidelines by searching IEEE, PubMed, and Scopus for publications published between 2010 and 2022. Once records were located, they were reviewed to determine which ones should be included in the analysis. Finally, the analysis was completed, and the relevant data were included in the review along with the relevant articles. Results: As a result of the comprehensive search procedures, 87 papers were deemed relevant for further review. In addition, the results are discussed to evaluate the development and use of smart wearable devices for cardiovascular disease management, and the results demonstrate the high efficiency of such wearable devices. Conclusions: The results clearly show that interest in this topic has increased. Although the results show that smart wearables are quite accurate in detecting, predicting, and even treating cardiovascular disease, further research is needed to improve their use.

Recent advancements in digital health management using multi-modal signal monitoring

Healthcare is the method of keeping or enhancing physical and mental well-being with its aid of illness and injury prevention, diagnosis, and treatment. The majority of conventional healthcare practices involve manual management and upkeep of client demographic information, case histories, diagnoses, medications, invoicing, and drug stock upkeep, which can result in human errors that have an impact on clients. By linking all the essential parameter monitoring equipment through a network with a decision-support system, digital health management based on Internet of Things (IoT) eliminates human errors and aids the doctor in making more accurate and timely diagnoses. The term "Internet of Medical Things" (IoMT) refers to medical devices that have the ability to communicate data over a network without requiring human-to-human or human-to-computer interaction. Meanwhile, more effective monitoring gadgets have been made due to the technology advancements, and these devices can typically record a few physiological signals simultaneously, including the electrocardiogram (ECG) signal, the electroglottography (EGG) signal, the electroencephalogram (EEG) signal, and the electrooculogram (EOG) signal. Yet, there has not been much research on the connection between digital health management and multi-modal signal monitoring. To bridge the gap, this article reviews the latest advancements in digital health management using multi-modal signal monitoring. Specifically, three digital health processes, namely, lower-limb data collection, statistical analysis of lower-limb data, and lower-limb rehabilitation via digital health management, are covered in this article, with the aim to fully review the current application of digital health technology in lower-limb symptom recovery.

A guide to multi-omics data collection and integration for translational medicine

The emerging high-throughput technologies have led to the shift in the design of translational medicine projects towards collecting multi-omics patient samples and, consequently, their integrated analysis. However, the complexity of integrating these datasets has triggered new questions regarding the appropriateness of the available computational methods. Currently, there is no clear consensus on the best combination of omics to include and the data integration methodologies required for their analysis. This article aims to guide the design of multi-omics studies in the field of translational medicine regarding the types of omics and the integration method to choose. We review articles that perform the integration of multiple omics measurements from patient samples. We identify five objectives in translational medicine applications: (i) detect disease-associated molecular patterns, (ii) subtype identification, (iii) diagnosis/prognosis, (iv) drug response prediction, and (v) understand regulatory processes. We describe common trends in the selection of omic types combined for different objectives and diseases. To guide the choice of data integration tools, we group them into the scientific objectives they aim to address. We describe the main computational methods adopted to achieve these objectives and present examples of tools. We compare tools based on how they deal with the computational challenges of data integration and comment on how they perform against predefined objective-specific evaluation criteria. Finally, we discuss examples of tools for downstream analysis and further extraction of novel insights from multi-omics datasets.

Smart Wearables for the Detection of Occupational Physical Fatigue: A Literature Review

Today's world is changing dramatically due to the influence of various factors. Whether due to the rapid development of technological tools, advances in telecommunication methods, global economic and social events, or other reasons, almost everything is changing. As a result, the concepts of a "job" or work have changed as well, with new work shifts being introduced and the office no longer being the only place where work is done. In addition, our non-stop active society has increased the stress and pressure at work, causing fatigue to spread worldwide and becoming a global problem. Moreover, it is medically proven that persistent fatigue is a cause of serious diseases and health problems. Therefore, monitoring and detecting fatigue in the workplace is essential to improve worker safety in the long term. In this paper, we provide an overview of the use of smart wearable devices to monitor and detect occupational physical fatigue. In addition, we present and discuss the challenges that hinder this field and highlight what can be done to advance the use of smart wearables in workplace fatigue detection.