Improving the quality of machine learning in health applications and clinical research

Clinical translation of nanomedicine with integrated digital medicine and machine learning interventions

Nanomaterials based therapeutics transform the ways of disease prevention, diagnosis and treatment with increasing sophistications in nanotechnology at a breakneck pace, but very few could reach to the clinic due to inconsistencies in preclinical studies followed by regulatory hinderances. To tackle this, integrating the nanomedicine discovery with digital medicine provide technologies as tools of specific biological activity measurement. Hence, overcome the redundancies in nanomedicine discovery by the on-site data acquisition and analytics through integrating intelligent sensors and artificial intelligence (AI) or machine learning (ML). Integrated AI/ML wearable sensors directly gather clinically relevant biochemical information from the subject's body and process data for physicians to make right clinical decision(s) in a time and cost-effective way. This review summarizes insights and recommend the infusion of actionable big data computation enabled sensors in burgeoning field of nanomedicine at academia, research institutes, and pharmaceutical industries, with a potential of clinical translation. Furthermore, many blind spots are present in modern clinically relevant computation, one of which could prevent ML-guided low-cost new nanomedicine development from being successfully translated into the clinic was also discussed.

Machine-learning-based prediction of disability progression in multiple sclerosis: An observational, international, multi-center study

BACKGROUND

Disability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of the probability of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking.

METHODS

Data of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were followed. Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expanded disability status scale, treatment, relapse information, and MS course. To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated with the area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error. All our preprocessing and model code are available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS.

FINDINGS

Machine learning models achieved a ROC-AUC of 0⋅71 ± 0⋅01, an AUC-PR of 0⋅26 ± 0⋅02, a Brier score of 0⋅1 ± 0⋅01 and an expected calibration error of 0⋅07 ± 0⋅04. The history of disability progression was identified as being more predictive for future disability progression than the treatment or relapses history.

CONCLUSIONS

Good discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This suggests machine-learning models can reliably inform clinicians about the future occurrence of progression and are mature for a clinical impact study.

Machine Learning-Based Predictive Models for Patients with Venous Thromboembolism: A Systematic Review

BACKGROUND

 Venous thromboembolism (VTE) is a chronic disorder with a significant health and economic burden. Several VTE-specific clinical prediction models (CPMs) have been used to assist physicians in decision-making but have several limitations. This systematic review explores if machine learning (ML) can enhance CPMs by analyzing extensive patient data derived from electronic health records. We aimed to explore ML-CPMs' applications in VTE for risk stratification, outcome prediction, diagnosis, and treatment.

METHODS

 Three databases were searched: PubMed, Google Scholar, and IEEE electronic library. Inclusion criteria focused on studies using structured data, excluding non-English publications, studies on non-humans, and certain data types such as natural language processing and image processing. Studies involving pregnant women, cancer patients, and children were also excluded. After excluding irrelevant studies, a total of 77 studies were included.

RESULTS

 Most studies report that ML-CPMs outperformed traditional CPMs in terms of receiver operating area under the curve in the four clinical domains that were explored. However, the majority of the studies were retrospective, monocentric, and lacked detailed model architecture description and external validation, which are essential for quality audit. This review identified research gaps and highlighted challenges related to standardized reporting, reproducibility, and model comparison.

CONCLUSION

 ML-CPMs show promise in improving risk assessment and individualized treatment recommendations in VTE. Apparently, there is an urgent need for standardized reporting and methodology for ML models, external validation, prospective and real-world data studies, as well as interventional studies to evaluate the impact of artificial intelligence in VTE.

Predicting anxiety, depression, and insomnia among Bangladeshi university students using tree-based machine learning models

Background and Aims

Mental health problem is a rising public health concern. People of all ages, specially Bangladeshi university students, are more affected by this burden. Thus, the objective of the study was to use tree-based machine learning (ML) models to identify major risk factors and predict anxiety, depression, and insomnia in university students.

Methods

A social media-based cross-sectional survey was employed for data collection. We used Generalized Anxiety Disorder (GAD-7), Patient Health Questionnaire (PHQ-9) and Insomnia Severity Index (ISI-7) scale for measuring students' anxiety, depression and insomnia problems. The tree-based supervised decision tree (DT), random forest (RF) and robust eXtreme Gradient Boosting (XGBoost) ML algorithms were used to build the prediction models and their predictive performance was evaluated using confusion matrix and receiver operating characteristic (ROC) curves.

Results

Of the 1250 students surveyed, 64.7% were male and 35.3% were female. The students' ages ranged from 18 to 26 years old, with an average age of 22.24 years (SD = 1.30). Majority of the students (72.6%) were from rural areas and social media addicted (56.6%). Almost 83.3% of the students had moderate to severe anxiety, 84.7% had moderate to severe depression and 76.5% had moderate to severe insomnia problems. Students' social media addiction, age, academic performance, smoking status, monthly family income and morningness-eveningness are the main risk factors of anxiety, depression and insomnia. The highest predictive performance was observed from the XGBoost model for anxiety, depression and insomnia.

Conclusion

The study findings offer valuable insights for stakeholders, families and policymakers enabling a more profound comprehension of the pressing mental health disorders. This understanding can guide the formulation of improved policy strategies, initiatives for mental health promotion, and the development of effective counseling services within university campus. Additionally, our proposed model might play a critical role in diagnosing and predicting mental health problems among Bangladeshi university students and similar settings.

A trustworthy AI reality-check: the lack of transparency of artificial intelligence products in healthcare

Trustworthy medical AI requires transparency about the development and testing of underlying algorithms to identify biases and communicate potential risks of harm. Abundant guidance exists on how to achieve transparency for medical AI products, but it is unclear whether publicly available information adequately informs about their risks. To assess this, we retrieved public documentation on the 14 available CE-certified AI-based radiology products of the II b risk category in the EU from vendor websites, scientific publications, and the European EUDAMED database. Using a self-designed survey, we reported on their development, validation, ethical considerations, and deployment caveats, according to trustworthy AI guidelines. We scored each question with either 0, 0.5, or 1, to rate if the required information was "unavailable", "partially available," or "fully available." The transparency of each product was calculated relative to all 55 questions. Transparency scores ranged from 6.4% to 60.9%, with a median of 29.1%. Major transparency gaps included missing documentation on training data, ethical considerations, and limitations for deployment. Ethical aspects like consent, safety monitoring, and GDPR-compliance were rarely documented. Furthermore, deployment caveats for different demographics and medical settings were scarce. In conclusion, public documentation of authorized medical AI products in Europe lacks sufficient public transparency to inform about safety and risks. We call on lawmakers and regulators to establish legally mandated requirements for public and substantive transparency to fulfill the promise of trustworthy AI for health.

"Facekit"-Toward an Automated Facial Analysis App Using a Machine Learning-Derived Facial Recognition Algorithm

Introduction: Multiple tools have been developed for facial feature measurements and analysis using facial recognition machine learning techniques. However, several challenges remain before these will be useful in the clinical context for reconstructive and aesthetic plastic surgery. Smartphone-based applications utilizing open-access machine learning tools can be rapidly developed, deployed, and tested for use in clinical settings. This research compares a smartphone-based facial recognition algorithm to direct and digital measurement performance for use in facial analysis. Methods: Facekit is a camera application developed for Android that utilizes ML Kit, an open-access computer vision Application Programing Interface developed by Google. Using the facial landmark module, we measured 4 facial proportions in 15 healthy subjects and compared them to direct surface and digital measurements using intraclass correlation (ICC) and Pearson correlation. Results: Measurement of the naso-facial proportion achieved the highest ICC of 0.321, where ICC > 0.75 is considered an excellent agreement between methods. Repeated measures analysis of variance of proportion measurements between ML Kit, direct and digital methods, were significantly different (F[2,14] = 6-26, P<<.05). Facekit measurements of orbital, orbitonasal, naso-oral, and naso-facial ratios had overall low correlation and agreement to both direct and digital measurements (R<<0.5, ICC<<0.75). Conclusion: Facekit is a smartphone camera application for rapid facial feature analysis. Agreement between Facekit's machine learning measurements and direct and digital measurements was low. We conclude that the chosen pretrained facial recognition software is not accurate enough for conducting a clinically useful facial analysis. Custom models trained on accurate and clinically relevant landmarks may provide better performance.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet.

OBJECTIVES

To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS.

SEARCH METHODS

We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies.

SELECTION CRITERIA

We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome.

DATA COLLECTION AND ANALYSIS

Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression.

MAIN RESULTS

We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal.

AUTHORS' CONCLUSIONS

The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.

Multi-institutional Prognostic Modeling in Head and Neck Cancer: Evaluating Impact and Generalizability of Deep Learning and Radiomics

Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging (radiomics). However, the development of prognostic models is complex as no modeling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records (EMR) and pretreatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or EMR data. The model with the highest accuracy used multitask learning on clinical data and tumor volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks. We have developed highly prognostic models for overall survival in HNC using EMRs and pretreatment radiological images based on a large, retrospective dataset of 2,552 patients from our institution.Diverse ML approaches were used by independent investigators. The model with the highest accuracy used multitask learning on clinical data and tumor volume.External validation of the top three performing models on three datasets (873 patients) with significant differences in the distributions of clinical and demographic variables demonstrated significant decreases in model performance.

Significance

ML combined with simple prognostic factors outperformed multiple advanced CT radiomics and deep learning methods. ML models provided diverse solutions for prognosis of patients with HNC but their prognostic value is affected by differences in patient populations and require extensive validation.

A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 1-Data From Wearable Devices

OBJECTIVES

With the emerging use of machine learning (ML) techniques, there has been particular interest in using wearable data for health economics and outcomes research (HEOR). We aimed to understand the emerging patterns of how ML has been applied to wearable data in HEOR.

METHODS

We identified studies published in PubMed between January 2016 and March 2021. Studies that included at least 1 HEOR-related Medical Subject Headings term, applied an ML, and used wearable data were eligible for inclusion. Two reviewers abstracted information including ML application types and data on which ML was applied and analyzed them using descriptive analyses.

RESULTS

A total of 148 studies were identified from PubMed, among which 32 studies met the inclusion criteria. There has been an increase over time in the number of ML studies using wearable data. ML has been more frequently used for monitoring events in real time (78%) than to predict future events (22%). There has been a wide range of outcomes examined, ranging from general physical or mental health (24%) to more disease-specific outcomes (eg, disease incidence [19%] and progression [13%]) and treatment-related outcomes (eg, treatment adherence [9%] and outcomes [9%]). Data for ML models were more often derived from wearable devices with specific medical purposes (60%) than those without (40%).

CONCLUSION

There has been a wide range of applications of ML to wearable data. Both medical and nonmedical wearable devices have been used as a data source, showing the potential for providing rich data for ML studies in HEOR.

An assessment of random forest technique using simulation study: illustration with infant mortality in Bangladesh

We aimed to assess different machine learning techniques for predicting infant mortality (<1 year) in Bangladesh. The decision tree (DT), random forest (RF), support vector machine (SVM) and logistic regression (LR) approaches were evaluated through accuracy, sensitivity, specificity, precision, F1-score, receiver operating characteristics curve and k-fold cross-validation via simulations. The Boruta algorithm and chi-square ( χ 2 ) test were used for features selection of infant mortality. Overall, the RF technique (Boruta: accuracy = 0.8890, sensitivity = 0.0480, specificity = 0.9789, precision = 0.1960, F1-score = 0.0771, AUC = 0.6590; χ 2 : accuracy = 0.8856, sensitivity = 0.0536, specificity = 0.9745, precision = 0.1837, F1-score = 0.0828, AUC = 0.6480) showed higher predictive performance for infant mortality compared to other approaches. Age at first marriage and birth, body mass index (BMI), birth interval, place of residence, religion, administrative division, parents education, occupation of mother, media-exposure, wealth index, gender of child, birth order, children ever born, toilet facility and cooking fuel were potential determinants of infant mortality in Bangladesh. Study findings may help women, stakeholders and policy-makers to take necessary steps for reducing infant mortality by creating awareness, expanding educational programs at community levels and public health interventions.

A machine learning framework supporting prospective clinical decisions applied to risk prediction in oncology

We present a general framework for developing a machine learning (ML) tool that supports clinician assessment of patient risk using electronic health record-derived real-world data and apply the framework to a quality improvement use case in an oncology setting to identify patients at risk for a near-term (60 day) emergency department (ED) visit who could potentially be eligible for a home-based acute care program. Framework steps include defining clinical quality improvement goals, model development and validation, bias assessment, retrospective and prospective validation, and deployment in clinical workflow. In the retrospective analysis for the use case, 8% of patient encounters were associated with a high risk (pre-defined as predicted probability ≥20%) for a near-term ED visit by the patient. Positive predictive value (PPV) and negative predictive value (NPV) for future ED events was 26% and 91%, respectively. Odds ratio (OR) of ED visit (high- vs. low-risk) was 3.5 (95% CI: 3.4–3.5). The model appeared to be calibrated across racial, gender, and ethnic groups. In the prospective analysis, 10% of patients were classified as high risk, 76% of whom were confirmed by clinicians as eligible for home-based acute care. PPV and NPV for future ED events was 22% and 95%, respectively. OR of ED visit (high- vs. low-risk) was 5.4 (95% CI: 2.6–11.0). The proposed framework for an ML-based tool that supports clinician assessment of patient risk is a stepwise development approach; we successfully applied the framework to an ED visit risk prediction use case.

Patient and public involvement to build trust in artificial intelligence: A framework, tools, and case studies

Artificial intelligence (AI) is increasingly taking on a greater role in healthcare. However, hype and negative news reports about AI abound. Integrating patient and public involvement (PPI) in healthcare AI projects may help in adoption and acceptance of these technologies.

We argue that AI algorithms should also be co-designed with patients and healthcare workers.

We specifically suggest (1) including patients with lived experience of the disease, and (2) creating a research advisory group (RAG) and using these group meetings to walk patients through the process of AI model building, starting with simple (e.g., linear) models.

We present a framework, case studies, best practices, and tools for applying participative data science to healthcare, enabling data scientists, clinicians, and patients to work together. The strategy of co-designing with patients can help set more realistic expectations for all stakeholders, since conventional narratives of AI revolve around dystopia or limitless optimism.

Hype and negative news reports about artificial intelligence (AI) abound. Involving patients in healthcare AI projects may help in adoption and acceptance of these technologies. We argue that AI algorithms should be co-designed with patients and healthcare workers.

We show examples of how to involve patients in AI research and how patients can build trust in algorithms. We share some best practices, case studies, a framework, and computational tools.

Avenues for future work include guidelines for patient and public involvement in AI healthcare research for funding bodies and regulatory agencies.

An understanding of what AI can and cannot do, and a realistic appraisal of risks and benefits, may help in adoption and democratize access to AI for healthcare.

ASAS-NANP symposium: mathematical modeling in animal nutrition: the progression of data analytics and artificial intelligence in support of sustainable development in animal science

A renewed interest in data analytics and decision support systems in developing automated computer systems is facilitating the emergence of hybrid intelligent systems by combining artificial intelligence (AI) algorithms with classical modeling paradigms such as mechanistic modeling (HIMM) and agent-based models (iABM). Data analytics have evolved remarkably, and the scientific community may not yet fully grasp the power and limitations of some tools. Existing statistical assumptions might need to be re-assessed to provide a more thorough competitive advantage in animal production systems towards sustainability. This paper discussed the evolution of data analytics from a competitive advantage perspective within academia and illustrated the combination of different advanced technological systems in developing HIMM. The progress of analytical tools was divided into three stages: collect and respond, predict and prescribe, and smart learning and policy making, depending on the level of their sophistication (simple to complicated analysis). The collect and respond stage is responsible for ensuring the data is correct and free of influential data points, and it represents the data and information phases for which data are cataloged and organized. The predict and prescribe stage results in gained knowledge from the data and comprises most predictive modeling paradigms, and optimization and risk assessment tools are used to prescribe future decision-making opportunities. The third stage aims to apply the information obtained in the previous stages to foment knowledge and use it for rational decisions. This stage represents the pinnacle of acquired knowledge that leads to wisdom, and AI technology is intrinsic. Although still incipient, HIMM and iABM form the forthcoming stage of competitive advantage. HIMM may not increase our ability to understand the underlying mechanisms controlling the outcomes of a system, but it may increase the predictive ability of existing models by helping the analyst explain more of the data variation. The scientific community still has some issues to be resolved, including the lack of transparency and reporting of AI that might limit code reproducibility. It might be prudent for the scientific community to avoid the shiny object syndrome (i.e., AI) and look beyond the current knowledge to understand the mechanisms that might improve productivity and efficiency to lead agriculture towards sustainable and responsible achievements.

The Effectiveness of the Digital Environment and Perfectionism on Anxiety and Depression in the Light of the COVID-19 Pandemic in Northern Iraq

This investigation is intended to explore the effect of the digital environment as well as perfectionism during the COVID-19 pandemic on anxiety and depression. The study used a mixed-methods design; a mixed research methodology was used regarding explanatory design by using a qualitative sub-sample from quantitative sample data. The researcher conducted the study on a sample of 980 students and non-students smartphone and internet users using both qualitative (self-reported) and quantitative (questionnaires) approach. Three different questionnaires were used: The Multidimensional Perfectionism Scale, Beck Depression (BD), and Beck Anxiety Inventory (BA). On the other hand, the researcher designed a self-reported interview for the qualitative part that included 5 major questions and 10 sub-questions. The gathered data were investigated using SPSS version 22 to analyze the collected data for this study, and simple descriptive statistics and coding were used. The results denote that the digital environment is significantly correlated with multidimensional perfectionism in a low positive manner for students. However, the correlation effects for non-students are significantly high as evidenced by a significant positive correlation. Another finding suggests that there is a positive significant association between perfectionism and being depressed and anxious.

Machine Learning for Healthcare Wearable Devices: The Big Picture

Using artificial intelligence and machine learning techniques in healthcare applications has been actively researched over the last few years. It holds promising opportunities as it is used to track human activities and vital signs using wearable devices and assist in diseases' diagnosis, and it can play a great role in elderly care and patient's health monitoring and diagnostics. With the great technological advances in medical sensors and miniaturization of electronic chips in the recent five years, more applications are being researched and developed for wearable devices. Despite the remarkable growth of using smart watches and other wearable devices, a few of these massive research efforts for machine learning applications have found their way to market. In this study, a review of the different areas of the recent machine learning research for healthcare wearable devices is presented. Different challenges facing machine learning applications on wearable devices are discussed. Potential solutions from the literature are presented, and areas open for improvement and further research are highlighted.

Intelligent Bio-Impedance System for Personalized Continuous Blood Pressure Measurement

Continuous blood pressure (BP) measurement is crucial for long-term cardiovascular monitoring, especially for prompt hypertension detection. However, most of the continuous BP measurements rely on the pulse transit time (PTT) from multiple-channel physiological acquisition systems that impede wearable applications. Recently, wearable and smart health electronics have become significant for next-generation personalized healthcare progress. This study proposes an intelligent single-channel bio-impedance system for personalized BP monitoring. Compared to the PTT-based methods, the proposed sensing configuration greatly reduces the hardware complexity, which is beneficial for wearable applications. Most of all, the proposed system can extract the significant BP features hidden from the measured bio-impedance signals by an ultra-lightweight AI algorithm, implemented to further establish a tailored BP model for personalized healthcare. In the human trial, the proposed system demonstrates the BP accuracy in terms of the mean error (ME) and the mean absolute error (MAE) within 1.7 ± 3.4 mmHg and 2.7 ± 2.6 mmHg, respectively, which agrees with the criteria of the Association for the Advancement of Medical Instrumentation (AAMI). In conclusion, this work presents a proof-of-concept for an AI-based single-channel bio-impedance BP system. The new wearable smart system is expected to accelerate the artificial intelligence of things (AIoT) technology for personalized BP healthcare in the future.

An Artificial Neural Network Model for Pediatric Mortality Prediction in Two Tertiary Pediatric Intensive Care Units in South Africa. A Development Study

OBJECTIVES

The performance of mortality prediction models remain a challenge in lower- and middle-income countries. We developed an artificial neural network (ANN) model for the prediction of mortality in two tertiary pediatric intensive care units (PICUs) in South Africa using free to download and use software and commercially available computers. These models were compared to a logistic regression model and a recalibrated version of the Pediatric Index of Mortality 3.

DESIGN

This study used data from a retrospective cohort study to develop an artificial neural model and logistic regression model for mortality prediction. The outcome evaluated was death in PICU.

SETTING

Two tertiary PICUs in South Africa.

PATIENTS

2,089 patients up to the age of 13 completed years were included in the study.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The AUROC was higher for the ANN (0.89) than for the logistic regression model (LR) (0.87) and the recalibrated PIM3 model (0.86). The precision recall curve however favors the ANN over logistic regression and recalibrated PIM3 (AUPRC = 0.6 vs. 0.53 and 0.58, respectively. The slope of the calibration curve was 1.12 for the ANN model (intercept 0.01), 1.09 for the logistic regression model (intercept 0.05) and 1.02 (intercept 0.01) for the recalibrated version of PIM3. The calibration curve was however closer to the diagonal for the ANN model.

CONCLUSIONS

Artificial neural network models are a feasible method for mortality prediction in lower- and middle-income countries but significant challenges exist. There is a need to conduct research directed toward the acquisition of large, complex data sets, the integration of documented clinical care into clinical research and the promotion of the development of electronic health record systems in lower and middle income settings.

Agreement and Reliability Analysis of Machine Learning Scaling and Wireless Monitoring in the Assessment of Acute Proximal Weakness by Experts and Non-Experts: A Feasibility Study

Assessing the symptoms of proximal weakness caused by neurological deficits requires the knowledge and experience of neurologists. Recent advances in machine learning and the Internet of Things have resulted in the development of automated systems that emulate physicians’ assessments. The application of those systems requires not only accuracy in the classification but also reliability regardless of users’ proficiency in the real environment for the clinical point-of-care and the personalized health management. This study provides an agreement and reliability analysis of using a machine learning-based scaling of Medical Research Council (MRC) proximal scores to evaluate proximal weakness by experts and non-experts. The system trains an ensemble learning model using the signals from sensors attached to the limbs of patients in a neurological intensive care unit. For the agreement analysis, we investigated the percent agreement of MRC proximal scores and Bland-Altman plots of kinematic features between the expert- and non-expert scaling. We also analyzed the intra-class correlation coefficients (ICCs) of kinematic features and Krippendorff’s alpha of the observers’ scaling for the reliability analysis. The mean percent agreement between the expert- and the non-expert scaling was 0.542 for manual scaling and 0.708 for autonomous scaling. The ICCs of kinematic features measured using sensors ranged from 0.742 to 0.850, whereas the Krippendorff’s alpha of manual scaling for the three observers was 0.275. The autonomous assessment system can be utilized by the caregivers, paramedics, or other observers during an emergency to evaluate acute stroke patients.

Public Covid-19 X-ray datasets and their impact on model bias - A systematic review of a significant problem

Computer-aided-diagnosis and stratification of COVID-19 based on chest X-ray suffers from weak bias assessment and limited quality-control. Undetected bias induced by inappropriate use of datasets, and improper consideration of confounders prevents the translation of prediction models into clinical practice. By adopting established tools for model evaluation to the task of evaluating datasets, this study provides a systematic appraisal of publicly available COVID-19 chest X-ray datasets, determining their potential use and evaluating potential sources of bias. Only 9 out of more than a hundred identified datasets met at least the criteria for proper assessment of risk of bias and could be analysed in detail. Remarkably most of the datasets utilised in 201 papers published in peer-reviewed journals, are not among these 9 datasets, thus leading to models with high risk of bias. This raises concerns about the suitability of such models for clinical use. This systematic review highlights the limited description of datasets employed for modelling and aids researchers to select the most suitable datasets for their task.

Prevalence and predicting factors of perceived stress among Bangladeshi university students using machine learning algorithms

Background

Stress-related mental health problems are one of the most common causes of the burden in university students worldwide. Many studies have been conducted to predict the prevalence of stress among university students, however most of these analyses were predominantly performed using the basic logistic regression (LR) model. As an alternative, we used the advanced machine learning (ML) approaches for detecting significant risk factors and to predict the prevalence of stress among Bangladeshi university students.

Methods

This prevalence study surveyed 355 students from twenty-eight different Bangladeshi universities using questions concerning anthropometric measurements, academic, lifestyles, and health-related information, which referred to the perceived stress status of the respondents (yes or no). Boruta algorithm was used in determining the significant prognostic factors of the prevalence of stress. Prediction models were built using decision tree (DT), random forest (RF), support vector machine (SVM), and LR, and their performances were evaluated using parameters of confusion matrix, receiver operating characteristics (ROC) curves, and k-fold cross-validation techniques.

Results

One-third of university students reported stress within the last 12 months. Students’ pulse rate, systolic and diastolic blood pressures, sleep status, smoking status, and academic background were selected as the important features for predicting the prevalence of stress. Evaluated performance revealed that the highest performance observed from RF (accuracy = 0.8972, precision = 0.9241, sensitivity = 0.9250, specificity = 0.8148, area under the ROC curve (AUC) = 0.8715, k-fold accuracy = 0.8983) and the lowest from LR (accuracy = 0.7476, precision = 0.8354, sensitivity = 0.8250, specificity = 0.5185, AUC = 0.7822, k-fold accuracy = 07713) and SVM with polynomial kernel of degree 2 (accuracy = 0.7570, precision = 0.7975, sensitivity = 0.8630, specificity = 0.5294, AUC = 0.7717, k-fold accuracy = 0.7855). Overall, the RF model performs better and authentically predicted stress compared with other ML techniques, including individual and interaction effects of predictors.

Conclusion

The machine learning framework can be detected the significant prognostic factors and predicted this psychological problem more accurately, thereby helping the policy-makers, stakeholders, and families to understand and prevent this serious crisis by improving policy-making strategies, mental health promotion, and establishing effective university counseling services.

Multicenter Validation of Individual Preoperative Motor Outcome Prediction for Deep Brain Stimulation in Parkinson's Disease

BACKGROUND

Subthalamic nucleus deep brain stimulation (STN DBS) is an established therapy for Parkinson's disease (PD) patients suffering from motor response fluctuations despite optimal medical treatment, or severe dopaminergic side effects. Despite careful clinical selection and surgical procedures, some patients do not benefit from STN DBS. Preoperative prediction models are suggested to better predict individual motor response after STN DBS. We validate a preregistered model, DBS-PREDICT, in an external multicenter validation cohort.

METHODS

DBS-PREDICT considered eleven, solely preoperative, clinical characteristics and applied a logistic regression to differentiate between weak and strong motor responders. Weak motor response was defined as no clinically relevant improvement on the Unified Parkinson's Disease Rating Scale (UPDRS) II, III, or IV, 1 year after surgery, defined as, respectively, 3, 5, and 3 points or more. Lower UPDRS III and IV scores and higher age at disease onset contributed most to weak response predictions. Individual predictions were compared with actual clinical outcomes.

RESULTS

322 PD patients treated with STN DBS from 6 different centers were included. DBS-PREDICT differentiated between weak and strong motor responders with an area under the receiver operator curve of 0.76 and an accuracy up to 77%.

CONCLUSION

Proving generalizability and feasibility of preoperative STN DBS outcome prediction in an external multicenter cohort is an important step in creating clinical impact in DBS with data-driven tools. Future prospective studies are required to overcome several inherent practical and statistical limitations of including clinical decision support systems in DBS care.

Can Autism Be Diagnosed with Artificial Intelligence? A Narrative Review

Radiomics with deep learning models have become popular in computer-aided diagnosis and have outperformed human experts on many clinical tasks. Specifically, radiomic models based on artificial intelligence (AI) are using medical data (i.e., images, molecular data, clinical variables, etc.) for predicting clinical tasks such as autism spectrum disorder (ASD). In this review, we summarized and discussed the radiomic techniques used for ASD analysis. Currently, the limited radiomic work of ASD is related to the variation of morphological features of brain thickness that is different from texture analysis. These techniques are based on imaging shape features that can be used with predictive models for predicting ASD. This review explores the progress of ASD-based radiomics with a brief description of ASD and the current non-invasive technique used to classify between ASD and healthy control (HC) subjects. With AI, new radiomic models using the deep learning techniques will be also described. To consider the texture analysis with deep CNNs, more investigations are suggested to be integrated with additional validation steps on various MRI sites.

Improving the compromise between accuracy, interpretability and personalization of rule-based machine learning in medical problems

One of the key challenges when developing a predictive model is the capability to describe the domain knowledge and the cause-effect relationships in a simple way. Decision rules are a useful and important methodology in this context, justifying their application in several areas, particularly in clinical practice. Several machine-learning classifiers have exploited the advantageous properties of decision rules to build intelligent prediction models, namely decision trees and ensembles of trees (ETs). However, such methodologies usually suffer from a trade-off between interpretability and predictive performance. Some procedures consider a simplification of ETs, using heuristic approaches to select an optimal reduced set of decision rules. In this paper, we introduce a novel step to those methodologies. We create a new component to predict if a given rule will be correct or not for a particular patient, which introduces personalization into the procedure. Furthermore, the validation results using three public clinical datasets suggest that it also allows to increase the predictive performance of the selected set of rules, improving the mentioned trade-off.

Public Covid-19 X-ray datasets and their impact on model bias - A systematic review of a significant problem

Computer-aided-diagnosis and stratification of COVID-19 based on chest X-ray suffers from weak bias assessment and limited quality-control. Undetected bias induced by inappropriate use of datasets, and improper consideration of confounders prevents the translation of prediction models into clinical practice. By adopting established tools for model evaluation to the task of evaluating datasets, this study provides a systematic appraisal of publicly available COVID-19 chest X-ray datasets, determining their potential use and evaluating potential sources of bias. Only 9 out of more than a hundred identified datasets met at least the criteria for proper assessment of risk of bias and could be analysed in detail. Remarkably most of the datasets utilised in 201 papers published in peer-reviewed journals, are not among these 9 datasets, thus leading to models with high risk of bias. This raises concerns about the suitability of such models for clinical use. This systematic review highlights the limited description of datasets employed for modelling and aids researchers to select the most suitable datasets for their task.

Systematic Review of the Effectiveness of Machine Learning Algorithms for Classifying Pain Intensity, Phenotype or Treatment Outcomes Using Electroencephalogram Data

Recent attempts to utilize machine learning (ML) to predict pain-related outcomes from Electroencephalogram (EEG) data demonstrate promising results. The primary aim of this review was to evaluate the effectiveness of ML algorithms for predicting pain intensity, phenotypes or treatment response from EEG. Electronic databases MEDLINE, EMBASE, Web of Science, PsycINFO and The Cochrane Library were searched. A total of 44 eligible studies were identified, with 22 presenting attempts to predict pain intensity, 15 investigating the prediction of pain phenotypes and seven assessing the prediction of treatment response. A meta-analysis was not considered appropriate for this review due to heterogenos methods and reporting. Consequently, data were narratively synthesized. The results demonstrate that the best performing model of the individual studies allows for the prediction of pain intensity, phenotypes and treatment response with accuracies ranging between 62 to 100%, 57 to 99% and 65 to 95.24%, respectively. The results suggest that ML has the potential to effectively predict pain outcomes, which may eventually be used to assist clinical care. However, inadequate reporting and potential bias reduce confidence in the results. Future research should improve reporting standards and externally validate models to decrease bias, which would increase the feasibility of clinical translation. PERSPECTIVE: This systematic review explores the state-of-the-art machine learning methods for predicting pain intensity, phenotype or treatmentresponse from EEG data. Results suggest that machine learning may demonstrate clinical utility, pending further research and development. Areas for improvement, including standardized processing, reporting and the need for better methodological assessment tools, are discussed.

PassFlow: a multimodal workflow for predicting deep brain stimulation outcomes

PURPOSE

Deep Brain Stimulation (DBS) is a proven therapy for Parkinson's Disease (PD), frequently resulting in an enhancement of motor function. Nonetheless, several undesirable side effects can occur after DBS, which can worsen the quality of life of the patient. Thus, the clinical team has to carefully select patients on whom to perform DBS. Over the past decade, there have been some attempts to relate pre-operative data and DBS clinical outcomes, with most focused on the motor symptomatology. In this paper, we propose a machine learning-based method able to predict a large number of DBS clinical outcomes for PD.

METHODS

We propose a multimodal pipeline, referred to as PassFlow, which predicts 84 clinical post-operative clinical scores. PassFlow is composed of an artificial neural network to compress clinical information, an image processing method from the state-of-the-art to extract morphological biomarkers our of T1 imaging, and an SVM to perform the regressions. We validated PassFlow on 196 PD patients who undergone a DBS.

RESULTS

PassFlow showed correlation coefficients as high as 0.71 and were able to significantly predict 63 out of the 84 scores, outperforming a comparative linear method. The number of metrics that are predicted with this pre-operative information was also found to be correlated with the number of patients with this information available, indicating that the PassFlow method is still actively learning.

CONCLUSION

We presented a novel, machine learning-based pipeline to predict a variety of post-operative clinical outcomes of DBS for PD patients. PassFlow took into account various bio-markers, arising from different data modalities, showing high correlation coefficients for some scores from pre-operative data only. It indicates that many clinical outcomes of DBS can be predicted agnostic to the specific simulation parameters, as PassFlow has been validated without such stimulation-related information.

Machine Learning Use for Prognostic Purposes in Multiple Sclerosis

The course of multiple sclerosis begins with a relapsing-remitting phase, which evolves into a secondarily progressive form over an extremely variable period, depending on many factors, each with a subtle influence. To date, no prognostic factors or risk score have been validated to predict disease course in single individuals. This is increasingly frustrating, since several treatments can prevent relapses and slow progression, even for a long time, although the possible adverse effects are relevant, in particular for the more effective drugs. An early prediction of disease course would allow differentiation of the treatment based on the expected aggressiveness of the disease, reserving high-impact therapies for patients at greater risk. To increase prognostic capacity, approaches based on machine learning (ML) algorithms are being attempted, given the failure of other approaches. Here we review recent studies that have used clinical data, alone or with other types of data, to derive prognostic models. Several algorithms that have been used and compared are described. Although no study has proposed a clinically usable model, knowledge is building up and in the future strong tools are likely to emerge.