The need to separate the wheat from the chaff in medical informatics: Introducing a comprehensive checklist for the (self)-assessment of medical AI studies

Machine learning for predicting duration of surgery and length of stay: A literature review on joint arthroplasty

INTRODUCTION

In recent years, different factors such as population aging have caused escalating demand for hip and knee arthroplasty straining already limited hospitals' resources. To address this challenge, focus is put on medical and operational efficiency improvements. This includes an increased use of machine learning (ML) to predict duration of surgery (DOS) and length of stay (LOS) for total knee and total hip arthroplasty, which can be utilized for optimizing resource allocation to satisfy medical and operational limitations. This paper explores the development and performance of ML models in predicting DOS and LOS.

METHODS

A systematic search of publications between 2010-2023 was conducted following PRISMA guidelines. Considering the inclusion and exclusion criteria, 28 out of 722 gathered papers from PubMed, Web of Science, and manual search were included in the study. Descriptive statistics was used to analyze the extracted data regarding data preprocessing, model development, and model performance assessment.

RESULTS

Most of the papers work on LOS as a binary variable. Patient's age was identified as the most frequently used and reported as important variable for predicting DOS and LOS. Investigations also illustrated that within the resulting 28 papers, more than 71% of models reached good to perfect performance based on the area under the receiver operating characteristic curve (AUC), where artificial neural networks and ensemble learning models had the biggest share among the best-performing models.

CONCLUSION

The utilization of ML models is increasing in the literature. The current performance level indicates that ML can potentially turn to powerful tools in predicting DOS and LOS for different purposes. Meanwhile, the literature is not matured yet in reporting real-life application. Future studies can focus on model specification and validation by considering empirical application.

Exploring the Trade-Off between generalist and specialized Models: A center-based comparative analysis for glioblastoma segmentation

INTRODUCTION

Inherent variations between inter-center data can undermine the robustness of segmentation models when applied at a specific center (dataset shift). We investigated whether specialized center-specific models are more effective compared to generalist models based on multi-center data, and how center-specific data could enhance the performance of generalist models within a particular center using a fine-tuning transfer learning approach. For this purpose, we studied the dataset shift at center level and conducted a comparative analysis to assess the impact of data source on glioblastoma segmentation models.

METHODS & MATERIALS

The three key components of dataset shift were studied: prior probability shift-variations in tumor size or tissue distribution among centers; covariate shift-inter-center MRI alterations; and concept shift-different criteria for tumor segmentation. BraTS 2021 dataset was used, which includes 1251 cases from 23 centers. Thereafter, 155 deep-learning models were developed and compared, including 1) generalist models trained with multi-center data, 2) specialized models using only center-specific data, and 3) fine-tuned generalist models using center-specific data.

RESULTS

The three key components of dataset shift were characterized. The amount of covariate shift was substantial, indicating large variations in MR imaging between different centers. Glioblastoma segmentation models tend to perform best when using data from the application center. Generalist models, trained with over 700 samples, achieved a median Dice score of 88.98%. Specialized models surpassed this with 200 cases, while fine-tuned models outperformed with 50 cases.

CONCLUSIONS

The influence of dataset shift on model performance is evident. Fine-tuned and specialized models, utilizing data from the evaluated center, outperform generalist models, which rely on data from other centers. These approaches could encourage medical centers to develop customized models for their local use, enhancing the accuracy and reliability of glioblastoma segmentation in a context where dataset shift is inevitable.

Examining individual and contextual predictors of disability in Chinese older adults: A machine learning approach

BACKGROUND

There is a large gap of understanding the determinants of disability, especially the contextual characteristics. Therefore, this study aimed to examine the important predictors of disability in Chinese older adults based on the social ecological framework.

METHODS

We used the China Health and Retirement Longitudinal Study to examine predictors of disability, defined as self-report of any difficulty for six activity of daily living items. We restricted analytical sample to older adults aged 65 or above (N=1816). We considered 44 predictors, including personal-, behavioral-, interpersonal-, community-, and policy-level characteristics. The built-in variable importance measure (VIM) of random forest and SHapley Additive exPlanations (SHAP) were applied to assess key predictors of disability. A multilevel logit regression was further used to examine the associations of individual and contextual characteristics with disability.

RESULTS

The mean age of study sample was 72.62 years old (standard deviation: 5.77). During a 2-year of follow-up, 518 (28.52 %) of them developed into disability. Walking speed, age, and peak expiratory flow were the top important predictors in both VIM and SHAP. Contextual characteristics such as humidity, PM2.5, temperature, normalized difference vegetation index, and landscape also showed promise in predicting disability. Multilevel logit regression showed that people with male gender, arthritis, vision impairment, unable to finish semi tandem, no social activity, lower grip strength, and higher waist circumference, had much higher risk of disability.

CONCLUSION

Disability prevention strategies should specifically focus on multilevel factors such as individual and contextual characteristics, although the latter is warranted to be verified in future studies.

Machine learning-based prediction of sarcopenia in community-dwelling middle-aged and older adults: findings from the CHARLS

BACKGROUND

Sarcopenia is a prominent issue among aging populations and associated with poor health outcomes. This study aimed to examine the predictive value of questionnaire and biomarker data for sarcopenia, and to further develop a user-friendly calculator for community-dwelling middle-aged and older adults.

METHODS

We used two waves (2011 and 2013) of the China Health and Retirement Longitudinal Study (CHARLS) to predict sarcopenia, defined by the Asian Working Group for Sarcopenia 2019 criteria. We restricted the analytical sample to adults aged 45 or above (N = 2934). Five machine learning models were used to construct Q-based (only questionnaire variables), Bio-based (only biomarker variables), and combined (questionnaire plus biomarker variables) models. Area under the receiver operating characteristic curve (AUROC) was used for performance assessment. Temporal external validation was performed based on two datasets from CHARLS. Important predictors were identified by Shapley values and coefficients.

RESULTS

Extreme gradient boosting (XGBoost), considering both questionnaire and biomarker characteristics, emerged as the optimal model, and its AUROC was 0.759 (95% CI: 0.747-0.771) at a decision threshold of 0.20 on the test set. Models also performed well on the external datasets. We found that cognitive function was the most important predictor in both Q-based and combined models, and blood urea nitrogen was the most important predictor in the Bio-based model. Other key predictors included education, haematocrit, total cholesterol, drinking, number of chronic diseases, and instrumental activities of daily living score.

CONCLUSIONS

Our findings offer a potential for early screening and targeted prevention of sarcopenia among middle-aged and older adults in the community setting.

Machine learning models to further identify advantaged populations that can achieve functional cure of chronic hepatitis B virus infection after receiving Peg-IFN alpha treatment

OBJECTIVE

Functional cure is currently the highest goal of hepatitis B virus(HBV) treatment.Pegylated interferon(Peg-IFN) alpha is an important drug for this purpose,but even in the hepatitis B e antigen(HBeAg)-negative population,there is still a portion of the population respond poorly to it.Therefore,it is important to explore the influencing factors affecting the response rate of Peg-IFN alpha and establish a prediction model to further identify advantaged populations.

METHODS

We retrospectively analyzed 382 patients.297 patients were in the training set and 85 patients from another hospital were in the test set.The intersect features were extracted from all variables using the recursive feature elimination(RFE) algorithm, Boruta algorithm, and least absolute shrinkage and selection operator(LASSO) regression algorithm in the training dataset.Then,we employed six machine learning(ML) algorithms-Logistic Regression(LR),Random Forest(RF),Support Vector Machines(SVM),K Nearest Neighbors(KNN),Light Gradient Boosting Machine(LightGBM) and Extreme Gradient Boosting(XGBoost)-to develop the model.Internal 10-fold cross-validation helped determine the best-performing model,which was then tested externally.Model performance was assessed using metrics such as area under the curve(AUC) and other metrics.SHapley Additive exPlanations(SHAP) plots were used to interpret variable significance.

RESULTS

138/382(36.13 %) patients achieved functional cure.HBsAg at baseline,HBsAg decline at week12,non-alcoholic fatty liver disease(NAFLD) and age were identified as significant variables.RF performed the best,with AUC value of 0.988,and maintained good performance in test set.The SHapley Additive exPlanations(SHAP) plot highlighted HBsAg at baseline and HBsAg decline at week 12 are the top two predictors.The web-calculator was designed to predict functional cure more conveniently(https://www.xsmartanalysis.com/model/list/predict/model/html?mid = 17054&symbol = 317ad245Hx628ko3uW51).

CONCLUSION

We developed a prediction model,which can be used to not only accurately identifies advantageous populations with Peg-IFN alpha,but also determines whether to continue subsequent Peg-IFN alpha.

Machine Learning for Treatment Management Prediction in Laryngeal Fractures

OBJECTIVES

Laryngeal fractures are rare but potentially life-threatening traumas. Complications, such as airway obstruction and disrupted laryngeal anatomy, associate with significant morbidity. Early identification of at-risk patients and optimal management remain crucial for improved outcomes. Recently, machine learning (ML) has attained great attention as a unique and novel technique for evaluating complex nonlinear relationships between multiple observations to create a predictive model with greater accuracy. This study aimed to demonstrate the potential of ML in predicting airway and surgical management of laryngeal fracture patients and identify key contributing parameters for the predictive performance of the ML models.

METHODS

The ML models were developed using a patient series managed at the Helsinki University Hospital during 2005-2019. The developed models were further evaluated independently using a different cohort collected from the same institution between 1995 and 2004.

RESULTS

The ML showed a weighted area under curve (AUC) of 0.93 and accuracy of 0.86 following training for airway management. Likewise, for treatment approach, weighted AUC was 0.85 and accuracy 0.78. Injury type, Schaefer-Fuhrman grade (SF gr), age at incident, cause of injury, and fracture of the cricoid, in decreasing order of significance, were the most prominent features for the model's predictive performance for airway management. Similarly, our model identified SF gr, fracture of the cricoid, injury type, age at incident, and cause of injury as the most significant predictors for surgical treatment approach.

CONCLUSIONS

The proposed prediction of management approach by an ML technique can provide accurate predictions and thus aid clinicians in administering early and personalized interventions. The model may serve as a supporting tool in recognizing at-risk patients and in timely decision-making. Further independent external validation is warranted for model generalizability.

Development and validation of interpretable machine learning models to predict glomerular filtration rate in chronic kidney disease Colombian patients

BACKGROUND

ML predictive models have shown their capability to improve risk prediction and assist medical decision-making, nevertheless, there is a lack of accuracy systems to early identify future rapid CKD progressors in Colombia and even in South America.

OBJECTIVE

The purpose of this study was to develop a series of interpretable machine learning models that predict GFR at 6-months, 9-months, and 12-months.

STUDY DESIGN AND SETTING

Over 29,000 CKD patients stage 1 to 3b (estimated GFR, <60 mL/min/1.73 m2) with an average of 3-year follow-up data were included. We used the machine learning extreme gradient boosting (XGBoost) to build three models to predict the next eGFR. Models were internally and externally validated. In addition, we included SHapley Additive exPlanation (SHAP) values to offer interpretable global and local prediction models.

RESULTS

All models showed a good performance in development and external validation. However, the 6-months XGBoost prediction model showed the best performance in internal (MAE average = 6.07; RSME = 78.87), and in external validation (MAE average = 6.45, RSME = 18.94). The top 3 most influential features that pushed the predicted eGFR value to lower values were the interpolated values for eGFR and creatinine, and eGFR at baseline.

CONCLUSION

In the current study we have developed and validated machine learning models to predict the next eGFR value at different intervals. Furthermore, we attempted to approach the need for prediction explanation by offering transparent predictions.

Ten quick tips for clinical electroencephalographic (EEG) data acquisition and signal processing

Electroencephalography (EEG) is a medical engineering technique aimed at recording the electric activity of the human brain. Brain signals derived from an EEG device can be processed and analyzed through computers by using digital signal processing, computational statistics, and machine learning techniques, that can lead to scientifically-relevant results and outcomes about how the brain works. In the last decades, the spread of EEG devices and the higher availability of EEG data, of computational resources, and of software packages for electroencephalography analysis has made EEG signal processing easier and faster to perform for any researcher worldwide. This increased ease to carry out computational analyses of EEG data, however, has made it easier to make mistakes, as well. And these mistakes, if unnoticed or treated wrongly, can in turn lead to wrong results or misleading outcomes, with worrisome consequences for patients and for the advancements of the knowledge about human brain. To tackle this problem, we present here our ten quick tips to perform electroencephalography signal processing analyses avoiding common mistakes: a short list of guidelines designed for beginners on what to do, how to do it, and what not to do when analyzing EEG data with a computer. We believe that following our quick recommendations can lead to better, more reliable and more robust results and outcome in clinical neuroscientific research.

Predictive analysis on the factors associated with birth Outcomes: A machine learning perspective

BACKGROUND

Recent studies reveal that around 1.9 million stillbirths occur annually worldwide, with Sub-Saharan Africa having among the highest cases. Some Sub-Saharan African countries, including Ghana, failed to meet Millennium Development Goal 5 (MDG5) by 2015 and may struggle to meet Sustainable Development Goal 3 (SDG3) despite maternal healthcare interventions. Concerns arise about Ghana's ability to achieve the World Health Organization's neonatal mortality goal of 12 per 1000 live births by 2030. This study aims to identify key factors influencing childbirth outcomes and create a predictive method for high-risk pregnancies.

METHODS

We compared four machine learning classifiers (Extreme Gradient Boosting, Random Forest, Logistic Regression, and Artificial Neural Network) in predicting childbirth outcomes using data from a tertiary health facility in Ghana. To address class imbalance, we employed the Synthetic Minority Over-sampling Technique (SMOTE).

RESULTS

Our findings show that fetal heartbeat, gestation age at birth are the most influential factors on birth outcome (stillbirth or live birth), while there is no significant association with maternal age, number of babies, and type of delivery method. Among the machine learning models considered, Random Forest emerged as the optimal model achieving an accuracy, F1-score, and AUC values of approximately 0.98, 0.99, and 0.90 respectively.

CONCLUSION

Our study identifies key factors affecting childbirth outcomes and highlights the potential of machine learning for early high-risk pregnancy detection in clinical settings. These findings are crucial for Ghana and other Sub-Saharan African countries striving to meet maternal and neonatal healthcare goals. Further research and policy initiatives can use these results to improve healthcare in the region and work toward the World Health Organization's objectives by 2030.

Inflammatory bowel disease genomics, transcriptomics, proteomics and metagenomics meet artificial intelligence

Various extrinsic and intrinsic factors such as drug exposures, antibiotic treatments, smoking, lifestyle, genetics, immune responses, and the gut microbiome characterize ulcerative colitis and Crohn's disease, collectively called inflammatory bowel disease (IBD). All these factors contribute to the complexity and heterogeneity of the disease etiology and pathogenesis leading to major challenges for the scientific community in improving management, medical treatments, genetic risk, and exposome impact. Understanding the interaction(s) among these factors and their effects on the immune system in IBD patients has prompted advances in multi-omics research, the development of new tools as part of system biology, and more recently, artificial intelligence (AI) approaches. These innovative approaches, supported by the availability of big data and large volumes of digital medical datasets, hold promise in better understanding the natural histories, predictors of disease development, severity, complications and treatment outcomes in complex diseases, providing decision support to doctors, and promising to bring us closer to the realization of the "precision medicine" paradigm. This review aims to provide an overview of current IBD omics based on both individual (genomics, transcriptomics, proteomics, metagenomics) and multi-omics levels, highlighting how AI can facilitate the integration of heterogeneous data to summarize our current understanding of the disease and to identify current gaps in knowledge to inform upcoming research in this field.

Analyzing Wav2Vec 1.0 Embeddings for Cross-Database Parkinson's Disease Detection and Speech Features Extraction

Advancements in deep learning speech representations have facilitated the effective use of extensive unlabeled speech datasets for Parkinson's disease (PD) modeling with minimal annotated data. This study employs the non-fine-tuned wav2vec 1.0 architecture to develop machine learning models for PD speech diagnosis tasks, such as cross-database classification and regression to predict demographic and articulation characteristics. The primary aim is to analyze overlapping components within the embeddings on both classification and regression tasks, investigating whether latent speech representations in PD are shared across models, particularly for related tasks. Firstly, evaluation using three multi-language PD datasets showed that wav2vec accurately detected PD based on speech, outperforming feature extraction using mel-frequency cepstral coefficients in the proposed cross-database classification scenarios. In cross-database scenarios using Italian and English-read texts, wav2vec demonstrated performance comparable to intra-dataset evaluations. We also compared our cross-database findings against those of other related studies. Secondly, wav2vec proved effective in regression, modeling various quantitative speech characteristics related to articulation and aging. Ultimately, subsequent analysis of important features examined the presence of significant overlaps between classification and regression models. The feature importance experiments discovered shared features across trained models, with increased sharing for related tasks, further suggesting that wav2vec contributes to improved generalizability. The study proposes wav2vec embeddings as a next promising step toward a speech-based universal model to assist in the evaluation of PD.

Automated Interpretation of Lung Sounds by Deep Learning in Children With Asthma: Scoping Review and Strengths, Weaknesses, Opportunities, and Threats Analysis

BACKGROUND

The interpretation of lung sounds plays a crucial role in the appropriate diagnosis and management of pediatric asthma. Applying artificial intelligence (AI) to this task has the potential to better standardize assessment and may even improve its predictive potential.

OBJECTIVE

This study aims to objectively review the literature on AI-assisted lung auscultation for pediatric asthma and provide a balanced assessment of its strengths, weaknesses, opportunities, and threats.

METHODS

A scoping review on AI-assisted lung sound analysis in children with asthma was conducted across 4 major scientific databases (PubMed, MEDLINE Ovid, Embase, and Web of Science), supplemented by a gray literature search on Google Scholar, to identify relevant studies published from January 1, 2000, until May 23, 2023. The search strategy incorporated a combination of keywords related to AI, pulmonary auscultation, children, and asthma. The quality of eligible studies was assessed using the ChAMAI (Checklist for the Assessment of Medical Artificial Intelligence).

RESULTS

The search identified 7 relevant studies out of 82 (9%) to be included through an academic literature search, while 11 of 250 (4.4%) studies from the gray literature search were considered but not included in the subsequent review and quality assessment. All had poor to medium ChAMAI scores, mostly due to the absence of external validation. Identified strengths were improved predictive accuracy of AI to allow for prompt and early diagnosis, personalized management strategies, and remote monitoring capabilities. Weaknesses were the heterogeneity between studies and the lack of standardization in data collection and interpretation. Opportunities were the potential of coordinated surveillance, growing data sets, and new ways of collaboratively learning from distributed data. Threats were both generic for the field of medical AI (loss of interpretability) but also specific to the use case, as clinicians might lose the skill of auscultation.

CONCLUSIONS

To achieve the opportunities of automated lung auscultation, there is a need to address weaknesses and threats with large-scale coordinated data collection in globally representative populations and leveraging new approaches to collaborative learning.

Evaluating Inflammatory Bowel Disease-Related Quality of Life Using an Interpretable Machine Learning Approach: A Multicenter Study in China

Purpose

Impaired quality of life (QOL) is common in patients with inflammatory bowel disease (IBD). A tool to more quickly identify IBD patients at high risk of impaired QOL improves opportunities for earlier intervention and improves long-term prognosis. The purpose of this study was to use a machine learning (ML) approach to develop risk stratification models for evaluating IBD-related QOL impairments.

Patients and Methods

An online questionnaire was used to collect clinical data on 2478 IBD patients from 42 hospitals distributed across 22 provinces in China from September 2021 to May 2022. Eight ML models used to predict the risk of IBD-related QOL impairments were developed and validated. Model performance was evaluated using a set of indexes and the best ML model was explained using a Local Interpretable Model-Agnostic Explanations (LIME) algorithm.

Results

The support vector machine (SVM) classifier algorithm-based model outperformed other ML models with an area under the receiver operating characteristic curve (AUC) and an accuracy of 0.80 and 0.71, respectively. The feature importance calculated by the SVM classifier algorithm revealed that glucocorticoid use, anxiety, abdominal pain, sleep disorders, and more severe disease contributed to a higher risk of impaired QOL, while longer disease course and the use of biological agents and immunosuppressants were associated with a lower risk.

Conclusion

An ML approach for assessing IBD-related QOL impairments is feasible and effective. This mechanism is a promising tool for gastroenterologists to identify IBD patients at high risk of impaired QOL.

Self-reported checklists and quality scoring tools in radiomics: a meta-research

OBJECTIVE

To evaluate the use of reporting checklists and quality scoring tools for self-reporting purposes in radiomics literature.

METHODS

Literature search was conducted in PubMed (date, April 23, 2023). The radiomics literature was sampled at random after a sample size calculation with a priori power analysis. A systematic assessment for self-reporting, including the use of documentation such as completed checklists or quality scoring tools, was conducted in original research papers. These eligible papers underwent independent evaluation by a panel of nine readers, with three readers assigned to each paper. Automatic annotation was used to assist in this process. Then, a detailed item-by-item confirmation analysis was carried out on papers with checklist documentation, with independent evaluation of two readers.

RESULTS

The sample size calculation yielded 117 papers. Most of the included papers were retrospective (94%; 110/117), single-center (68%; 80/117), based on their private data (89%; 104/117), and lacked external validation (79%; 93/117). Only seven papers (6%) had at least one self-reported document (Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), or Checklist for Artificial Intelligence in Medical Imaging (CLAIM)), with a statistically significant binomial test (p < 0.001). Median rate of confirmed items for all three documents was 81% (interquartile range, 6). For quality scoring tools, documented scores were higher than suggested scores, with a mean difference of - 7.2 (standard deviation, 6.8).

CONCLUSION

Radiomic publications often lack self-reported checklists or quality scoring tools. Even when such documents are provided, it is essential to be cautious, as the accuracy of the reported items or scores may be questionable.

CLINICAL RELEVANCE STATEMENT

Current state of radiomic literature reveals a notable absence of self-reporting with documentation and inaccurate reporting practices. This critical observation may serve as a catalyst for motivating the radiomics community to adopt and utilize such tools appropriately, thereby fostering rigor, transparency, and reproducibility of their research, moving the field forward.

KEY POINTS

• In radiomics literature, there has been a notable absence of self-reporting with documentation. • Even if such documents are provided, it is critical to exercise caution because the accuracy of the reported items or scores may be questionable. • Radiomics community needs to be motivated to adopt and appropriately utilize the reporting checklists and quality scoring tools.

Deep learning-based platform performs high detection sensitivity of intracranial aneurysms in 3D brain TOF-MRA: An external clinical validation study

PURPOSE

To evaluate the diagnostic efficacy of a developed artificial intelligence (AI) platform incorporating deep learning algorithms for the automated detection of intracranial aneurysms in time-of-flight (TOF) magnetic resonance angiography (MRA).

METHOD

This retrospective study encompassed 3D TOF MRA images acquired between January 2023 and June 2023, aiming to validate the presence of intracranial aneurysms via our developed AI platform. The manual segmentation results by experienced neuroradiologists served as the "gold standard". Following annotation of MRA images by neuroradiologists using InferScholar software, the AI platform conducted automatic segmentation of intracranial aneurysms. Various metrics including accuracy (ACC), balanced ACC, area under the curve (AUC), sensitivity (SE), specificity (SP), F1 score, Brier Score, and Net Benefit were utilized to evaluate the generalization of AI platform. Comparison of intracranial aneurysm identification performance was conducted between the AI platform and six radiologists with experience ranging from 3 to 12 years in interpreting MR images. Additionally, a comparative analysis was carried out between radiologists' detection performance based on independent visual diagnosis and AI-assisted diagnosis. Subgroup analyses were also performed based on the size and location of the aneurysms to explore factors impacting aneurysm detectability.

RESULTS

510 patients were enrolled including 215 patients (42.16 %) with intracranial aneurysms and 295 patients (57.84 %) without aneurysms. Compared with six radiologists, the AI platform showed competitive discrimination power (AUC, 0.96), acceptable calibration (Brier Score loss, 0.08), and clinical utility (Net Benefit, 86.96 %). The AI platform demonstrated superior performance in detecting aneurysms with an overall SE, SP, ACC, balanced ACC, and F1 score of 91.63 %, 92.20 %, 91.96 %, 91.92 %, and 90.57 % respectively, outperforming the detectability of the two resident radiologists. For subgroup analysis based on aneurysm size and location, we observed that the SE of the AI platform for identifying tiny (diameter＜3mm), small (3 mm ≤ diameter＜5mm), medium (5 mm ≤ diameter＜7mm) and large aneurysms (diameter ≥ 7 mm) was 87.80 %, 93.14 %, 95.45 %, and 100 %, respectively. Furthermore, the SE for detecting aneurysms in the anterior circulation was higher than that in the posterior circulation. Utilizing the AI assistance, six radiologists (i.e., two residents, two attendings and two professors) achieved statistically significant improvements in mean SE (residents: 71.40 % vs. 88.37 %; attendings: 82.79 % vs. 93.26 %; professors: 90.07 % vs. 97.44 %; P < 0.05) and ACC (residents: 85.29 % vs. 94.12 %; attendings: 91.76 % vs. 97.06 %; professors: 95.29 % vs. 98.82 %; P < 0.05) while no statistically significant change was observed in SP. Overall, radiologists' mean SE increased by 11.40 %, mean SP increased by 1.86 %, and mean ACC increased by 5.88 %, mean balanced ACC promoted by 6.63 %, mean F1 score grew by 7.89 %, and Net Benefit rose by 12.52 %, with a concurrent decrease in mean Brier score declined by 0.06.

CONCLUSIONS

The deep learning algorithms implemented in the AI platform effectively detected intracranial aneurysms on TOF-MRA and notably enhanced the diagnostic capabilities of radiologists. This indicates that the AI-based auxiliary diagnosis model can provide dependable and precise prediction to improve the diagnostic capacity of radiologists.

Exploring Pattern of Relapse in Pediatric Patients with Acute Lymphocytic Leukemia and Acute Myeloid Leukemia Undergoing Stem Cell Transplant Using Machine Learning Methods

Background. Leukemic relapse remains the primary cause of treatment failure and death after allogeneic hematopoietic stem cell transplant. Changes in post-transplant donor chimerism have been identified as a predictor of relapse. A better predictive model of relapse incorporating donor chimerism has the potential to improve leukemia-free survival by allowing earlier initiation of post-transplant treatment on individual patients. We explored the use of machine learning, a suite of analytical methods focusing on pattern recognition, to improve post-transplant relapse prediction. Methods. Using a cohort of 63 pediatric patients with acute lymphocytic leukemia (ALL) and 46 patients with acute myeloid leukemia (AML) who underwent stem cell transplant at a single institution, we built predictive models of leukemic relapse with both pre-transplant and post-transplant patient variables (specifically lineage-specific chimerism) using the random forest classifier. Local Interpretable Model-Agnostic Explanations, an interpretable machine learning tool was used to confirm our random forest classification result. Results. Our analysis showed that a random forest model using these hyperparameter values achieved 85% accuracy, 85% sensitivity, 89% specificity for ALL, while for AML 81% accuracy, 75% sensitivity, and 100% specificity at predicting relapses within 24 months post-HSCT in cross validation. The Local Interpretable Model-Agnostic Explanations tool was able to confirm many variables that the random forest classifier identified as important for the relapse prediction. Conclusions. Machine learning methods can reveal the interaction of different risk factors of post-transplant leukemic relapse and robust predictions can be obtained even with a modest clinical dataset. The random forest classifier distinguished different important predictive factors between ALL and AML in our relapse models, consistent with previous knowledge, lending increased confidence to adopting machine learning prediction to clinical management.

Good machine learning practices: Learnings from the modern pharmaceutical discovery enterprise

Machine Learning (ML) and Artificial Intelligence (AI) have become an integral part of the drug discovery and development value chain. Many teams in the pharmaceutical industry nevertheless report the challenges associated with the timely, cost effective and meaningful delivery of ML and AI powered solutions for their scientists. We sought to better understand what these challenges were and how to overcome them by performing an industry wide assessment of the practices in AI and Machine Learning. Here we report results of the systematic business analysis of the personas in the modern pharmaceutical discovery enterprise in relation to their work with the AI and ML technologies. We identify 23 common business problems that individuals in these roles face when they encounter AI and ML technologies at work, and describe best practices (Good Machine Learning Practices) that address these issues.

International Variation in Severe Exacerbation Rates in Patients With Severe Asthma

BACKGROUND

Exacerbation frequency strongly influences treatment choices in patients with severe asthma.

RESEARCH QUESTION

What is the extent of the variability of exacerbation rate across countries and its implications in disease management?

STUDY DESIGN AND METHODS

We retrieved data from the International Severe Asthma Registry, an international observational cohort of patients with a clinical diagnosis of severe asthma. We identified patients aged ≥ 18 years who did not initiate any biologics prior to baseline visit. A severe exacerbation was defined as the use of oral corticosteroids for ≥ 3 days or asthma-related hospitalization/ED visit. A series of negative binomial models were applied to estimate country-specific severe exacerbation rates during 365 days of follow-up, starting from a naive model with country as the only variable to an adjusted model with country as a random-effect term and patient and disease characteristics as independent variables.

RESULTS

The final sample included 7,510 patients from 17 countries (56% from the United States), contributing to 1,939 severe exacerbations (0.27/person-year). There was large between-country variation in observed severe exacerbation rate (minimum, 0.04 [Argentina]; maximum, 0.88 [Saudi Arabia]; interquartile range, 0.13-0.54), which remained substantial after adjusting for patient characteristics and sampling variability (interquartile range, 0.16-0.39).

INTERPRETATION

Individuals with similar patient characteristics but coming from different jurisdictions have varied severe exacerbation risks, even after controlling for patient and disease characteristics. This suggests unknown patient factors or system-level variations at play. Disease management guidelines should recognize such between-country variability. Risk prediction models that are calibrated for each jurisdiction will be needed to optimize treatment strategies.

Constructing and implementing a performance evaluation indicator set for artificial intelligence decision support systems in pediatric outpatient clinics: an observational study

Artificial intelligence (AI) decision support systems in pediatric healthcare have a complex application background. As an AI decision support system (AI-DSS) can be costly, once applied, it is crucial to focus on its performance, interpret its success, and then monitor and update it to ensure ongoing success consistently. Therefore, a set of evaluation indicators was explicitly developed for AI-DSS in pediatric healthcare, enabling continuous and systematic performance monitoring. The study unfolded in two stages. The first stage encompassed establishing the evaluation indicator set through a literature review, a focus group interview, and expert consultation using the Delphi method. In the second stage, weight analysis was conducted. Subjective weights were calculated based on expert opinions through analytic hierarchy process, while objective weights were determined using the entropy weight method. Subsequently, subject and object weights were synthesized to form the combined weight. In the two rounds of expert consultation, the authority coefficients were 0.834 and 0.846, Kendall's coordination coefficient was 0.135 in Round 1 and 0.312 in Round 2. The final evaluation indicator set has three first-class indicators, fifteen second-class indicators, and forty-seven third-class indicators. Indicator I-1(Organizational performance) carries the highest weight, followed by Indicator I-2(Societal performance) and Indicator I-3(User experience performance) in the objective and combined weights. Conversely, 'Societal performance' holds the most weight among the subjective weights, followed by 'Organizational performance' and 'User experience performance'. In this study, a comprehensive and specialized set of evaluation indicators for the AI-DSS in the pediatric outpatient clinic was established, and then implemented. Continuous evaluation still requires long-term data collection to optimize the weight proportions of the established indicators.

"Using network analysis modularity to group health code systems and decrease dimensionality in machine learning models"

Background

Machine learning (ML) prediction models in healthcare and pharmacy-related research face challenges with encoding high-dimensional Healthcare Coding Systems (HCSs) such as ICD, ATC, and DRG codes, given the trade-off between reducing model dimensionality and minimizing information loss.

Objectives

To investigate using Network Analysis modularity as a method to group HCSs to improve encoding in ML models.

Methods

The MIMIC-III dataset was utilized to create a multimorbidity network in which ICD-9 codes are the nodes and the edges are the number of patients sharing the same ICD-9 code pairs. A modularity detection algorithm was applied using different resolution thresholds to generate 6 sets of modules. The impact of four grouping strategies on the performance of predicting 90-day Intensive Care Unit readmissions was assessed. The grouping strategies compared: 1) binary encoding of codes, 2) encoding codes grouped by network modules, 3) grouping codes to the highest level of ICD-9 hierarchy, and 4) grouping using the single-level Clinical Classification Software (CCS). The same methodology was also applied to encode DRG codes but limiting the comparison to a single modularity threshold to binary encoding.The performance was assessed using Logistic Regression, Support Vector Machine with a non-linear kernel, and Gradient Boosting Machines algorithms. Accuracy, Precision, Recall, AUC, and F1-score with 95% confidence intervals were reported.

Results

Models utilized modularity encoding outperformed ungrouped codes binary encoding models. The accuracy improved across all algorithms ranging from 0.736 to 0.78 for the modularity encoding, to 0.727 to 0.779 for binary encoding. AUC, recall, and precision also improved across almost all algorithms. In comparison with other grouping approaches, modularity encoding generally showed slightly higher performance in AUC, ranging from 0.813 to 0.837, and precision, ranging from 0.752 to 0.782.

Conclusions

Modularity encoding enhances the performance of ML models in pharmacy research by effectively reducing dimensionality and retaining necessary information. Across the three algorithms used, models utilizing modularity encoding showed superior or comparable performance to other encoding approaches. Modularity encoding introduces other advantages such as it can be used for both hierarchical and non-hierarchical HCSs, the approach is clinically relevant, and can enhance ML models' clinical interpretation. A Python package has been developed to facilitate the use of the approach for future research.

Machine learning methods in predicting the risk of malignant transformation of oral potentially malignant disorders: A systematic review

BACKGROUND

Oral Potentially Malignant Disorders (OPMDs) refer to a heterogenous group of clinical presentations with heightened rate of malignant transformation. Identification of risk levels in OPMDs is crucial to determine the need for active intervention in high-risk patients and routine follow-up in low-risk ones. Machine learning models has shown tremendous potential in several areas of dentistry that strongly suggest its application to estimate rate of malignant transformation of precancerous lesions.

METHODS

A comprehensive literature search was performed on Pubmed/MEDLINE, Web of Science, Scopus, Embase, Cochrane Library database to identify articles including machine learning models and algorithms to predict malignant transformation in OPMDs. Relevant bibliographic data, study characteristics, and outcomes were extracted for eligible studies. Quality of the included studies was assessed through the IJMEDI checklist.

RESULTS

Fifteen articles were found suitable for the review as per the PECOS criteria. Amongst all studies, highest sensitivity (100%) was recorded for U-net architecture, Peaks Random forest model, and Partial least squares discriminant analysis (PLSDA). Highest specificity (100%) was noted for PLSDA. Range of overall accuracy in risk prediction was between 95.4% and 74%.

CONCLUSION

Machine learning proved to be a viable tool in risk prediction, demonstrating heightened sensitivity, automation, and improved accuracy for predicting transformation of OPMDs. It presents an effective approach for incorporating multiple variables to monitor the progression of OPMDs and predict their malignant potential. However, its sensitivity to dataset characteristics necessitates the optimization of input parameters to maximize the efficiency of the classifiers.

Patient classification and attribute assessment based on machine learning techniques in the qualification process for surgical treatment of adrenal tumours

Adrenal gland incidentaloma is frequently identified through computed tomography and poses a common clinical challenge. Only selected cases require surgical intervention. The primary aim of this study was to compare the effectiveness of selected machine learning (ML) techniques in proper qualifying patients for adrenalectomy and to identify the most accurate algorithm, providing a valuable tool for doctors to simplify their therapeutic decisions. The secondary aim was to assess the significance of attributes for classification accuracy. In total, clinical data were collected from 33 patients who underwent adrenalectomy. Histopathological assessments confirmed the proper selection of 21 patients for surgical intervention according to the guidelines, with accuracy reaching 64%. Statistical analysis showed that Supported Vector Machines (linear) were significantly better than the baseline (p < 0.05), with accuracy reaching 91%, and imaging features of the tumour were found to be the most crucial attributes. In summarise, ML methods may be helpful in qualifying patients for adrenalectomy.

Reliable prediction of difficult airway for tracheal intubation from patient preoperative photographs by machine learning methods

BACKGROUND

Estimating the risk of a difficult tracheal intubation should help clinicians in better anaesthesia planning, to maximize patient safety. Routine bedside screenings suffer from low sensitivity.

OBJECTIVE

To develop and evaluate machine learning (ML) and deep learning (DL) algorithms for the reliable prediction of intubation risk, using information about airway morphology.

METHODS

Observational, prospective cohort study enrolling n=623 patients who underwent tracheal intubation: 53/623 difficult cases (prevalence 8.51%). First, we used our previously validated deep convolutional neural network (DCNN) to extract 2D image coordinates for 27 + 13 relevant anatomical landmarks in two preoperative photos (frontal and lateral views). Here we propose a method to determine the 3D pose of the camera with respect to the patient and to obtain the 3D world coordinates of these landmarks. Then we compute a novel set of dM=59 morphological features (distances, areas, angles and ratios), engineered with our anaesthesiologists to characterize each individual's airway anatomy towards prediction. Subsequently, here we propose four ad hoc ML pipelines for difficult intubation prognosis, each with four stages: feature scaling, imputation, resampling for imbalanced learning, and binary classification (Logistic Regression, Support Vector Machines, Random Forests and eXtreme Gradient Boosting). These compound ML pipelines were fed with the dM=59 morphological features, alongside dD=7 demographic variables. Here we trained them with automatic hyperparameter tuning (Bayesian search) and probability calibration (Platt scaling). In addition, we developed an ad hoc multi-input DCNN to estimate the intubation risk directly from each pair of photographs, i.e. without any intermediate morphological description. Performance was evaluated using optimal Bayesian decision theory. It was compared against experts' judgement and against state-of-the-art methods (three clinical formulae, four ML, four DL models).

RESULTS

Our four ad hoc ML pipelines with engineered morphological features achieved similar discrimination capabilities: median AUCs between 0.746 and 0.766. They significantly outperformed both expert judgement and all state-of-the-art methods (highest AUC at 0.716). Conversely, our multi-input DCNN yielded low performance due to overfitting. This same behaviour occurred for the state-of-the-art DL algorithms. Overall, the best method was our XGB pipeline, with the fewest false negatives at the optimal Bayesian decision threshold.

CONCLUSIONS

We proposed and validated ML models to assist clinicians in anaesthesia planning, providing a reliable calibrated estimate of airway intubation risk, which outperformed expert assessments and state-of-the-art methods. Our novel set of engineered features succeeded in providing informative descriptions for prognosis.

Development of a Predictive Model for Survival Over Time in Patients With Out-of-Hospital Cardiac Arrest Using Ensemble-Based Machine Learning

As of now, a model for predicting the survival of patients with out-of-hospital cardiac arrest has not been established. This study aimed to develop a model for identifying predictors of survival over time in patients with out-of-hospital cardiac arrest during their stay in the emergency department, using ensemble-based machine learning. A total of 26 013 patients from the Korean nationwide out-of-hospital cardiac arrest registry were enrolled between January 1 and December 31, 2019. Our model, comprising 38 variables, was developed using the Survival Quilts model to improve predictive performance. We found that changes in important variables of patients with out-of-hospital cardiac arrest were observed 10 minutes after arrival at the emergency department. The important score of the predictors showed that the influence of patient age decreased, moving from the highest rank to the fifth. In contrast, the significance of reperfusion attempts increased, moving from the fourth to the highest rank. Our research suggests that the ensemble-based machine learning model, particularly the Survival Quilts, offers a promising approach for predicting survival in patients with out-of-hospital cardiac arrest. The Survival Quilts model may potentially assist emergency department staff in making informed decisions quickly, reducing preventable deaths.

Development of machine-learning models using pharmacy inquiry database for predicting dose-related inquiries in a tertiary teaching hospital

BACKGROUND

Drug-related problems (DRPs) are a significant concern in healthcare. Pharmacists play a vital role in detecting and resolving DRPs to improve patient safety. A pharmacy inquiry program was established in a tertiary teaching hospital to document inquiries about physicians' orders, aimed at preventing potential DRPs or providing medication information during order reviews.

OBJECTIVE

We aimed to develop machine-learning models using a pharmacy inquiry database to predict dose-related inquiries based on prescriptions and patient information.

METHODS

This retrospective study analyzed 20,393 pharmacy inquiries collected between January 2018 and February 2023. Data included prescription information (drug ingredient, dose, unit, and frequency), patient characteristics (age, sex, weight, and department), and renal function. The inquiries were categorized into two classes: dose-related inquiries (e.g., wrong dose and inappropriate regimen) and non-dose-related inquiries (e.g., inappropriate drug form and administration route). Six machine-learning models were developed: logistic regression, support vector classifier, decision tree, random forest, extreme gradient boosting, and categorical boosting. To evaluate the performance of the models, the area under the receiver operating characteristic curve and the accuracy were compared.

RESULTS

The CatBoost model achieved the highest performance (sensitivity: 0.92; accuracy: 0.79). The SHapley Additive exPlanations values highlighted the importance of features in the model predictions, drug ingredients, units, and renal function, in that order. Notably, lower renal function positively contributed to the prediction of dose-related inquiries. Additionally, the subsequent feature importance among drug ingredients showed that drugs such as acetylsalicylic acid, famotidine, metformin, and spironolactone strongly influenced the prediction of dose-related inquiries.

CONCLUSION

Machine-learning models that use pharmacy inquiry data can effectively predict dose-related inquiries. Further external validation and refinement of the models are required for broader applications in healthcare settings. These findings provide valuable guidance for healthcare professionals and highlight the potential of machine learning in pharmacists' decision-making.

A text analytics approach for mining public discussions in online cancer forum: Analysis of multi-intent lung cancer treatment dataset

BACKGROUND

Online cancer forums (OCF) are increasingly popular platforms for patients and caregivers to discuss, seek information on, and share opinions about diseases and treatments. This interaction generates a substantial amount of unstructured text data, necessitating deeper exploration. Using time series data, our study exploits topic modeling in the novel domain of online cancer forums (OCFs) to identify meaningful topics and changing dynamics of online discussion across different lung cancer treatment intent groups.

METHODS

For this purpose, a dataset comprising 27,998 forum posts about lung cancer was collected from three OCFs: lungcancer.net, lungevity.org, and reddit.com, spanning the years 2016 to 2018.

RESULTS

The analysis reflects the public discussion on multi-intent lung cancer treatment over time, taking into account seasonal variations. Discussions on cancer symptoms and prevention garnered the most attention, dominating both curative and palliative care discussions. There were distinct seasonal peaks: curative care topics surged from winter to late spring, while palliative care topics peaked from late summer to mid-autumn. Keyword analysis highlighted that lung cancer diagnosis and treatment were primary topics, whereas cancer prevention and treatment outcomes were predominant across multi-care contexts. For the study period, curative care discussions predominantly revolved around informational support and disease syndromes. In contrast, social support and cancer prevention prevailed in the palliative care context. Notably, topics such as cancer screening and cancer treatment exhibit pronounced seasonal variations in curative care, peaking in frequency during the summers (May to August) of the study period. Meanwhile, the topic of tumor control within palliative care showed significant seasonal influence during the winters and summers of 2017 and 2018.

CONCLUSION

Our text analysis approach using OCF data shows potential for computational methods in this novel domain to gain insights into trends in public cancer communication and seasonal variations for a better understanding of improving personalized care, decision support, treatment outcomes, and quality of life.

Exploring surgical infection prediction: A comparative study of established risk indexes and a novel model

BACKGROUND

Surgical site infections are a major health problem that deteriorates the patients' health and increases health care costs. A reliable method to identify patients with modifiable risk of surgical site infection is necessary to reduce the incidence of them but data are limited. Hence the objective is to assess the predictive validity of a logistic regression model compared to risk indexes to identify patients at risk of surgical site infections.

METHODS

In this study, we evaluated the predictive validity of a new model which incorporates important predictors based on logistic regression model compared to three state-of-the-art risk indexes to identify high risk patients, recruited from 2016 to 2020 from a medium size hospital in North Norway, prone to surgical site infection.

RESULTS

The logistic regression model demonstrated significantly higher scores, defined as high-risk, in 110 patients with surgical site infections than in 110 patients without surgical site infections (p < 0.001, CI 19-44) compared to risk indexes. The logistic regression model achieved an area under the curve of 80 %, which was better than the risk indexes SSIRS (77 %), NNIS (59 %), and JSS-SSI (52 %) for predicting surgical site infections. The logistic regression model identified operating time and length of stay as the major predictors of surgical site infections.

CONCLUSIONS

The logistic regression model demonstrated better performance in predicting surgical site infections compared to three state-of-the-art risk indexes. The model could be further developed into a decision support tool, by incorporating predictors available prior to surgery, to identify patients with modifiable risk prone to surgical site infection.

Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learning

OBJECTIVE

Aim to establish a multimodal model for predicting severe acute pancreatitis (SAP) using machine learning (ML) and deep learning (DL).

METHODS

In this multicentre retrospective study, patients diagnosed with acute pancreatitis at admission were enrolled from January 2017 to December 2021. Clinical information within 24 h and CT scans within 72 h of admission were collected. First, we trained Model α based on clinical features selected by least absolute shrinkage and selection operator analysis. Second, radiomics features were extracted from 3D-CT scans and Model β was developed on the features after dimensionality reduction using principal component analysis. Third, Model γ was trained on 2D-CT images. Lastly, a multimodal model, namely PrismSAP, was constructed based on aforementioned features in the training set. The predictive accuracy of PrismSAP was verified in the validation and internal test sets and further validated in the external test set. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, recall, precision and F1-score.

RESULTS

A total of 1,221 eligible patients were randomly split into a training set (n = 864), a validation set (n = 209) and an internal test set (n = 148). Data of 266 patients were for external testing. In the external test set, PrismSAP performed best with the highest AUC of 0.916 (0.873-0.960) among all models [Model α: 0.709 (0.618-0.800); Model β: 0.749 (0.675-0.824); Model γ: 0.687 (0.592-0.782); MCTSI: 0.778 (0.698-0.857); RANSON: 0.642 (0.559-0.725); BISAP: 0.751 (0.668-0.833); SABP: 0.710 (0.621-0.798)].

CONCLUSION

The proposed multimodal model outperformed any single-modality models and traditional scoring systems.

ARTEMIS: An alarm threshold and policy mining system for the intensive care unit

BACKGROUND

Alarm fatigue is a major technology-induced hazard for patients and staff in intensive care units. Too many - mostly unnecessary - alarms cause desensitisation and lack of response in medical staff. Unsuitable alarm policies are one reason for alarm fatigue. But changing alarm policies is a delicate issue since it concerns patient safety.

OBJECTIVE

We present ARTEMIS, a novel, computer-aided clinical decision support system for policy makers that can help to considerably improve alarm policies using data from hospital information systems.

METHODS

Policy makers can use different policy components from ARTEMIS' internal library to assemble tailor-made alarm policies for their intensive care units. Alternatively, policy makers can provide even more highly customised policy components as Python functions using data the hospital information systems. This can even include machine learning models - for example for setting alarm thresholds. Finally, policy makers can evaluate their system of policies and compare the resulting alarm loads.

RESULTS

ARTEMIS reports and compares numbers of alarms caused by different alarm policies for an easily adaptable target population. ARTEMIS can compare policies side-by-side and provides grid comparisons and heat maps for parameter optimisation. For example, we found that the utility of alarm delays varies based on target population. Furthermore, policy makers can introduce virtual parameters that are not in the original data by providing a formula to compute them. Virtual parameters help measuring and alarming on the right metric, even if the patient monitors do not directly measure this metric.

CONCLUSION

ARTEMIS does not release the policy maker from assessing the policy from a medical standpoint. But as a knowledge discovery and clinical decision support system, it provides a strong quantitative foundation for medical decisions. At comparatively low cost of implementation, ARTEMIS can have a substantial impact on patients and staff alike - with organisational, economic, and clinical benefits for the implementing hospital.

Robust diagnosis recommendation system for Primary Care Telemedicine using long short-term memory multi-class sequence classification

Background

Telemedicine offers opportunity for robust diagnoses recommendations to support healthcare providers intra-consultation in a way that does not limit providers ability to explore diagnostic codes and make the most appropriate selection for each consultation.

Objective

The objective of this work was to develop a recommendation system for ICD-10 coding using multiclass sequence classification and deep learning. The recommendations are intended to support telemedicine clinicians in making timely and appropriate diagnosis selections. The recommendations allow clinicians to find and select the best diagnosis code much quicker and without leaving the telemedicine platform to search codes and code descriptions.

Methods

We developed an LSTM model for multi-class text sequence classification to make diagnosis recommendations. The LSTM recommender used text-based symptoms, complaints, and consultation request reasons as model inputs. Data were extracted from a live telemedicine platform which spans general medicine, dermatology, and mental health clinical specialties. A popularity-based model was used for baseline comparison.

Results

Using over 2.8 MM telemedicine consultations during 2021 and 2022, our LSTM recommender average accuracy was 31.7%. LSTM recommender average coverage in the top 20 recommended diagnoses was 85.8% with an average personalization score of 0.87.

Conclusions

LSTM multi-class sequence classification recommends diagnoses specific to individual consultations, is retrainable on regular intervals, and could improve diagnoses recommendations such that providers require less time and resources searching for diagnosis codes. In addition, the LSTM recommender is robust enough to make recommendations across clinical specialties such as general medicine, dermatology, and mental health.

Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques

BACKGROUND

The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data.

METHODS

This study used EHR data for children and youth aged 4-17 seeking services at McMaster Children's Hospital's Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores.

RESULTS

The GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value.

CONCLUSIONS

This study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.

Diagnostic and comparative performance for the prediction of tuberculous pleural effusion using machine learning algorithms

OBJECTIVE

Early diagnosis and differential diagnosis of tuberculous pleural effusion (TPE) remains challenging and is critical to the patients' prognosis. The present study aimed to develop nine machine learning (ML) algorithms for early diagnosis of TPE and compare their performance.

METHODS

A total of 1435 untreated patients with pleural effusions (PEs) were retrospectively included and divided into the training set (80%) and the test set (20%). The demographic and laboratory variables were collected, preprocessed, and analyzed to select features, which were fed into nine ML algorithms to develop an optimal diagnostic model for TPE. The proposed model was validated by an independently external data. The decision curve analysis (DCA) and the SHapley Additive exPlanations (SHAP) were also applied.

RESULTS

Support vector machine (SVM) was the best model in discriminating TPE from non-TPE, with a balanced accuracy of 87.7%, precision of 85.3%, area under the curve (AUC) of 0.914, sensitivity of 94.7%, specificity of 80.7%, and F1-score of 86.0% among the nine ML algorithms. The excellent diagnostic performance was also validated by the external data (a balanced accuracy of 87.7%, precision of 85.2%, and AUC of 0.898). Neural network (NN) and K-nearest neighbor (KNN) had better net benefits in clinical usefulness. Besides, PE adenosine deaminase (ADA), PE carcinoembryonic antigen (CEA), and serum CYFRA21-1 were identified as the top three important features for diagnosing TPE.

CONCLUSIONS

This study developed and validated a SVM model for the early diagnosis of TPE, which might help clinicians provide better diagnosis and treatment for TPE patients.

The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review

BACKGROUND

Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine.

OBJECTIVE

We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies.

METHODS

We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility.

RESULTS

After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant.

CONCLUSIONS

The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

The modified Polsby-Popper score, a novel quantitative histomorphological biomarker and its potential to predict lymph node positivity and cancer-specific survival in oral tongue squamous cell carcinoma

BACKGROUND

The significance of different histological spreading patterns of tumor tissue in oral tongue squamous cell carcinoma (TSCC) is well known. Our aim was to construct a numeric parameter on a continuous scale, that is, the modified Polsby-Popper (MPP) score, to describe the aggressiveness of tumor growth and infiltration, with the potential to analyze hematoxylin and eosin-stained whole slide images (WSIs) in an automated manner. We investigated the application of the MPP score in predicting survival and cervical lymph node metastases as well as in determining patients at risk in the context of different surgical margin scenarios.

METHODS

We developed a semiautomated image analysis pipeline to detect areas belonging to the tumor tissue compartment. Perimeter and area measurements of all detected tissue regions were derived, and a specific mathematical formula was applied to reflect the perimeter/area ratio in a comparable, observer-independent manner across digitized WSIs. We demonstrated the plausibility of the MPP score by correlating it with well-established clinicopathologic parameters. We then performed survival analysis to assess the relevance of the MPP score, with an emphasis on different surgical margin scenarios. Machine learning models were developed to assess the relevance of the MPP score in predicting survival and occult cervical nodal metastases.

RESULTS

The MPP score was associated with unfavorable tumor growth and infiltration patterns, the presence of lymph node metastases, the extracapsular spread of tumor cells, and higher tumor thickness. Higher MPP scores were associated with worse overall survival (OS) and tongue carcinoma-specific survival (TCSS), both when assessing all pT-categories and pT1-pT2 categories only; moreover, higher MPP scores were associated with a significantly worse TCSS in cases where a cancer-free surgical margin of <5 mm could be achieved on the main surgical specimen. This discriminatory capacity remained constant when examining pT1-pT2 categories only. Importantly, the MPP score could successfully define cases at risk in terms of metastatic disease in pT1-pT2 cancer where tumor thickness failed to exhibit a significant predictive value. Machine learning (ML) models incorporating the MPP score could predict the 5-year TCSS efficiently. Furthermore, we demonstrated that machine learning models that predict occult cervical lymph node involvement can benefit from including the MPP score.

CONCLUSIONS

We introduced an objective, quantifiable, and observer-independent parameter, the MPP score, representing the aggressiveness of tumor growth and infiltration in TSCC. We showed its prognostic relevance especially in pT1-pT2 category TSCC, and its possible use in ML models predicting TCSS and occult lymph node metastases.

Dataset size versus homogeneity: A machine learning study on pooling intervention data in e-mental health dropout predictions

Objective

This study proposes a way of increasing dataset sizes for machine learning tasks in Internet-based Cognitive Behavioral Therapy through pooling interventions. To this end, it (1) examines similarities in user behavior and symptom data among online interventions for patients with depression, social anxiety, and panic disorder and (2) explores whether these similarities suffice to allow for pooling the data together, resulting in more training data when prediction intervention dropout.

Methods

A total of 6418 routine care patients from the Internet Psychiatry in Stockholm are analyzed using (1) clustering and (2) dropout prediction models. For the latter, prediction models trained on each individual intervention's data are compared to those trained on all three interventions pooled into one dataset. To investigate if results vary with dataset size, the prediction is repeated using small and medium dataset sizes.

Results

The clustering analysis identified three distinct groups that are almost equally spread across interventions and are instead characterized by different activity levels. In eight out of nine settings investigated, pooling the data improves prediction results compared to models trained on a single intervention dataset. It is further confirmed that models trained on small datasets are more likely to overestimate prediction results.

Conclusion

The study reveals similar patterns of patients with depression, social anxiety, and panic disorder regarding online activity and intervention dropout. As such, this work offers pooling different interventions' data as a possible approach to counter the problem of small dataset sizes in psychological research.

Research related to the diagnosis of prostate cancer based on machine learning medical images: A review

BACKGROUND

Prostate cancer is currently the second most prevalent cancer among men. Accurate diagnosis of prostate cancer can provide effective treatment for patients and greatly reduce mortality. The current medical imaging tools for screening prostate cancer are mainly MRI, CT and ultrasound. In the past 20 years, these medical imaging methods have made great progress with machine learning, especially the rise of deep learning has led to a wider application of artificial intelligence in the use of image-assisted diagnosis of prostate cancer.

METHOD

This review collected medical image processing methods, prostate and prostate cancer on MR images, CT images, and ultrasound images through search engines such as web of science, PubMed, and Google Scholar, including image pre-processing methods, segmentation of prostate gland on medical images, registration between prostate gland on different modal images, detection of prostate cancer lesions on the prostate.

CONCLUSION

Through these collated papers, it is found that the current research on the diagnosis and staging of prostate cancer using machine learning and deep learning is in its infancy, and most of the existing studies are on the diagnosis of prostate cancer and classification of lesions, and the accuracy is low, with the best results having an accuracy of less than 0.95. There are fewer studies on staging. The research is mainly focused on MR images and much less on CT images, ultrasound images.

DISCUSSION

Machine learning and deep learning combined with medical imaging have a broad application prospect for the diagnosis and staging of prostate cancer, but the research in this area still has more room for development.

Assessing the relevance of mental health factors in fibromyalgia severity: A data-driven case study using explainable AI

BACKGROUND AND OBJECTIVE

Fibromyalgia is a chronic disease that causes pain and affects patients' quality of life. Current treatments focus on pharmacological therapies for pain reduction. However, patients' psychological well-being is also affected, with depression and pain catastrophizing being common. This research addresses the clinicians' need to assess the influence of mental health factors on FM severity compared to pain factors.

METHODS

A co-development study between FM clinicians and data scientists analyzed data from 166 FM-diagnosed patients to assess the influence of mental health factors on FM severity in comparison to pain factors. The study used the Polysymptomatic Distress Scale (PDS) and Fibromyalgia Impact Questionnaire (FIQ) as FM severity indicators and collected 15 variables including regarding demographics, pain intensity perceived, and mental health factors. The team used an author's developed framework to identify the optimal FM severity classifier and explainability by selecting a number of features that lead to obtaining the best classification result. Machine learning classifiers employed in the framework were: decision trees, logistic regression, support vector machines, random forests, AdaBoost, extra trees, and RUSBoost. Explainability analyses were conducted using the following explainable AI techniques: SHapley Additive exPlanations (SHAP), Partial Dependence Plot (PDP), and Mean Decrease Impurity (MDI).

RESULTS

A balanced random forest with 6 features achieved the best performance with PDS (AUC_ROC, mean = 0.81, std = 0.07). Being FIQ the target variable, due to the imbalance in FM severity levels, a binary and a multiclass classification approaches were considered achieving the optimal performance, respectively, a logistic regression classifier (AUC_ROC, mean = 0.83, std = 0.08) with 6 selected features, and a random forest (AUC_ROC, mean = 0.91, std = 0.04) with 8 selected features. Next, the explainability analysis determined mental health factors were found to be more relevant than pain perceived factors for FM severity.

CONCLUSIONS

This study's findings, validated by clinicians, are potentially aligned with FM international guidelines that promote non-pharmacological interventions such as promoting mental well-being of FM patients.

Artificial Intelligence in Predicting Microsatellite Instability and KRAS, BRAF Mutations from Whole-Slide Images in Colorectal Cancer: A Systematic Review

Mismatch repair deficiency (d-MMR)/microsatellite instability (MSI), KRAS, and BRAF mutational status are crucial for treating advanced colorectal cancer patients. Traditional methods like immunohistochemistry or polymerase chain reaction (PCR) can be challenged by artificial intelligence (AI) based on whole slide images (WSI) to predict tumor status. In this systematic review, we evaluated the role of AI in predicting MSI status, KRAS, and BRAF mutations in colorectal cancer. Studies published in PubMed up to June 2023 were included (n = 17), and we reported the risk of bias and the performance for each study. Some studies were impacted by the reduced number of slides included in the data set and the lack of external validation cohorts. Deep learning models for the d-MMR/MSI status showed a good performance in training cohorts (mean AUC = 0.89, [0.74-0.97]) but slightly less than expected in the validation cohort when available (mean AUC = 0.82, [0.63-0.98]). Contrary to the MSI status, the prediction of KRAS and BRAF mutations was less explored with a less robust methodology. The performance was lower, with a maximum of 0.77 in the training cohort, 0.58 in the validation cohort for KRAS, and 0.82 AUC in the training cohort for BRAF.

Artificial Intelligence on Diagnostic Aid of Leprosy: A Systematic Literature Review

Leprosy is a neglected tropical disease that can cause physical injury and mental disability. Diagnosis is primarily clinical, but can be inconclusive due to the absence of initial symptoms and similarity to other dermatological diseases. Artificial intelligence (AI) techniques have been used in dermatology, assisting clinical procedures and diagnostics. In particular, AI-supported solutions have been proposed in the literature to aid in the diagnosis of leprosy, and this Systematic Literature Review (SLR) aims to characterize the state of the art. This SLR followed the preferred reporting items for systematic reviews and meta-analyses (PRISMA) framework and was conducted in the following databases: ACM Digital Library, IEEE Digital Library, ISI Web of Science, Scopus, and PubMed. Potentially relevant research articles were retrieved. The researchers applied criteria to select the studies, assess their quality, and perform the data extraction process. Moreover, 1659 studies were retrieved, of which 21 were included in the review after selection. Most of the studies used images of skin lesions, classical machine learning algorithms, and multi-class classification tasks to develop models to diagnose dermatological diseases. Most of the reviewed articles did not target leprosy as the study's primary objective but rather the classification of different skin diseases (among them, leprosy). Although AI-supported leprosy diagnosis is constantly evolving, research in this area is still in its early stage, then studies are required to make AI solutions mature enough to be transformed into clinical practice. Expanding research efforts on leprosy diagnosis, coupled with the advocacy of open science in leveraging AI for diagnostic support, can yield robust and influential outcomes.

Machine learning models to detect and predict patient safety events using electronic health records: A systematic review

INTRODUCTION

Identifying patient safety events using electronic health records (EHRs) and automated machine learning-based detection methods can help improve the efficiency and quality of healthcare service provision.

OBJECTIVE

This study aimed to systematically review machine learning-based methods and techniques, as well as their results for patient safety event management using EHRs.

METHODS

We reviewed the studies that focused on machine learning techniques, including automatic prediction and detection of patient safety events and medical errors through EHR analysis to manage patient safety events. The data were collected by searching Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE Xplore databases.

RESULTS

After screening, 41 papers were reviewed. Support vector machine (SVM), random forest, conditional random field (CRF), and bidirectional long short-term memory with conditional random field (BiLSTM-CRF) algorithms were mostly applied to predict, identify, and classify patient safety events using EHRs; however, they had different performances. BiLSTM-CRF was employed in most of the studies to extract and identify concepts, e.g., adverse drug events (ADEs) and adverse drug reactions (ADRs), as well as relationships between drug and severity, drug and ADEs, drug and ADRs. Recurrent neural networks (RNN) and BiLSTM-CRF had the best results in detecting ADEs compared to other patient safety events. Linear classifiers and Naive Bayes (NB) had the highest performance for ADR detection. Logistic regression had the best results in detecting surgical site infections. According to the findings, the quality of articles has non-significantly improved in recent years, but they had low average scores.

CONCLUSIONS

Machine learning can be useful in automatic detection and prediction of patient safety events. However, most of these algorithms have not yet been externally validated or prospectively tested. Therefore, further studies are required to improve the performance of these automated systems.

Machine learning as a tool to identify inpatients who are not at risk of adverse drug events in a large dataset of a tertiary care hospital in the USA

AIMS

Adverse drug events (ADEs) are a major threat to inpatients in the United States of America (USA). It is unknown how well machine learning (ML) is able to predict whether or not a patient will suffer from an ADE during hospital stay based on data available at hospital admission for emergency department patients of all ages (binary classification task). It is further unknown whether ML is able to outperform logistic regression (LR) in doing so, and which variables are the most important predictors.

METHODS

In this study, 5 ML models- namely a random forest, gradient boosting machine (GBM), ridge regression, least absolute shrinkage and selection operator (LASSO) regression, and elastic net regression-as well as a LR were trained and tested for the prediction of inpatient ADEs identified using ICD-10-CM codes based on comprehensive previous work in a diverse population. In total, 210 181 observations from patients who were admitted to a large tertiary care hospital after emergency department stay between 2011 and 2019 were included. The area under the receiver operating characteristics curve (AUC) and AUC-precision-recall (AUC-PR) were used as primary performance indicators.

RESULTS

Tree-based models performed best with respect to AUC and AUC-PR. The gradient boosting machine (GBM) reached an AUC of 0.747 (95% confidence interval (CI): 0.735 to 0.759) and an AUC-PR of 0.134 (95% CI: 0.131 to 0.137) on unforeseen test data, while the random forest reached an AUC of 0.743 (95% CI: 0.731 to 0.755) and an AUC-PR of 0.139 (95% CI: 0.135 to 0.142), respectively. ML statistically significantly outperformed LR both on AUC and AUC-PR. Nonetheless, overall, models did not differ much with respect to their performance. Most important predictors were admission type, temperature and chief complaint for the best performing model (GBM).

CONCLUSIONS

The study demonstrated a first application of ML to predict inpatient ADEs based on ICD-10-CM codes, and a comparison with LR. Future research should address concerns arising from low precision and related problems.

Artificial intelligence in emergency medicine. A systematic literature review

Motivation and objective: Emergency medicine is becoming a popular application area for artificial intelligence methods but remains less investigated than other healthcare branches. The need for time-sensitive decision-making on the basis of high data volumes makes the use of quantitative technologies inevitable. However, the specifics of healthcare regulations impose strict requirements for such applications. Published contributions cover separate parts of emergency medicine and use disparate data and algorithms. This study aims to systematize the relevant contributions, investigate the main obstacles to artificial intelligence applications in emergency medicine, and propose directions for further studies.

METHODS

The contributions selection process was conducted with systematic electronic databases querying and filtering with respect to established exclusion criteria. Among the 380 papers gathered from IEEE Xplore, ACM Digital Library, Springer Library, ScienceDirect, and Nature databases 116 were considered to be a part of the survey. The main features of the selected papers are the focus on emergency medicine and the use of machine learning or deep learning algorithms.

FINDINGS AND DISCUSSION

The selected papers were classified into two branches: diagnostics-specific and triage-specific. The former ones are focused on either diagnosis prediction or decision support. The latter covers such applications as mortality, outcome, admission prediction, condition severity estimation, and urgent care prediction. The observed contributions are highly specialized within a single disease or medical operation and often use privately collected retrospective data, making them incomparable. These and other issues can be addressed by creating an end-to-end solution based on human-machine interaction.

CONCLUSION

Artificial intelligence applications are finding their place in emergency medicine, while most of the corresponding studies remain isolated and lack higher generalization and more sophisticated methodology, which can be a matter of forthcoming improvements.

Finding the Best Match - a Case Study on the (Text-)Feature and Model Choice in Digital Mental Health Interventions

With the need for psychological help long exceeding the supply, finding ways of scaling, and better allocating mental health support is a necessity. This paper contributes by investigating how to best predict intervention dropout and failure to allow for a need-based adaptation of treatment. We systematically compare the predictive power of different text representation methods (metadata, TF-IDF, sentiment and topic analysis, and word embeddings) in combination with supplementary numerical inputs (socio-demographic, evaluation, and closed-question data). Additionally, we address the research gap of which ML model types - ranging from linear to sophisticated deep learning models - are best suited for different features and outcome variables. To this end, we analyze nearly 16.000 open-text answers from 849 German-speaking users in a Digital Mental Health Intervention (DMHI) for stress. Our research proves that - contrary to previous findings - there is great promise in using neural network approaches on DMHI text data. We propose a task-specific LSTM-based model architecture to tackle the challenge of long input sequences and thereby demonstrate the potential of word embeddings (AUC scores of up to 0.7) for predictions in DMHIs. Despite the relatively small data set, sequential deep learning models, on average, outperform simpler features such as metadata and bag-of-words approaches when predicting dropout. The conclusion is that user-generated text of the first two sessions carries predictive power regarding patients' dropout and intervention failure risk. Furthermore, the match between the sophistication of features and models needs to be closely considered to optimize results, and additional non-text features increase prediction results.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00148-z.

Machine learning in medication prescription: A systematic review

BACKGROUND

Medication prescription is a complex process that could benefit from current research and development in machine learning through decision support systems. Particularly pediatricians are forced to prescribe medications "off-label" as children are still underrepresented in clinical studies, which leads to a high risk of an incorrect dose and adverse drug effects.

METHODS

PubMed, IEEE Xplore and PROSPERO were searched for relevant studies that developed and evaluated well-performing machine learning algorithms following the PRISMA statement. Quality assessment was conducted in accordance with the IJMEDI checklist. Identified studies were reviewed in detail, including the required variables for predicting the correct dose, especially of pediatric medication prescription.

RESULTS

The search identified 656 studies, of which 64 were reviewed in detail and 36 met the inclusion criteria. According to the IJMEDI checklist, five studies were considered to be of high quality. 19 of the 36 studies dealt with the active substance warfarin. Overall, machine learning algorithms based on decision trees or regression methods performed superior regarding their predictive power than algorithms based on neural networks, support vector machines or other methods. The use of ensemble methods like bagging or boosting generally enhanced the accuracy of the dose predictions. The required input and output variables of the algorithms were considerably heterogeneous and differ strongly among the respective substance.

CONCLUSIONS

By using machine learning algorithms, the prescription process could be simplified and dosing correctness could be enhanced. Despite the heterogenous results among the different substances and cases and the lack of pediatric use cases, the identified approaches and required variables can serve as an excellent starting point for further development of algorithms predicting drug doses, particularly for children. Especially the combination of physiologically-based pharmacokinetic models with machine learning algorithms represents a great opportunity to enhance the predictive power and accuracy of the developed algorithms.

Risk predictions of hospital-acquired pressure injury in the intensive care unit based on a machine learning algorithm

Pressure injury (PI), or local damage to soft tissues and skin caused by prolonged pressure, remains controversial in the medical world. Patients in intensive care units (ICUs) were frequently reported to suffer PIs, with a heavy burden on their life and expenditures. Machine learning (ML) is a Section of artificial intelligence (AI) that has emerged in nursing practice and is increasingly used for diagnosis, complications, prognosis, and recurrence prediction. This study aims to investigate hospital-acquired PI (HAPI) risk predictions in ICU based on a ML algorithm by R programming language analysis. The former evidence was gathered through PRISMA guidelines. The logical analysis was applied via an R programming language. ML algorithms based on usage rate included logistic regression (LR), Random Forest (RF), Distributed tree (DT), Artificial neural networks (ANN), SVM (Support Vector Machine), Batch normalisation (BN), GB (Gradient Boosting), expectation-maximisation (EM), Adaptive Boosting (AdaBoost), and Extreme Gradient Boosting (XGBoost). Six cases were related to risk predictions of HAPI in the ICU based on an ML algorithm from seven obtained studies, and one study was associated with the Detection of PI risk. Also, the most estimated risksSerum Albumin, Lack of Activity, mechanical ventilation (MV), partial pressure of oxygen (PaO2), Surgery, Cardiovascular adequacy, ICU stay, Vasopressor, Consciousness, Skin integrity, Recovery Unit, insulin and oral antidiabetic (INS&OAD), Complete blood count (CBC), acute physiology and chronic health evaluation (APACHE) II score, Spontaneous bacterial peritonitis (SBP), Steroid, Demineralized Bone Matrix (DBM), Braden score, Faecal incontinence, Serum Creatinine (SCr) and age. In sum, HAPI prediction and PI risk detection are two significant areas for using ML in PI analysis. Also, the current data showed that the ML algorithm, including LR and RF, could be regarded as the practical platform for developing AI tools for diagnosing, prognosis, and treating PI in hospital units, especially ICU.

Augmented intelligence facilitates concept mapping across different electronic health records

INTRODUCTION

With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports.

METHODS

We used the manually mapped parameter names from the multicenter "Dutch ICU data warehouse against COVID-19" sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons.

RESULTS

The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters.

CONCLUSION

Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.

Machine learning for prediction of viral hepatitis: A systematic review and meta-analysis

BACKGROUND

Lack of accurate and timely diagnosis of hepatitis poses obstacles to effective treatment, disease progression prevention, complication reduction, and life-saving interventions of patients. Utilizing machine learning can greatly enhance the achievement of timely and precise disease diagnosis. Therefore, we carried out this systematic review and meta-analysis to explore the performance of machine learning algorithms in predicting viral hepatitis.

METHODS

Using an extensive literature search in PubMed, Scopus, and Web of Science databases until June 15, 2023, English publications on hepatitis prediction using machine learning algorithms were included. Two authors independently extracted pertinent information from the selected studies. The PRISMA 2020 checklist was followed for study selection and result reporting. The risk of bias was checked using the International Journal of Medical Informatics (IJMEDI) checklist. Data were analyzed using the 'metandi' command in Stata 17.

RESULTS

Twenty-one original studies were included, covering 82 algorithms. Sixteen studies utilized five algorithms to predict hepatitis B. Ten studies used five algorithms for hepatitis C prediction. For hepatitis B prediction, the SVM algorithms demonstrated the highest sensitivity (90.0%; 95% confidence interval (CI): 77.0%-96.0%), specificity (94%; 95% CI: 90.0%-97.0%), and a diagnostic odds ratio (DOR) of 145 (95% CI: 37.0-559.0). In the case of hepatitis C, the KNN algorithms exhibited the highest sensitivity (80%; 95% CI:30.0%-97.0%), specificity (95%; 95% CI: 58.0%-99.0%), and DOR (72; 95% CI: 3.0-1644.0) for prediction.

CONCLUSION

SVM and KNN demonstrated superior performance in predicting hepatitis. The proper algorithm along with clinical practice could improve hepatitis prediction and management.

Prediction of effective sociodemographic variables in modeling health literacy: A machine learning approach

INTRODUCTION

Health literacy is becoming a more important concept for the effective use of health systems day by day. The main purpose of the study is to determine the importance levels of the variables by using Machine Learning methods in order to determine the main factors affecting health literacy, and to find the most important variables for health literacy.

MATERIAL AND METHODS

1001 participants with a mean age of 18.05 ± 0.81 standard deviations were included in the study. The European Health Literacy Scale was used to determine the health literacy level of the participants. The scale cut-off point is 25, and 516 (51.5%) of the participants have low health literacy and 485 (48.5%) have a high level of health literacy. In the study, XGBoost, random forest, logistic regression models from machine learning methods were used and indexes were calculated.

RESULTS

When the results of XGBoost, random forest, logistic regression models were evaluated, it was found that the model with the best performance was XGBoost. Sensitivity, specificity, F1-score, AUROC and Brier score values for the XGBoost models were obtained as 0.979, 0.965, 0.973, 0.983, 0.054 respectively.

CONCLUSION

It was found that HL levels differed significantly in the variables of gender, age, class, family education, place of residence, economic situation, and covering health expenses (p < 0.05). According to the XGBoost model, it was found that the variable with the highest level of importance was reading the newspaper, while the variable with the lowest level of importance was the educational status of the mother. With the help of the established model, the basic variables that will affect the HL level were determined. The designed model will constitute the basic step of an supporting design system to improve physician-patient communication.

Machine learning techniques for periodontitis and dental caries detection: A narrative review

OBJECTIVES

In recent years, periodontitis, and dental caries have become common in humans and need to be diagnosed in the early stage to prevent severe complications and tooth loss. These dental issues are diagnosed by visual inspection, measuring pocket probing depth, and radiographs findings from experienced dentists. Though a glut of machine learning (ML) algorithms has been proposed for the automated detection of periodontitis, and dental caries, determining which ML techniques are suitable for clinical practice remains under debate. This review aims to identify the research challenges by analyzing the limitations of current methods and how to address these to obtain robust systems suitable for clinical use or point-of-care testing.

METHODS

An extensive search of the literature published from 2015 to 2022 written in English, related to the subject of study was sought by searching the electronic databases: PubMed, Institute of Electrical and Electronics Engineers (IEEE) Xplore, and ScienceDirect.

RESULTS

The initial electronic search yielded 1743 titles, and 55 studies were eventually included based on the selection criteria adopted in this review. Studies selected were on ML applications for the automatic detection of periodontitis and dental caries and related dental issues: Apical lessons, Periodontal bone loss, and Vertical root fracture.

CONCLUSION

While most of the ML-based studies use radiograph images for the detection of periodontitis and dental caries, few pieces of the literature revealed that good diagnostic accuracy could be achieved by training the ML model even with mobile photos representing the images of dental issues. Nowadays smartphones are used in every sector for different applications. Training the ML model with as many images of dental issues captured by the smartphone can achieve good accuracy, reduce the cost of clinical diagnosis, and provide user interaction.