Algor-ethics: charting the ethical path for AI in critical care

The integration of Clinical Decision Support Systems (CDSS) based on artificial intelligence (AI) in healthcare is groundbreaking evolution with enormous potential, but its development and ethical implementation, presents unique challenges, particularly in critical care, where physicians often deal with life-threating conditions requiring rapid actions and patients unable to participate in the decisional process. Moreover, development of AI-based CDSS is complex and should address different sources of bias, including data acquisition, health disparities, domain shifts during clinical use, and cognitive biases in decision-making. In this scenario algor-ethics is mandatory and emphasizes the integration of 'Human-in-the-Loop' and 'Algorithmic Stewardship' principles, and the benefits of advanced data engineering. The establishment of Clinical AI Departments (CAID) is necessary to lead AI innovation in healthcare, ensuring ethical integrity and human-centered development in this rapidly evolving field.

Learning Ordinal-Hierarchical Constraints for Deep Learning Classifiers

Real-world classification problems may disclose different hierarchical levels where the categories are displayed in an ordinal structure. However, no specific deep learning (DL) models simultaneously learn hierarchical and ordinal constraints while improving generalization performance. To fill this gap, we propose the introduction of two novel ordinal-hierarchical DL methodologies, namely, the hierarchical cumulative link model (HCLM) and hierarchical-ordinal binary decomposition (HOBD), which are able to model the ordinal structure within different hierarchical levels of the labels. In particular, we decompose the hierarchical-ordinal problem into local and global graph paths that may encode an ordinal constraint for each hierarchical level. Thus, we frame this problem as simultaneously minimizing global and local losses. Furthermore, the ordinal constraints are set by two approaches ordinal binary decomposition (OBD) and cumulative link model (CLM) within each global and local function. The effectiveness of the proposed approach is measured on four real-use case datasets concerning industrial, biomedical, computer vision, and financial domains. The extracted results demonstrate a statistically significant improvement to state-of-the-art nominal, ordinal, and hierarchical approaches.

A deep learning-based telemonitoring application to automatically assess oral diadochokinesis in patients with bulbar amyotrophic lateral sclerosis

BACKGROUND AND OBJECTIVES

Timely identification of dysarthria progression in patients with bulbar-onset amyotrophic lateral sclerosis (ALS) is relevant to have a comprehensive assessment of the disease evolution. To this goal literature recognized the utmost importance of the assessment of the number of syllables uttered by a subject during the oral diadochokinesis (DDK) test.

METHODS

To support clinicians, this work proposes a remote deep learning-based system, which consists (i) of a web application to acquire audio tracks of bulbar-onset ALS patients and healthy control subjects while performing the oral DDK test (i.e., repeating the /pa/, /pa-ta-ka/ and /oo-ee/ syllables) and (ii) a DDK-AID network designed to process the acquired audio signals which have different duration and to output the number of per-task syllables repeated by the subject.

RESULTS

The DDK-AID network overcomes the comparative method achieving a mean Accuracy of 90.23 in counting syllables repeated by the eleven bulbar-onset ALS-patients while performing the oral DDK test.

CONCLUSIONS

The proposed remote monitoring system, in the light of the achieved performance, represents an important step towards the implementation of self-service telemedicine systems which may ensure customised care plans.

A novel missing data imputation approach based on clinical conditional Generative Adversarial Networks applied to EHR datasets

The missing data mechanism is a relevant problem in Machine Learning (ML) and biomedical informatics communities. Real-world Electronic Health Record (EHR) datasets comprise several missing values, thus revealing a high level of spatiotemporal sparsity in the predictors' matrix. Several approaches in the state-of-the-art tried to deal with this problem by proposing different data imputation strategies that (i) are often unrelated to the ML model, (ii) are not conceived for EHR data where laboratory exams are not prescribed uniformly over time and percentage of missing values is high (iii) exploit only univariate and linear information on the observed features. Our paper proposes a data imputation strategy based on a clinical conditional Generative Adversarial Network (ccGAN) capable of imputing missing values by exploiting non-linear and multivariate information across patients. Unlike other GAN data imputation-based approaches, our method deals explicitly with the high level of missingness of routine EHR data by conditioning the imputing strategy to the observable values and those fully-annotated. We demonstrated the statistical significance of the ccGAN to other state-of-the-art approaches in terms of imputation (around 19.79% of gain to the best competitor) and predictive performance (up to 1.60% of gain to the best competitor) on a real multi-diabetic centers dataset. We also demonstrated its robustness across different missingness rates (up to 1.61% of gain to the best competitor in the highest missingness rates condition) on an additional benchmark EHR dataset.

A store-and-forward cloud-based telemonitoring system for automatic assessing dysarthria evolution in neurological diseases from video-recording analysis

BACKGROUND AND OBJECTIVES

Patients suffering from neurological diseases may develop dysarthria, a motor speech disorder affecting the execution of speech. Close and quantitative monitoring of dysarthria evolution is crucial for enabling clinicians to promptly implement patients' management strategies and maximizing effectiveness and efficiency of communication functions in term of restoring, compensating or adjusting. In the clinical assessment of orofacial structures and functions, at rest condition or during speech and non-speech movements, a qualitative evaluation is usually performed, throughout visual observation.

METHODS

To overcome limitations posed by qualitative assessments, this work presents a store-and-forward self-service telemonitoring system that integrates, within its cloud architecture, a convolutional neural network (CNN) for analyzing video recordings acquired by individuals with dysarthria. This architecture - called facial landmark Mask RCNN - aims at locating facial landmarks as a prior for assessing the orofacial functions related to speech and examining dysarthria evolution in neurological diseases.

RESULTS

When tested on the Toronto NeuroFace dataset, a publicly available annotated dataset of video recordings from patients with amyotrophic lateral sclerosis (ALS) and stroke, the proposed CNN achieved a normalized mean error equal to 1.79 on localizing the facial landmarks. We also tested our system in a real-life scenario on 11 bulbar-onset ALS subjects, obtaining promising outcomes in terms of facial landmark position estimation.

DISCUSSION AND CONCLUSIONS

This preliminary study represents a relevant step towards the use of remote tools to support clinicians in monitoring the evolution of dysarthria.

A preliminary study on self-care telemonitoring of dysarthria in spinal muscular atrophy

Spinal muscular atrophy (SMA) is a rare neuromuscular disease which may cause impairments in oro-facial musculature. Most of the individuals with SMA present bulbar signs such as flaccid dysarthria which mines their abilities to speak and, as consequence, their psychic balance. To support clinicians, recent work has demonstrated the feasibility of video-based techniques for assessing the oro-facial functions in patients with neurological disorders such as amyotrophic lateral sclerosis. However, no work has so far focused on automatic and quantitative monitoring of dysarthria in SMA. To overcome limitations this work's aim is to propose a cloud-based store-and-forward telemonitoring system for automatic and quantitative evaluation of oro-facial muscles in individuals with SMA. The system integrates a convolutional neural network (CNN) aimed at identifying the position of facial landmarks from video recordings acquired via a web application by an SMA patient.Clinical relevance- The proposed work is in the preliminary stage, but it represents the first step towards a better understanding of the bulbar-functions' evolution in patients with SMA.

Uncovering the potential of innovation ecosystems in the healthcare sector after the COVID-19 crisis

Industry 4.0 technologies are expected to enhance healthcare quality at the minimum cost feasible by using innovative solutions based on a fruitful exchange of knowledge and resources among institutions, firms and academia. These collaborative mechanisms are likely to occur in an innovation ecosystem where different stakeholders and resources interact to provide ground-breaking solutions to the market. The paper proposes a framework for studying the creation and development of innovation ecosystems in the healthcare sector by using a set of interrelated dimensions including, technology, value, and capabilities within a Triple-Helix model guided by focal actors. The model is applied to an exemplary Italian innovation ecosystem providing cloud and artificial intelligence-based solutions to general practitioners (GPs) under the focal role of the Italian association of GPs. Primary and secondary data are examined starting from the innovation ecosystem's origins and continuing until the COVID-19 crisis. The findings show that the pandemic represented the turning point that altered the ecosystem's dimensions in order to find immediate solutions for monitoring health conditions and organizing the booking of swabs and vaccines. The data triangulation points out the technical, organizational, and administrative barriers hindering the widespread adoption of these solutions at the national and regional levels, revealing several implications for health policy.

A review on deep-learning algorithms for fetal ultrasound-image analysis

Deep-learning (DL) algorithms are becoming the standard for processing ultrasound (US) fetal images. A number of survey papers in the field is today available, but most of them are focusing on a broader area of medical-image analysis or not covering all fetal US DL applications. This paper surveys the most recent work in the field, with a total of 153 research papers published after 2017. Papers are analyzed and commented from both the methodology and the application perspective. We categorized the papers into (i) fetal standard-plane detection, (ii) anatomical structure analysis and (iii) biometry parameter estimation. For each category, main limitations and open issues are presented. Summary tables are included to facilitate the comparison among the different approaches. In addition, emerging applications are also outlined. Publicly-available datasets and performance metrics commonly used to assess algorithm performance are summarized, too. This paper ends with a critical summary of the current state of the art on DL algorithms for fetal US image analysis and a discussion on current challenges that have to be tackled by researchers working in the field to translate the research methodology into actual clinical practice.

Generating depth images of preterm infants in given poses using GANs

BACKGROUND AND OBJECTIVES

The use of deep learning for preterm infant's movement monitoring has the potential to support clinicians in early recognizing motor and behavioural disorders. The development of deep learning algorithms is, however, hampered by the lack of publicly available annotated datasets.

METHODS

To mitigate the issue, this paper presents a Generative Adversarial Network-based framework to generate images of preterm infants in a given pose. The framework consists of a bibranch encoder and a conditional Generative Adversarial Network, to generate a rough image and a refined version of it, respectively.

RESULTS

Evaluation was performed on the Moving INfants In RGB-D dataset which has 12.000 depth frames from 12 preterm infants. A low Fréchet inception distance (142.9) and an inception score (2.8) close to that of real-image distribution (2.6) are obtained. The results achieved show the potentiality of the framework in generating realistic depth images of preterm infants in a given pose.

CONCLUSIONS

Pursuing research on the generation of new data may enable researchers to propose increasingly advanced and effective deep learning-based monitoring systems.

A deep learning approach to median nerve evaluation in ultrasound images of carpal tunnel inlet

Ultrasound (US) imaging is recognized as a useful support for Carpal Tunnel Syndrome (CTS) assessment through the evaluation of median nerve morphology. However, US is still far to be systematically adopted to evaluate this common entrapment neuropathy, due to US intrinsic challenges, such as its operator dependency and the lack of standard protocols. To support sonographers, the present study proposes a fully-automatic deep learning approach to median nerve segmentation from US images. We collected and annotated a dataset of 246 images acquired in clinical practice involving 103 rheumatic patients, regardless of anatomical variants (bifid nerve, closed vessels). We developed a Mask R-CNN with two additional transposed layers at segmentation head to accurately segment the median nerve directly on transverse US images. We calculated the cross-sectional area (CSA) of the predicted median nerve. Proposed model achieved good performances both in median nerve detection and segmentation: Precision (Prec), Recall (Rec), Mean Average Precision (mAP) and Dice Similarity Coefficient (DSC) values are 0.916 ± 0.245, 0.938 ± 0.233, 0.936 ± 0.235 and 0.868 ± 0.201, respectively. The CSA values measured on true positive predictions were comparable with the sonographer manual measurements with a mean absolute error (MAE) of 0.918 mm2. Experimental results showed the potential of proposed model, which identified and segmented the median nerve section in normal anatomy images, while still struggling when dealing with infrequent anatomical variants. Future research will expand the dataset including a wider spectrum of normal anatomy and pathology to support sonographers in daily practice.

Graphical abstract

Prediction of complications of type 2 Diabetes: A Machine learning approach

AIM

To construct predictive models of diabetes complications (DCs) by big data machine learning, based on electronic medical records.

METHODS

Six groups of DCs were considered: eye complications, cardiovascular, cerebrovascular, and peripheral vascular disease, nephropathy, diabetic neuropathy. A supervised, tree-based learning approach (XGBoost) was used to predict the onset of each complication within 5 years (task 1). Furthermore, a separate prediction for early (within 2 years) and late (3-5 years) onset of complication (task 2) was performed. A dataset of 147.664 patients seen during 15 years by 23 centers was used. External validation was performed in five additional centers. Models were evaluated by considering accuracy, sensitivity, specificity, and area under the ROC curve (AUC).

RESULTS

For all DCs considered, the predictive models in task 1 showed an accuracy > 70 %, and AUC largely exceeded 0.80, reaching 0.97 for nephropathy. For task 2, all predictive models showed an accuracy > 70 % and an AUC > 0.85. Sensitivity in predicting the early occurrence of the complication ranged between 83.2 % (peripheral vascular disease) and 88.5 % (nephropathy).

CONCLUSIONS

Machine learning approach offers the opportunity to identify patients at greater risk of complications. This can help overcoming clinical inertia and improving the quality of diabetes care.

Development of a convolutional neural network for the identification and the measurement of the median nerve on ultrasound images acquired at carpal tunnel level

Background

Deep learning applied to ultrasound (US) can provide a feedback to the sonographer about the correct identification of scanned tissues and allows for faster and standardized measurements. The most frequently adopted parameter for US diagnosis of carpal tunnel syndrome is the increasing of the cross-sectional area (CSA) of the median nerve. Our aim was to develop a deep learning algorithm, relying on convolutional neural networks (CNNs), for the localization and segmentation of the median nerve and the automatic measurement of its CSA on US images acquired at the proximal inlet of the carpal tunnel.

Methods

Consecutive patients with rheumatic and musculoskeletal disorders were recruited. Transverse US images were acquired at the carpal tunnel inlet, and the CSA was manually measured. Anatomical variants were registered. The dataset consisted of 246 images (157 for training, 40 for validation, and 49 for testing) from 103 patients each associated with manual annotations of the nerve boundary. A Mask R-CNN, state-of-the-art CNN for image semantic segmentation, was trained on this dataset to accurately localize and segment the median nerve section. To evaluate the performances on the testing set, precision (Prec), recall (Rec), mean average precision (mAP), and Dice similarity coefficient (DSC) were computed. A sub-analysis excluding anatomical variants was performed. The CSA was automatically measured by the algorithm.

Results

The algorithm correctly identified the median nerve in 41/49 images (83.7%) and in 41/43 images (95.3%) excluding anatomical variants. The following metrics were obtained (with and without anatomical variants, respectively): Prec 0.86 ± 0.33 and 0.96 ± 0.18, Rec 0.88 ± 0.33 and 0.98 ± 0.15, mAP 0.88 ± 0.33 and 0.98 ± 0.15, and DSC 0.86 ± 0.19 and 0.88 ± 0.19. The agreement between the algorithm and the sonographer CSA measurements was excellent [ICC 0.97 (0.94–0.98)].

Conclusions

The developed algorithm has shown excellent performances, especially if excluding anatomical variants. Future research should aim at expanding the US image dataset including a wider spectrum of normal anatomy and pathology. This deep learning approach has shown very high potentiality for a fully automatic support for US assessment of carpal tunnel syndrome.

Social media analytics system for action inspection on social networks

Social networks are increasingly used for discussing all kinds of topics, including those related to politics, serving as a virtual arena. Consequently, analysing online conversations, for example, to predict election outcomes, is becoming a popular and challenging research area. On social networking sites, citizens express themselves spontaneously regarding political topics, often driven by specific events in social life. Real-time analysis of social media can provide valuable feedback and insights to both politicians and news agencies. In this paper, we discuss the design and implementation of a system for tracking and analysing social media. The SocMINT system provides an easy-to-use, visual dashboard to monitor the discussion on specific topics, to capture trends in communities and, by iteratively applying multidimensional data analysis and filtering, to deeply analyse posts and influencers. SocMINT aggregates data from multiple social sources and performs sentiment analysis on textual, visual and mixed content via a specifically designed neural network architecture. The system was applied in a real context by administrative staff of a political party to effectively analyse candidates’ political communication on Facebook, Instagram and Twitter and the related online community reactions and discussion. In the paper, we report on this real-world case study, showing how the system meaningfully captures trends in public opinion, comparing the main KPIs provided by SocMINT with the outcomes of traditional polls.

A Unified Hierarchical XGBoost model for classifying priorities for COVID-19 vaccination campaign

The current ML approaches do not fully focus to answer a still unresolved and topical challenge, namely the prediction of priorities of COVID-19 vaccine administration. Thus, our task includes some additional methodological challenges mainly related to avoiding unwanted bias while handling categorical and ordinal data with a highly imbalanced nature. Hence, the main contribution of this study is to propose a machine learning algorithm, namely Hierarchical Priority Classification eXtreme Gradient Boosting for priority classification for COVID-19 vaccine administration using the Italian Federation of General Practitioners dataset that contains Electronic Health Record data of 17k patients. We measured the effectiveness of the proposed methodology for classifying all the priority classes while demonstrating a significant improvement with respect to the state of the art. The proposed ML approach, which is integrated into a clinical decision support system, is currently supporting General Pracitioners in assigning COVID-19 vaccine administration priorities to their assistants.

Development of a measurement setup to detect the level of physical activity and social distancing of ageing people in a social garden during COVID-19 pandemic

This study defines a methodology to measure physical activity (PA) in ageing people working in a social garden while maintaining social distancing (SD) during COVID-19 pandemic. A real-time location system (RTLS) with embedded inertial measurement unit (IMU) sensors is used for measuring PA and SD. The position of each person is tracked to assess their SD, finding that the RTLS/IMU can measure the time in which interpersonal distance is not kept with a maximum uncertainty of 1.54 min, which compared to the 15-min. limit suggested to reduce risk of transmission at less than 1.5 m, proves the feasibility of the measurement. The data collected by the accelerometers of the IMU sensors are filtered using discrete wavelet transform and used to measure the PA in ageing people with an uncertainty-based thresholding method. PA and SD time measurements were demonstrated exploiting the experimental test in a pilot case with real users.

Learning-Based Median Nerve Segmentation From Ultrasound Images For Carpal Tunnel Syndrome Evaluation

Carpal tunnel syndrome (CTS) is the most common entrapment neuropathy. Ultrasound imaging (US) may help to diagnose and assess CTS, through the evaluation of median nerve morphology. To support sonographers, this paper proposes a fully-automatic deep-learning approach to median nerve segmentation from US images. The approach relies on Mask R-CNN, a convolutional neural network that is trained end-to-end. The segmentation head of Mask R-CNN is here evaluated with three different configurations, with the goal of studying the effect of the segmentation-head output resolution on the overall Mask R-CNN segmentation performance. For this study, we collected and annotated a dataset of 151 images acquired in the actual clinical practice from 53 subjects with CTS. To our knowledge, this is the largest dataset in the field in terms of subjects. We achieved a median Dice similarity coefficient equal to 0.931 (IQR = 0.027), demonstrating the potentiality of the proposed approach. These results are a promising step towards providing an effective tool for CTS assessment in the actual clinical practice.

Asymmetric Three-dimensional Convolutions For Preterm Infants' Pose Estimation

Computer-assisted tools for preterm infants' movement monitoring in neonatal intensive care unit (NICU) could support clinicians in highlighting preterm-birth complications. With such a view, in this work we propose a deep-learning framework for preterm infants' pose estimation from depth videos acquired in the actual clinical practice. The pipeline consists of two consecutive convolutional neural networks (CNNs). The first CNN (inherited from our previous work) acts to roughly predict joints and joint-connections position, while the second CNN (Asy-regression CNN) refines such predictions to trace the limb pose. Asy-regression relies on asymmetric convolutions to temporally optimize both the training and predictions phase. Compared to its counterpart without asymmetric convolutions, Asy-regression experiences a reduction in training and prediction time of 66% , while keeping the root mean square error, computed against manual pose annotation, merely unchanged. Research mostly works to develop highly accurate models, few efforts have been invested to make the training and deployment of such models time-effective. With a view to make these monitoring technologies sustainable, here we focused on the second aspect and addressed the problem of designing a framework as trade-off between reliability and efficiency.

Improving Preterm Infants' Joint Detection in Depth Images Via Dense Convolutional Neural Networks

Preterm infants' spontaneous motility is a valuable diagnostic and prognostic index of motor and cognitive impairments. Despite being recognized as crucial, preterm infant's movement assessment is mostly based on clinicians' visual inspection. The aim of this work is to present a 2D dense convolutional neural network (denseCNN) to detect preterm infant's joints in depth images acquired in neonatal intensive care units. The denseCNN allows to improve the performance of our previous model in the detection of joints and joint connections, reaching a median recall value equal to 0.839. With a view to monitor preterm infants in a scenario where computational resources are scarce, we tested the architecture on a mid-range laptop. The prediction occurs in real-time (0.014 s per image), opening up the possibility of integrating such monitoring system in a domestic environment.

The INCREASE project: Intelligent Collections of food-legume genetic resources for European agrofood systems

Food legumes are crucial for all agriculture-related societal challenges, including climate change mitigation, agrobiodiversity conservation, sustainable agriculture, food security and human health. The transition to plant-based diets, largely based on food legumes, could present major opportunities for adaptation and mitigation, generating significant co-benefits for human health. The characterization, maintenance and exploitation of food-legume genetic resources, to date largely unexploited, form the core development of both sustainable agriculture and a healthy food system. INCREASE will implement, on chickpea (Cicer arietinum), common bean (Phaseolus vulgaris), lentil (Lens culinaris) and lupin (Lupinus albus and L. mutabilis), a new approach to conserve, manage and characterize genetic resources. Intelligent Collections, consisting of nested core collections composed of single-seed descent-purified accessions (i.e., inbred lines), will be developed, exploiting germplasm available both from genebanks and on-farm and subjected to different levels of genotypic and phenotypic characterization. Phenotyping and gene discovery activities will meet, via a participatory approach, the needs of various actors, including breeders, scientists, farmers and agri-food and non-food industries, exploiting also the power of massive metabolomics and transcriptomics and of artificial intelligence and smart tools. Moreover, INCREASE will test, with a citizen science experiment, an innovative system of conservation and use of genetic resources based on a decentralized approach for data management and dynamic conservation. By promoting the use of food legumes, improving their quality, adaptation and yield and boosting the competitiveness of the agriculture and food sector, the INCREASE strategy will have a major impact on economy and society and represents a case study of integrative and participatory approaches towards conservation and exploitation of crop genetic resources.

Machine learning using the extreme gradient boosting (XGBoost) algorithm predicts 5-day delta of SOFA score at ICU admission in COVID-19 patients

Background

Accurate risk stratification of critically ill patients with coronavirus disease 2019 (COVID-19) is essential for optimizing resource allocation, delivering targeted interventions, and maximizing patient survival probability. Machine learning (ML) techniques are attracting increased interest for the development of prediction models as they excel in the analysis of complex signals in data-rich environments such as critical care.

Methods

We retrieved data on patients with COVID-19 admitted to an intensive care unit (ICU) between March and October 2020 from the RIsk Stratification in COVID-19 patients in the Intensive Care Unit (RISC-19-ICU) registry. We applied the Extreme Gradient Boosting (XGBoost) algorithm to the data to predict as a binary outcome the increase or decrease in patients' Sequential Organ Failure Assessment (SOFA) score on day 5 after ICU admission. The model was iteratively cross-validated in different subsets of the study cohort.

Results

The final study population consisted of 675 patients. The XGBoost model correctly predicted a decrease in SOFA score in 320/385 (83%) critically ill COVID-19 patients, and an increase in the score in 210/290 (72%) patients. The area under the mean receiver operating characteristic curve for XGBoost was significantly higher than that for the logistic regression model (0.86 vs. 0.69, P < 0.01 [paired t-test with 95% confidence interval]).

Conclusions

The XGBoost model predicted the change in SOFA score in critically ill COVID-19 patients admitted to the ICU and can guide clinical decision support systems (CDSSs) aimed at optimizing available resources.

Mask-R[Formula: see text]CNN: a distance-field regression version of Mask-RCNN for fetal-head delineation in ultrasound images

BACKGROUND AND OBJECTIVES

Fetal head-circumference (HC) measurement from ultrasound (US) images provides useful hints for assessing fetal growth. Such measurement is performed manually during the actual clinical practice, posing issues relevant to intra- and inter-clinician variability. This work presents a fully automatic, deep-learning-based approach to HC delineation, which we named Mask-R[Formula: see text]CNN. It advances our previous work in the field and performs HC distance-field regression in an end-to-end fashion, without requiring a priori HC localization nor any postprocessing for outlier removal.

METHODS

Mask-R[Formula: see text]CNN follows the Mask-RCNN architecture, with a backbone inspired by feature-pyramid networks, a region-proposal network and the ROI align. The Mask-RCNN segmentation head is here modified to regress the HC distance field.

RESULTS

Mask-R[Formula: see text]CNN was tested on the HC18 Challenge dataset, which consists of 999 training and 335 testing images. With a comprehensive ablation study, we showed that Mask-R[Formula: see text]CNN achieved a mean absolute difference of 1.95 mm (standard deviation [Formula: see text] mm), outperforming other approaches in the literature.

CONCLUSIONS

With this work, we proposed an end-to-end model for HC distance-field regression. With our experimental results, we showed that Mask-R[Formula: see text]CNN may be an effective support for clinicians for assessing fetal growth.

A Semi-Supervised Multi-Task Learning Approach for Predicting Short-Term Kidney Disease Evolution

Kidney Disease (KD) may hide complex causes and is associated with a tremendous socio-economic impact. A timely identification and management from the first level of medical care represent the most effective strategy to address the growing global burden sustainably. Clinical practice guidelines suggest utilizing estimated Glomerular Filtration Rate (eGFR) for routine evaluation within a screening purpose. Accordingly, the analysis of Electronic Health Records (EHRs) using Machine Learning techniques offers great opportunities to monitor and predict the eGFR trend over time. This paper aims to propose a novel Semi-Supervised Multi-Task Learning (SS-MTL) approach for predicting short-term KD evolution on multiple General Practitioners EHR data. We demonstrated that the SS-MTL approach is able to (i) capture the eGFR temporal evolution by imposing a temporal relatedness between consecutive time-windows and (ii) exploit useful information from unlabeled patients when labeled patients are less numerous with a gain of up to 4.1 % in terms of Recall. This situation reflects the real-case scenario, where available labeled samples are limited, but those unlabeled much more abundant. The SS-MTL approach, also given the high level of interpretability, might be the ideal candidate in general practice to get integrated within a decision support system for KD screening purposes.

Artificial Intelligence for Ultrasound Informative Image Selection of Metacarpal Head Cartilage. A Pilot Study

Objectives: This study aims to develop an automatic deep-learning algorithm, which is based on Convolutional Neural Networks (CNNs), for ultrasound informative-image selection of hyaline cartilage at metacarpal head level. The algorithm performance and that of three beginner sonographers were compared with an expert assessment, which was considered the gold standard.

Methods: The study was divided into two steps. In the first one, an automatic deep-learning algorithm for image selection was developed using 1,600 ultrasound (US) images of the metacarpal head cartilage (MHC) acquired in 40 healthy subjects using a very high-frequency probe (up to 22 MHz). The algorithm task was to identify US images defined informative as they show enough information to fulfill the Outcome Measure in Rheumatology US definition of healthy hyaline cartilage. The algorithm relied on VGG16 CNN, which was fine-tuned to classify US images in informative and non-informative ones. A repeated leave-four-subject out cross-validation was performed using the expert sonographer assessment as gold-standard. In the second step, the expert assessed the algorithm and the beginner sonographers' ability to obtain US informative images of the MHC.

Results: The VGG16 CNN showed excellent performance in the first step, with a mean area (AUC) under the receiver operating characteristic curve, computed among the 10 models obtained from cross-validation, of 0.99 ± 0.01. The model that reached the best AUC on the testing set, which we named “MHC identifier 1,” was then evaluated by the expert sonographer. The agreement between the algorithm, and the expert sonographer was almost perfect [Cohen's kappa: 0.84 (95% confidence interval: 0.71–0.98)], whereas the agreement between the expert and the beginner sonographers using conventional assessment was moderate [Cohen's kappa: 0.63 (95% confidence interval: 0.49–0.76)]. The conventional obtainment of US images by beginner sonographers required 6.0 ± 1.0 min, whereas US videoclip acquisition by a beginner sonographer lasted only 2.0 ± 0.8 min.

Conclusion: This study paves the way for the automatic identification of informative US images for assessing MHC. This may redefine the US reliability in the evaluation of MHC integrity, especially in terms of intrareader reliability and may support beginner sonographers during US training.

A shape-constraint adversarial framework with instance-normalized spatio-temporal features for inter-fetal membrane segmentation

BACKGROUND AND OBJECTIVES

During Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the fetuses. In the current practice, this syndrome is surgically treated by closing the abnormal connections using laser ablation. Surgeons commonly use the inter-fetal membrane as a reference. Limited field of view, low fetoscopic image quality and high inter-subject variability make the membrane identification a challenging task. However, currently available tools are not optimal for automatic membrane segmentation in fetoscopic videos, due to membrane texture homogeneity and high illumination variability.

METHODS

To tackle these challenges, we present a new deep-learning framework for inter-fetal membrane segmentation on in-vivo fetoscopic videos. The framework enhances existing architectures by (i) encoding a novel (instance-normalized) dense block, invariant to illumination changes, that extracts spatio-temporal features to enforce pixel connectivity in time, and (ii) relying on an adversarial training, which constrains macro appearance.

RESULTS

We performed a comprehensive validation using 20 different videos (2000 frames) from 20 different surgeries, achieving a mean Dice Similarity Coefficient of 0.8780±0.1383.

CONCLUSIONS

The proposed framework has great potential to positively impact the actual surgical practice for TTTS treatment, allowing the implementation of surgical guidance systems that can enhance context awareness and potentially lower the duration of the surgeries.

A Machine Learning Approach for Postoperative Outcome Prediction: Surgical Data Science Application in a Thoracic Surgery Setting

BACKGROUND

The use of innovative methodologies, such as Surgical Data Science (SDS), based on artificial intelligence (AI) could prove to be useful for extracting knowledge from clinical data overcoming limitations inherent in medical registries analysis. The aim of the study is to verify if the application of an AI analysis to our database could develop a model able to predict cardiopulmonary complications in patients submitted to lung resection.

METHODS

We retrospectively analyzed data of patients submitted to lobectomy, bilobectomy, segmentectomy and pneumonectomy (January 2006-December 2018). Fifty preoperative characteristics were used for predicting the occurrence of cardiopulmonary complications. The prediction model was developed by training and testing a machine learning (ML) algorithm (XGBOOST) able to deal with registries characterized by missing data. We calculated the receiver operating characteristic curve, true positive rate (TPR), positive predictive value (PPV) and accuracy of the model.

RESULTS

We analyzed 1360 patients (lobectomy: 80.7%, segmentectomy: 11.9%, bilobectomy 3.7%, pneumonectomy: 3.7%) and 23.3% of them experienced cardiopulmonary complications. XGBOOST algorithm generated a model able to predict complications with an area under the curve of 0.75, a TPR of 0.76, a PPV of 0.68. The model's accuracy was 0.70. The algorithm included all the variables in the model regardless of their completeness.

CONCLUSIONS

Using SDS principles in thoracic surgery for the first time, we developed an ML model able to predict cardiopulmonary complications after lung resection based on 50 patient characteristics. The prediction was also possible even in the case of those patients for whom we had incomplete data. This model could improve the process of counseling and the perioperative management of lung resection candidates.

A regression framework to head-circumference delineation from US fetal images

BACKGROUND AND OBJECTIVES

Measuring head-circumference (HC) length from ultrasound (US) images is a crucial clinical task to assess fetus growth. To lower intra- and inter-operator variability in HC length measuring, several computer-assisted solutions have been proposed in the years. Recently, a large number of deep-learning approaches is addressing the problem of HC delineation through the segmentation of the whole fetal head via convolutional neural networks (CNNs). Since the task is a edge-delineation problem, we propose a different strategy based on regression CNNs.

METHODS

The proposed framework consists of a region-proposal CNN for head localization and centering, and a regression CNN for accurately delineate the HC. The first CNN is trained exploiting transfer learning, while we propose a training strategy for the regression CNN based on distance fields.

RESULTS

The framework was tested on the HC18 Challenge dataset, which consists of 999 training and 335 testing images. A mean absolute difference of 1.90 ( ±  1.76) mm and a Dice similarity coefficient of 97.75 ( ±  1.32) % were achieved, overcoming approaches in the literature.

CONCLUSIONS

The experimental results showed the effectiveness of the proposed framework, proving its potential in supporting clinicians during the clinical practice.

Functional evaluation of triceps surae during heel rise test: from EMG frequency analysis to machine learning approach

Soleus muscle flap as coverage tissue is a possible surgical solution adopted to cover the wounds due to open fractures. Despite this procedure presents many clinical advantages, relatively poor information is available about the loss of functionality of triceps surae of the treated leg. In this study, a group of patients who underwent a soleus muscle flap surgical procedure has been analyzed through the heel rise test (HRT), in order to explore the triceps surae residual functionalities. A frequency band analysis was performed in order to assess whether the residual heads of triceps surae exhibit different characteristics with respect to both the non-treated lower limb and an age-matched control group. Then, an in-depth analysis based on a machine learning approach was proposed for discriminating between groups by generalizing across new unseen subjects. Experimental results showed the reliability of the proposed analyses for discriminating between-group at a specific time epoch and the high interpretability of the proposed machine learning algorithm allowed the temporal localization of the most discriminative frequency bands. Findings of this study highlighted that significant differences can be recognized in the myoelectric spectral characteristics between the treated and contralateral leg in patients who underwent soleus flap surgery. These experimental results may support the clinical decision-making for assessing triceps surae performance and for supporting the choice of treatment in plastic and reconstructive surgery. Graphical Abstract The Graphical abstract presents the scope of the proposed analysis of myoelectric signals of soleus and gastrocnemius muscles of patiens groups during Hell Rise Test, highlighting the applied methods and the obtained results.

A Decision Support System for Diabetes Chronic Care Models Based on General Practitioner Engagement and EHR Data Sharing

Objective Decision support systems (DSS) have been developed and promoted for their potential to improve quality of health care. However, there is a lack of common clinical strategy and a poor management of clinical resources and erroneous implementation of preventive medicine. Methods To overcome this problem, this work proposed an integrated system that relies on the creation and sharing of a database extracted from GPs’ Electronic Health Records (EHRs) within the Netmedica Italian (NMI) cloud infrastructure. Although the proposed system is a pilot application specifically tailored for improving the chronic Type 2 Diabetes (T2D) care it could be easily targeted to effectively manage different chronic-diseases. The proposed DSS is based on EHR structure used by GPs in their daily activities following the most updated guidelines in data protection and sharing. The DSS is equipped with a Machine Learning (ML) method for analyzing the shared EHRs and thus tackling the high variability of EHRs. A novel set of T2D care-quality indicators are used specifically to determine the economic incentives and the T2D features are presented as predictors of the proposed ML approach. Results The EHRs from 41237 T2D patients were analyzed. No additional data collection, with respect to the standard clinical practice, was required. The DSS exhibited competitive performance (up to an overall accuracy of 98%±2% and macro-recall of 96%±1%) for classifying chronic care quality across the different follow-up phases. The chronic care quality model brought to a significant increase (up to 12%) of the T2D patients without complications. For GPs who agreed to use the proposed system, there was an economic incentive. A further bonus was assigned when performance targets are achieved. Conclusions The quality care evaluation in a clinical use-case scenario demonstrated how the empowerment of the GPs through the use of the platform (integrating the proposed DSS), along with the economic incentives, may speed up the improvement of care.

Heartbeat Detection by Laser Doppler Vibrometry and Machine Learning

Background: Heartbeat detection is a crucial step in several clinical fields. Laser Doppler Vibrometer (LDV) is a promising non-contact measurement for heartbeat detection. The aim of this work is to assess whether machine learning can be used for detecting heartbeat from the carotid LDV signal. Methods: The performances of Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF) and K-Nearest Neighbor (KNN) were compared using the leave-one-subject-out cross-validation as the testing protocol in an LDV dataset collected from 28 subjects. The classification was conducted on LDV signal windows, which were labeled as beat, if containing a beat, or no-beat, otherwise. The labeling procedure was performed using electrocardiography as the gold standard. Results: For the beat class, the f1-score (f1) values were 0.93, 0.93, 0.95, 0.96 for RF, DT, KNN and SVM, respectively. No statistical differences were found between the classifiers. When testing the SVM on the full-length (10 min long) LDV signals, to simulate a real-world application, we achieved a median macro-f1 of 0.76. Conclusions: Using machine learning for heartbeat detection from carotid LDV signals showed encouraging results, representing a promising step in the field of contactless cardiovascular signal analysis.

Faster R-CNN approach for detection and quantification of DNA damage in comet assay images

BACKGROUND AND OBJECTIVE

DNA damage analysis can provide valuable information in several areas ranging from the diagnosis/treatment of a disease to the monitoring of the effects of genetic and environmental influences. The evaluation of the damage is determined by comet scoring, which can be performed by a skilled operator with a manual procedure. However, this approach becomes very time-consuming and the operator dependency results in the subjectivity of the damage quantification and thus in a high inter/intra-operator variability.

METHODS

In this paper, we aim to overcome this issue by introducing a Deep Learning methodology based on Faster R-CNN to completely automatize the overall approach while discovering unseen discriminative patterns in comets.

RESULTS

The experimental results performed on two real use-case datasets reveal the higher performance (up to mean absolute precision of 0.74) of the proposed methodology against other state-of-the-art approaches. Additionally, the validation procedure performed by expert biologists highlights how the proposed approach is able to unveil true comets, often unseen from the human eye and standard computer vision methodology.

CONCLUSIONS

This work contributes to the biomedical informatics field by the introduction of a novel approach based on established object detection Deep Learning technique for evaluating the DNA damage. The main contribution is the application of Faster R-CNN for the detection and quantification of DNA damage in comet assay images, by fully automatizing the detection/classification DNA damage task. The experimental results extracted in two real use-case datasets demonstrated (i) the higher robustness of the proposed methodology against other state-of-the-art Deep Learning competitors, (ii) the speeding up of the comet analysis procedure and (iii) the minimization of the intra/inter-operator variability.

Preterm Infants' Pose Estimation With Spatio-Temporal Features

OBJECTIVE

Preterm infants' limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants' health status and motor/cognitive development. Herein, we propose a new approach to preterm infants' limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability.

METHODS

Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset).

RESULTS

When applied to pose estimation, the median root mean square distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27 pixels).

CONCLUSION

Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity).

SIGNIFICANCE

This article significantly enhances the state of art in automatic assessment of preterm infants' health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants' pose estimation.

TyG-er: An ensemble Regression Forest approach for identification of clinical factors related to insulin resistance condition using Electronic Health Records

BACKGROUND

Insulin resistance is an early-stage deterioration of Type 2 diabetes. Identification and quantification of insulin resistance requires specific blood tests; however, the triglyceride-glucose (TyG) index can provide a surrogate assessment from routine Electronic Health Record (EHR) data. Since insulin resistance is a multi-factorial condition, to improve its characterisation, this study aims to discover non-trivial clinical factors in EHR data to determine where the insulin-resistance condition is encoded.

METHODS

We proposed a high-interpretable Machine Learning approach (i.e., ensemble Regression Forest combined with data imputation strategies), named TyG-er. We applied three different experimental procedures to test TyG-er reliability on the Italian Federation of General Practitioners dataset, named FIMMG_obs dataset, which is publicly available and reflects the clinical use-case (i.e., not all laboratory exams are prescribed on a regular basis over time).

RESULTS

Results detected non-conventional clinical factors (i.e., uricemia, leukocytes, gamma-glutamyltransferase and protein profile) and provided novel insight into the best combination of clinical factors for detecting early glucose tolerance deterioration. The robustness of these extracted clinical factors was confirmed by the high agreement (from 0.664 to 0.911 of Lin's correlation coefficient (r_c)) of the TyG-er approach among different experimental procedures. Moreover, the results of the three experimental procedures outlined the predictive power of the TyG-er approach (up to a mean absolute error of 5.68% and r_c=0.666,p<.05).

CONCLUSIONS

The TyG-er approach is able to carry information about the identification of the TyG index, strictly correlated with the insulin-resistance condition, while extracting the most relevant non-glycemic features from routine data.

Sharing health data among general practitioners: The Nu.Sa. project

Today, e-health has entered the everyday work flow in the form of a variety of healthcare providers. General practitioners (GPs) are the largest category in the public sanitary service, with about 60,000 GPs throughout Italy. Here, we present the Nu.Sa. project, operating in Italy, which has established one of the first GP healthcare information systems based on heterogeneous data sources. This system connects all providers and provides full access to clinical and health-related data. This goal is achieved through a novel technological infrastructure for data sharing based on interoperability specifications recognised at the national level for messages transmitted from GP providers to the central domain. All data standards are publicly available and subjected to continuous improvement. Currently, the system manages more than 5,000 GPs with about 5,500,000 patients in total, with 4,700,000 pharmacological e-prescriptions and 1,700,000 e-prescriptions for laboratory exams per month. Hence, the Nu.Sa. healthcare system that has the capacity to gather standardised data from 16 different form of GP software, connecting patients, GPs, healthcare organisations, and healthcare professionals across a large and heterogeneous territory through the implementation of data standards with a strong focus on cybersecurity. Results show that the application of this scenario at a national level, with novel metrics on the architecture's scalability and the software's usability, affect the sanitary system and on GPs' professional activities.

Person Re-Identification with RGB-D Camera in Top-View Configuration through Multiple Nearest Neighbor Classifiers and Neighborhood Component Features Selection

Person re-identification is an important topic in retail, scene monitoring, human-computer interaction, people counting, ambient assisted living and many other application fields. A dataset for person re-identification TVPR (Top View Person Re-Identification) based on a number of significant features derived from both depth and color images has been previously built. This dataset uses an RGB-D camera in a top-view configuration to extract anthropometric features for the recognition of people in view of the camera, reducing the problem of occlusions while being privacy preserving. In this paper, we introduce a machine learning method for person re-identification using the TVPR dataset. In particular, we propose the combination of multiple k-nearest neighbor classifiers based on different distance functions and feature subsets derived from depth and color images. Moreover, the neighborhood component feature selection is used to learn the depth features’ weighting vector by minimizing the leave-one-out regularized training error. The classification process is performed by selecting the first passage under the camera for training and using the others as the testing set. Experimental results show that the proposed methodology outperforms standard supervised classifiers widely used for the re-identification task. This improvement encourages the application of this approach in the retail context in order to improve retail analytics, customer service and shopping space management.

HDOMO: Smart Sensor Integration for an Active and Independent Longevity of the Elderly

The aim of this paper is to present the main results of HDOMO, an Ambient Assisted Living (AAL) project that involved 16 Small and Medium Enterprises (SMEs) and 2 research institutes. The objective of the project was to create an autonomous and automated domestic environment, primarily for elderly people and people with physical and motor disabilities. A known and familiar environment should help users in their daily activities and it should act as a virtual caregiver by calling, if necessary, relief efforts. Substantially, the aim of the project is to simplify the life of people in need of support, while keeping them autonomous in their private environment. From a technical point of view, the project provides the use of different Smart Objects (SOs), able to communicate among each other, in a cloud base infrastructure, and with the assisted users and their caregivers, in a perspective of interoperability and standardization of devices, usability and effectiveness of alarm systems. In the state of the art there are projects that achieve only a few of the elements listed. The HDOMO project aims to achieve all of them in one single project effectively. The experimental trials performed in a real scenario demonstrated the accuracy and efficiency of the system in extracting and processing data in real time to promptly acting, and in providing timely response to the needs of the user by integrating and confirming main alarms with different interoperable smart sensors. The article proposes a new technique to improve the accuracy of the system in detecting alarms using a multi-SO approach with information fusion between different devices, proving that this architecture can provide robust and reliable results on real environments.

Embedded Vision Sensor Network for Planogram Maintenance in Retail Environments

A planogram is a detailed visual map that establishes the position of the products in a retail store. It is designed to supply the best location of a product for suppliers to support an innovative merchandising approach, to increase sales and profits and to better manage the shelves. Deviating from the planogram defeats the purpose of any of these goals, and maintaining the integrity of the planogram becomes a fundamental aspect in retail operations. We propose an embedded system, mainly based on a smart camera, able to detect and to investigate the most important parameters in a retail store by identifying the differences with respect to an “approved” planogram. We propose a new solution that allows concentrating all the surveys and the useful measures on a limited number of devices in communication among them. These devices are simple, low cost and ready for immediate installation, providing an affordable and scalable solution to the problem of planogram maintenance. Moreover, over an Internet of Things (IoT) cloud-based architecture, the system supplies many additional data that are not concerning the planogram, e.g., out-of-shelf events, promptly notified through SMS and/or mail. The application of this project allows the realization of highly integrated systems, which are economical, complete and easy to use for a large number of users. Experimental results have proven that the system can efficiently calculate the deviation from a normal situation by comparing the base planogram image with the images grabbed.