An Overview on Image Registration Techniques for Cardiac Diagnosis and Treatment

Independently Trained Multi-Scale Registration Network Based on Image Pyramid

Image registration is a fundamental task in various applications of medical image analysis and plays a crucial role in auxiliary diagnosis, treatment, and surgical navigation. However, cardiac image registration is challenging due to the large non-rigid deformation of the heart and the complex anatomical structure. To address this challenge, this paper proposes an independently trained multi-scale registration network based on an image pyramid. By down-sampling the original input image multiple times, we can construct image pyramid pairs, and design a multi-scale registration network using image pyramid pairs of different resolutions as the training set. Using image pairs of different resolutions, train each registration network independently to extract image features from the image pairs at different resolutions. During the testing stage, the large deformation registration is decomposed into a multi-scale registration process. The deformation fields of different resolutions are fused by a step-by-step deformation method, thereby addressing the challenge of directly handling large deformations. Experiments were conducted on the open cardiac dataset ACDC (Automated Cardiac Diagnosis Challenge); the proposed method achieved an average Dice score of 0.828 in the experimental results. Through comparative experiments, it has been demonstrated that the proposed method effectively addressed the challenge of heart image registration and achieved superior registration results for cardiac images.

Structure-aware independently trained multi-scale registration network for cardiac images

Image registration is a primary task in various medical image analysis applications. However, cardiac image registration is difficult due to the large non-rigid deformation of the heart and the complex anatomical structure. This paper proposes a structure-aware independently trained multi-scale registration network (SIMReg) to address this challenge. Using image pairs of different resolutions, independently train each registration network to extract image features of large deformation image pairs at different resolutions. In the testing stage, the large deformation registration is decomposed into a multi-scale registration process, and the deformation fields of different resolutions are fused by a step-by-step deformation method, thus solving the difficulty of directly processing large deformation. Meanwhile, the targeted introduction of MIND (modality independent neighborhood descriptor) structural features to guide network training enhances the registration of cardiac structural contours and improves the registration effect of local details. Experiments were carried out on the open cardiac dataset ACDC (automated cardiac diagnosis challenge), and the average Dice value of the experimental results of the proposed method was 0.833. Comparative experiments showed that the proposed SIMReg could better solve the problem of heart image registration and achieve a better registration effect on cardiac images.

Joint shape/texture representation learning for cardiovascular disease diagnosis from magnetic resonance imaging

Aims

Cardiovascular diseases (CVDs) are the leading cause of mortality worldwide. Cardiac image and mesh are two primary modalities to present the shape and structure of the heart and have been demonstrated to be efficient in CVD prediction and diagnosis. However, previous research has been generally focussed on a single modality (image or mesh), and few of them have tried to jointly consider the image and mesh representations of heart. To obtain efficient and explainable biomarkers for CVD prediction and diagnosis, it is needed to jointly consider both representations.

Methods and results

We design a novel multi-channel variational auto-encoder, mesh-image variational auto-encoder, to learn joint representation of paired mesh and image. After training, the shape-aware image representation (SAIR) can be learned directly from the raw images and applied for further CVD prediction and diagnosis. We demonstrate our method on data from UK Biobank study and two other datasets via extensive experiments. In acute myocardial infarction prediction, SAIR achieves 81.43% accuracy, significantly higher than traditional biomarkers like metadata and clinical indices (left ventricle and right ventricle clinical indices of cardiac function like chamber volume, mass, and ejection fraction).

Conclusion

Our mesh-image variational auto-encoder provides a novel approach for 3D cardiac mesh reconstruction from images. The extraction of SAIR is fast and without need of segmentation masks, and its focussing can be visualized in the corresponding cardiac meshes. SAIR archives better performance than traditional biomarkers and can be applied as an efficient supplement to them, which is of significant potential in CVD analysis.

Multi-modality cardiac image computing: A survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

Quantitative assessment of radiotherapy-induced myocardial damage using MRI: a systematic review

PURPOSE

To determine the role of magnetic resonance imaging (MRI)-based metrics to quantify myocardial toxicity following radiotherapy (RT) in human subjects through review of current literature.

METHODS

Twenty-one MRI studies published between 2011-2022 were identified from available databases. Patients received chest irradiation with/without other treatments for various malignancies including breast, lung, esophageal cancer, Hodgkin's, and non-Hodgkin's lymphoma. In 11 longitudinal studies, the sample size, mean heart dose, and follow-up times ranged from 10-81 patients, 2.0-13.9 Gy, and 0-24 months after RT (in addition to a pre-RT assessment), respectively. In 10 cross-sectional studies, the sample size, mean heart dose, and follow-up times ranged from 5-80 patients, 2.1-22.9 Gy, and 2-24 years from RT completion, respectively. Global metrics of left ventricle ejection fraction (LVEF) and mass/dimensions of cardiac chambers were recorded, along with global/regional values of T1/T2 signal, extracellular volume (ECV), late gadolinium enhancement (LGE), and circumferential/radial/longitudinal strain.

RESULTS

LVEF tended to decline at >20 years follow-up and in patients treated with older RT techniques. Changes in global strain were observed after shorter follow-up (13±2 months) for concurrent chemoradiotherapy. In concurrent treatments with longer follow-up (8.3 years), increases in left ventricle (LV) mass index were correlated with LV mean dose. In pediatric patients, increases in LV diastolic volume were correlated with heart/LV dose at 2 years post-RT. Regional changes were observed earlier post-RT. Dose-dependent responses were reported for several parameters, including: increased T1 signal in high-dose regions, a 0.136% increase of ECV per Gy, progressive increase of LGE with increasing dose at regions receiving >30 Gy, and correlation between increases in LV scarring volume and LV mean/V10/V25 Gy dose.

CONCLUSION

Global metrics only detected changes over longer follow-up, in older RT techniques, in concurrent treatments, and in pediatric patients. In contrast, regional measurements detected myocardial damage at shorter follow-up and in RT treatments without concurrent treatment and had greater potential for dose-dependent response. The early detection of regional changes suggests the importance of regional quantification of RT-induced myocardial toxicity at early stages, before damage becomes irreversible. Further works with homogeneous cohorts are required to examine this matter.

A bidirectional registration neural network for cardiac motion tracking using cine MRI images

Using cine magnetic resonance imaging (cine MRI) images to track cardiac motion helps users to analyze the myocardial strain, and is of great importance in clinical applications. At present, most of the automatic deep learning-based motion tracking methods compare two images without considering temporal information between MRI frames, which easily leads to the lack of consistency of the generated motion fields. Even though a small number of works take into account the temporal factor, they are usually computationally intensive or have limitations on image length. To solve this problem, we propose a bidirectional convolution neural network for motion tracking of cardiac cine MRI images. This network leverages convolutional blocks to extract spatial features from three-dimensional (3D) image registration pairs, and models the temporal relations through a bidirectional recurrent neural network to obtain the Lagrange motion field between the reference image and other images. Compared with previous pairwise registration methods, the proposed method can automatically learn spatiotemporal information from multiple images with fewer parameters. We evaluated our model on three public cardiac cine MRI datasets. The experimental results demonstrated that the proposed method can significantly improve the motion tracking accuracy. The average Dice coefficient between estimated segmentation and manual segmentation has reached almost 0.85 on the widely used Automatic Cardiac Diagnostic Challenge (ACDC) dataset.

The digitization process and the evolution of Clinical Risk Management concept: The role of Clinical Engineering in the operational management of biomedical technologies

Introduction

Digital transformation and technological innovation which have influenced several areas of social and productive life in recent years, are now also a tangible and concrete reality in the vast and strategic sector of public healthcare. The progressive introduction of digital technologies and their widespread diffusion in many segments of the population undoubtedly represent a driving force both for the evolution of care delivery methods and for the introduction of new organizational and management methods within clinical structures.

Methods

The CS Clinical Engineering of the "Spedali Civili Hospital in Brescia" decided to design a path that would lead to the development of a software for the management of biomedical technologies within its competence inside the hospital. The ultimate aim of this path stems from the need of Clinical Engineering Department to have up-to-date, realistic, and systematic control of all biomedical technologies present in the company. "Spedali Civili Hospital in Brescia" is not just one of the most important corporate realities in the city, but it is also the largest hospital in Lombardy and one of the largest in Italy. System development has followed the well-established phases: requirement analysis phase, development phase, release phase and evaluating and updating phase.

Results

Finally, cooperation between the various figures involved in the multidisciplinary working group led to the development of an innovative management software called "SIC Brescia".

Discussion

The contribution of the present paper is to illustrate the development of a complex implementation model for the digitization of processes, information relating to biomedical technologies and their management throughout the entire life cycle. The purpose of sharing this path is to highlight the methodologies followed for its realization, the results obtained and possible future developments. This may enable other realities in the healthcare context to undertake the same type of pathway inspired by an accomplished model. Furthermore, future implementation and data collection related to the proposed Key Performance Indicators, as well as the consequent development of new operational management models for biomedical technologies and maintenance processes will be possible. In this way, the Clinical Risk Management concept will also be able to evolve into a more controlled, safe, and efficient system for the patient and the user.

An Overview of Deep Learning Methods for Left Ventricle Segmentation

Cardiac health diseases are one of the key causes of death around the globe. The number of heart patients has considerably increased during the pandemic. Therefore, it is crucial to assess and analyze the medical and cardiac images. Deep learning architectures, specifically convolutional neural networks have profoundly become the primary choice for the assessment of cardiac medical images. The left ventricle is a vital part of the cardiovascular system where the boundary and size perform a significant role in the evaluation of cardiac function. Due to automatic segmentation and good promising results, the left ventricle segmentation using deep learning has attracted a lot of attention. This article presents a critical review of deep learning methods used for the left ventricle segmentation from frequently used imaging modalities including magnetic resonance images, ultrasound, and computer tomography. This study also demonstrates the details of the network architecture, software, and hardware used for training along with publicly available cardiac image datasets and self-prepared dataset details incorporated. The summary of the evaluation matrices with results used by different researchers is also presented in this study. Finally, all this information is summarized and comprehended in order to assist the readers to understand the motivation and methodology of various deep learning models, as well as exploring potential solutions to future challenges in LV segmentation.

Fully Automatic Left Ventricle Segmentation Using Bilateral Lightweight Deep Neural Network

The segmentation of the left ventricle (LV) is one of the fundamental procedures that must be performed to obtain quantitative measures of the heart, such as its volume, area, and ejection fraction. In clinical practice, the delineation of LV is still often conducted semi-automatically, leaving it open to operator subjectivity. The automatic LV segmentation from echocardiography images is a challenging task due to poorly defined boundaries and operator dependency. Recent research has demonstrated that deep learning has the capability to employ the segmentation process automatically. However, the well-known state-of-the-art segmentation models still lack in terms of accuracy and speed. This study aims to develop a single-stage lightweight segmentation model that precisely and rapidly segments the LV from 2D echocardiography images. In this research, a backbone network is used to acquire both low-level and high-level features. Two parallel blocks, known as the spatial feature unit and the channel feature unit, are employed for the enhancement and improvement of these features. The refined features are merged by an integrated unit to segment the LV. The performance of the model and the time taken to segment the LV are compared to other established segmentation models, DeepLab, FCN, and Mask RCNN. The model achieved the highest values of the dice similarity index (0.9446), intersection over union (0.8445), and accuracy (0.9742). The evaluation metrics and processing time demonstrate that the proposed model not only provides superior quantitative results but also trains and segments the LV in less time, indicating its improved performance over competing segmentation models.

An Intelligent Mental Health Identification Method for College Students: A Mixed-Method Study

PURPOSE

Mental health assessments that combine patients' facial expressions and behaviors have been proven effective, but screening large-scale student populations for mental health problems is time-consuming and labor-intensive. This study aims to provide an efficient and accurate intelligent method for further psychological diagnosis and treatment, which combines artificial intelligence technologies to assist in evaluating the mental health problems of college students.

MATERIALS AND METHODS

We propose a mixed-method study of mental health assessment that combines psychological questionnaires with facial emotion analysis to comprehensively evaluate the mental health of students on a large scale. The Depression Anxiety and Stress Scale-21(DASS-21) is used for the psychological questionnaire. The facial emotion recognition model is implemented by transfer learning based on neural networks, and the model is pre-trained using FER2013 and CFEE datasets. Among them, the FER2013 dataset consists of 48 × 48-pixel face gray images, a total of 35,887 face images. The CFEE dataset contains 950,000 facial images with annotated action units (au). Using a random sampling strategy, we sent online questionnaires to 400 college students and received 374 responses, and the response rate was 93.5%. After pre-processing, 350 results were available, including 187 male and 153 female students. First, the facial emotion data of students were collected in an online questionnaire test. Then, a pre-trained model was used for emotion recognition. Finally, the online psychological questionnaire scores and the facial emotion recognition model scores were collated to give a comprehensive psychological evaluation score.

RESULTS

The experimental results of the facial emotion recognition model proposed to show that its classification results are broadly consistent with the mental health survey results. This model can be used to improve efficiency. In particular, the accuracy of the facial emotion recognition model proposed in this paper is higher than that of the general mental health model, which only uses the traditional single questionnaire. Furthermore, the absolute errors of this study in the three symptoms of depression, anxiety, and stress are lower than other mental health survey results and are only 0.8%, 8.1%, 3.5%, and 1.8%, respectively.

CONCLUSION

The mixed method combining intelligent methods and scales for mental health assessment has high recognition accuracy. Therefore, it can support efficient large-scale screening of students' psychological problems.

Comparative studies of deep learning segmentation models for left ventricle segmentation

One of the primary factors contributing to death across all age groups is cardiovascular disease. In the analysis of heart function, analyzing the left ventricle (LV) from 2D echocardiographic images is a common medical procedure for heart patients. Consistent and accurate segmentation of the LV exerts significant impact on the understanding of the normal anatomy of the heart, as well as the ability to distinguish the aberrant or diseased structure of the heart. Therefore, LV segmentation is an important and critical task in medical practice, and automated LV segmentation is a pressing need. The deep learning models have been utilized in research for automatic LV segmentation. In this work, three cutting-edge convolutional neural network architectures (SegNet, Fully Convolutional Network, and Mask R-CNN) are designed and implemented to segment the LV. In addition, an echocardiography image dataset is generated, and the amount of training data is gradually increased to measure segmentation performance using evaluation metrics. The pixel's accuracy, precision, recall, specificity, Jaccard index, and dice similarity coefficients are applied to evaluate the three models. The Mask R-CNN model outperformed the other two models in these evaluation metrics. As a result, the Mask R-CNN model is used in this study to examine the effect of training data. For 4,000 images, the network achieved 92.21% DSC value, 85.55% Jaccard index, 98.76% mean accuracy, 96.81% recall, 93.15% precision, and 96.58% specificity value. Relatively, the Mask R-CNN outperformed other architectures, and the performance achieves stability when the model is trained using more than 4,000 training images.

Construction of a Diagnostic Model for Lymph Node Metastasis of the Papillary Thyroid Carcinoma Using Preoperative Ultrasound Features and Imaging Omics

In this paper, we mainly adopted 337 patients who had undergone the surgery on lymph node metastasis of papillary thyroid carcinoma (PTC) as the sample population. In order to provide clinical reference for the intelligent decision-making in treatment plan and improvement of prognosis, we utilized ultrasound features and imaging features to construct five early diagnosis models for patients based on the ultrasound features, imaging features, and combined features. The model integrated with broad learning system (BLS) showed the best performance, with the area under the curve (AUC) of 0.857 (95% confidence interval (CI): 0.811–0.902)) and the accuracy of 0.805 (95% CI: 0.759–0.850). For demographic and clinical features, the prediction effect was also good, with the AUC more than 0.700.

Prioritisation Assessment and Robust Predictive System for Medical Equipment: A Comprehensive Strategic Maintenance Management

The advancement of technology in medical equipment has significantly improved healthcare services. However, failures in upkeeping reliability, availability, and safety affect the healthcare services quality and significant impact can be observed in operations' expenses. The effective and comprehensive medical equipment assessment and monitoring throughout the maintenance phase of the asset life cycle can enhance the equipment reliability, availability, and safety. The study aims to develop the prioritisation assessment and predictive systems that measure the priority of medical equipment's preventive maintenance, corrective maintenance, and replacement programmes. The proposed predictive model is constructed by analysing features of 13,352 medical equipment used in public healthcare clinics in Malaysia. The proposed system comprises three stages: prioritisation analysis, model training, and predictive model development. In this study, we proposed 16 combinations of novel features to be used for prioritisation assessment and prediction of preventive maintenance, corrective maintenance, and replacement programme. The modified k-Means algorithm is proposed during the prioritisation analysis to automatically distinguish raw data into three main clusters of prioritisation assessment. Subsequently, these clusters are fed into and tested with six machine learning algorithms for the predictive prioritisation system. The best predictive models for medical equipment's preventive maintenance, corrective maintenance, and replacement programmes are selected among the tested machine learning algorithms. Findings indicate that the Support Vector Machine performs the best in preventive maintenance and replacement programme prioritisation predictive systems with the highest accuracy of 99.42 and 99.80%, respectively. Meanwhile, K-Nearest Neighbour yielded the highest accuracy in corrective maintenance prioritisation predictive systems with 98.93%. Based on the promising results, clinical engineers and healthcare providers can widely adopt the proposed prioritisation assessment and predictive systems in managing expenses, reporting, scheduling, materials, and workforce.

Multistage Optimization Using a Modified Gaussian Mixture Model in Sperm Motility Tracking

Infertility is a condition whereby pregnancy does not occur despite having unprotected sexual intercourse for at least one year. The main reason could originate from either the male or the female, and sometimes, both contribute to the fertility disorder. For the male, sperm disorder was found to be the most common reason for infertility. In this paper, we proposed male infertility analysis based on automated sperm motility tracking. The proposed method worked in multistages, where the first stage focused on the sperm detection process using an improved Gaussian Mixture Model. A new optimization protocol was proposed to accurately detect the motile sperms prior to the sperm tracking process. Since the optimization protocol was imposed in the proposed system, the sperm tracking and velocity estimation processes are improved. The proposed method attained the highest average accuracy, sensitivity, and specificity of 92.3%, 96.3%, and 72.4%, respectively, when tested on 10 different samples. Our proposed method depicted better sperm detection quality when qualitatively observed as compared to other state-of-the-art techniques.

Joint Deep Learning Framework for Image Registration and Segmentation of Late Gadolinium Enhanced MRI and Cine Cardiac MRI

Late gadolinium enhanced (LGE) cardiac magnetic resonance (CMR) imaging, the current benchmark for assessment of myocardium viability, enables the identification and quantification of the compromised myocardial tissue regions, as they appear hyper-enhanced compared to the surrounding, healthy myocardium. However, in LGE CMR images, the reduced contrast between the left ventricle (LV) myocardium and LV blood-pool hampers the accurate delineation of the LV myocardium. On the other hand, the balanced-Steady State Free Precession (bSSFP) cine CMR imaging provides high resolution images ideal for accurate segmentation of the cardiac chambers. In the interest of generating patient-specific hybrid 3D and 4D anatomical models of the heart, to identify and quantify the compromised myocardial tissue regions for revascularization therapy planning, in our previous work, we presented a spatial transformer network (STN) based convolutional neural network (CNN) architecture for registration of LGE and bSSFP cine CMR image datasets made available through the 2019 Multi-Sequence Cardiac Magnetic Resonance Segmentation Challenge (MS-CMRSeg). We performed a supervised registration by leveraging the region of interest (RoI) information using the manual annotations of the LV blood-pool, LV myocardium and right ventricle (RV) blood-pool provided for both the LGE and the bSSFP cine CMR images. In order to reduce the reliance on the number of manual annotations for training such network, we propose a joint deep learning framework consisting of three branches: a STN based RoI guided CNN for registration of LGE and bSSFP cine CMR images, an U-Net model for segmentation of bSSFP cine CMR images, and an U-Net model for segmentation of LGE CMR images. This results in learning of a joint multi-scale feature encoder by optimizing all three branches of the network architecture simultaneously. Our experiments show that the registration results obtained by training 25 of the available 45 image datasets in a joint deep learning framework is comparable to the registration results obtained by stand-alone STN based CNN model by training 35 of the available 45 image datasets and also shows significant improvement in registration performance when compared to the results achieved by the stand-alone STN based CNN model by training 25 of the available 45 image datasets.

To Align Multimodal Lumbar Spine Images via Bending Energy Constrained Normalized Mutual Information

To align multimodal images is important for information fusion, clinical diagnosis, treatment planning, and delivery, while few methods have been dedicated to matching computerized tomography (CT) and magnetic resonance (MR) images of lumbar spine. This study proposes a coarse-to-fine registration framework to address this issue. Firstly, a pair of CT-MR images are rigidly aligned for global positioning. Then, a bending energy term is penalized into the normalized mutual information for the local deformation of soft tissues. In the end, the framework is validated on 40 pairs of CT-MR images from our in-house collection and 15 image pairs from the SpineWeb database. Experimental results show high overlapping ratio (in-house collection, vertebrae 0.97 ± 0.02, blood vessel 0.88 ± 0.07; SpineWeb, vertebrae 0.95 ± 0.03, blood vessel 0.93 ± 0.10) and low target registration error (in-house collection, ≤2.00 ± 0.62 mm; SpineWeb, ≤2.37 ± 0.76 mm) are achieved. The proposed framework concerns both the incompressibility of bone structures and the nonrigid deformation of soft tissues. It enables accurate CT-MR registration of lumbar spine images and facilitates image fusion, spine disease diagnosis, and interventional treatment delivery.

A Supervised Image Registration Approach for Late Gadolinium Enhanced MRI and Cine Cardiac MRI Using Convolutional Neural Networks

Late gadolinium enhanced (LGE) cardiac magnetic resonance (CMR) imaging is the current gold standard for assessing myocardium viability for patients diagnosed with myocardial infarction, myocarditis or cardiomyopathy. This imaging method enables the identification and quantification of myocardial tissue regions that appear hyper-enhanced. However, the delineation of the myocardium is hampered by the reduced contrast between the myocardium and the left ventricle (LV) blood-pool due to the gadolinium-based contrast agent. The balanced-Steady State Free Precession (bSSFP) cine CMR imaging provides high resolution images with superior contrast between the myocardium and the LV blood-pool. Hence, the registration of the LGE CMR images and the bSSFP cine CMR images is a vital step for accurate localization and quantification of the compromised myocardial tissue. Here, we propose a Spatial Transformer Network (STN) inspired convolutional neural network (CNN) architecture to perform supervised registration of bSSFP cine CMR and LGE CMR images. We evaluate our proposed method on the 2019 Multi-Sequence Cardiac Magnetic Resonance Segmentation Challenge (MS-CMRSeg) dataset and use several evaluation metrics, including the center-to-center LV and right ventricle (RV) blood-pool distance, and the contour-to-contour blood-pool and myocardium distance between the LGE and bSSFP CMR images. Specifically, we showed that our registration method reduced the bSSFP to LGE LV blood-pool center distance from 3.28mm before registration to 2.27mm post registration and RV blood-pool center distance from 4.35mm before registration to 2.52mm post registration. We also show that the average surface distance (ASD) between bSSFP and LGE is reduced from 2.53mm to 2.09mm, 1.78mm to 1.40mm and 2.42mm to 1.73mm for LV blood-pool, LV myocardium and RV blood-pool, respectively.