Deep learning empowered volume delineation of whole-body organs-at-risk for accelerated radiotherapy

Automatic detection of cognitive impairment in patients with white matter hyperintensity and causal analysis of related factors using artificial intelligence of MRI

PURPOSE

White matter hyperintensity (WMH) is a common feature of brain aging, often linked with cognitive decline and dementia. This study aimed to employ deep learning and radiomics to develop models for detecting cognitive impairment in WMH patients and to analyze the causal relationships among cognitive impairment and related factors.

MATERIALS AND METHODS

A total of 79 WMH patients from hospital 1 were randomly divided into a training set (62 patients) and a testing set (17 patients). Additionally, 29 patients from hospital 2 were included as an independent testing set. All participants underwent formal neuropsychological assessments to determine cognitive status. Automated identification and segmentation of WMH were conducted using VB-net, with extraction of radiomics features from cortex, white matter, and nuclei. Four machine learning classifiers were trained on the training set and validated on the testing set to detect cognitive impairment. Model performances were evaluated and compared. Causal analyses were conducted among cortex, white matter, nuclei alterations, and cognitive impairment.

RESULTS

Among the models, the logistic regression (LR) model based on white matter features demonstrated the highest performance, achieving an AUC of 0.819 in the external test dataset. Causal analyses indicated that age, education level, alterations in cortex, white matter, and nuclei were causal factors of cognitive impairment.

CONCLUSION

The LR model based on white matter features exhibited high accuracy in detecting cognitive impairment in WMH patients. Furthermore, the possible causal relationships among alterations in cortex, white matter, nuclei, and cognitive impairment were elucidated.

Charged Gold Nanoparticles for Target Identification-Alignment and Automatic Segmentation of CT Image-Guided Adaptive Radiotherapy in Small Hepatocellular Carcinoma

Because of the challenges posed by anatomical uncertainties and the low resolution of plain computed tomography (CT) scans, implementing adaptive radiotherapy (ART) for small hepatocellular carcinoma (sHCC) using artificial intelligence (AI) faces obstacles in tumor identification-alignment and automatic segmentation. The current study aims to improve sHCC imaging for ART using a gold nanoparticle (Au NP)-based CT contrast agent to enhance AI-driven automated image processing. The synthesized charged Au NPs demonstrated notable in vitro aggregation, low cytotoxicity, and minimal organ toxicity. Over time, an in situ sHCC mouse model was established for in vivo CT imaging at multiple time points. The enhanced CT images processed using 3D U-Net and 3D Trans U-Net AI models demonstrated high geometric and dosimetric accuracy. Therefore, charged Au NPs enable accurate and automatic sHCC segmentation in CT images using classical AI models, potentially addressing the technical challenges related to tumor identification, alignment, and automatic segmentation in CT-guided online ART.

A Radiological-Radiomics model for differentiation between minimally invasive adenocarcinoma and invasive adenocarcinoma less than or equal to 3 cm: A two-center retrospective study

OBJECTIVE

To develop a Radiological-Radiomics (R-R) combined model for differentiation between minimal invasive adenocarcinoma (MIA) and invasive adenocarcinoma (IA) of lung adenocarcinoma (LUAD) and evaluate its predictive performance.

METHODS

The clinical, pathological, and imaging data of a total of 509 patients (522 lesions) with LUAD diagnosed by surgical pathology from 2 medical centres were retrospectively collected, with 392 patients (402 lesions) from center 1 trained and validated using a five-fold cross-validation method, and 117 patients (120 lesions) from center 2 serving as an independent external test set. The least absolute shrinkage and selection operator (LASSO) method was utilized to filter features. Logistic regression was used to construct three models for predicting IA, namely, Radiological model, Radiomics model, and R-R model. Also, receiver operating curve curves (ROCs) were plotted, generating corresponding area under the curve (AUC), sensitivity, specificity, and accuracy.

RESULTS

The R-R model for IA prediction achieved an AUC of 0.918 (95 % CI: 0.889-0.947), a sensitivity of 80.3 %, a specificity of 88.2 %, and an accuracy of 82.1 % in the training set. In the validation set, this model exhibited an AUC of 0.906 (95 % CI: 0.842-0.970), a sensitivity of 79.9 %, a specificity of 88.1 %, and an accuracy of 81.8 %. In the external test set, the AUC was 0.894 (95 % CI: 0.824-0.964), a sensitivity of 84.8 %, a specificity of 78.6 %, and an accuracy of 83.3 %.

CONCLUSION

The R-R model showed excellent diagnostic performance in differentiating MIA and IA, which can provide a certain reference for clinical diagnosis and surgical treatment plans.

Deep learning promoted target volumes delineation of total marrow and total lymphoid irradiation for accelerated radiotherapy: A multi-institutional study

BACKGROUND AND PURPOSE

One of the current roadblocks to the widespread use of Total Marrow Irradiation (TMI) and Total Marrow and Lymphoid Irradiation (TMLI) is the challenging difficulties in tumor target contouring workflow. This study aims to develop a hybrid neural network model that promotes accurate, automatic, and rapid segmentation of multi-class clinical target volumes.

MATERIALS AND METHODS

Patients who underwent TMI and TMLI from January 2018 to May 2022 were included. Two independent oncologists manually contoured eight target volumes for patients on CT images. A novel Dual-Encoder Alignment Network (DEA-Net) was developed and trained using 46 patients from one internal institution and independently evaluated on a total of 39 internal and external patients. Performance was evaluated on accuracy metrics and delineation time.

RESULTS

The DEA-Net achieved a mean dice similarity coefficient of 90.1 % ± 1.8 % for internal testing dataset (23 patients) and 91.1 % ± 2.5 % for external testing dataset (16 patients). The 95 % Hausdorff distance and average symmetric surface distance were 2.04 ± 0.62 mm and 0.57 ± 0.11 mm for internal testing dataset, and 2.17 ± 0.68 mm, and 0.57 ± 0.20 mm for external testing dataset, respectively, outperforming most of existing state-of-the-art methods. In addition, the automatic segmentation workflow reduced delineation time by 98 % compared to the conventional manual contouring process (mean 173 ± 29 s vs. 12168 ± 1690 s; P < 0.001). Ablation study validate the effectiveness of hybrid structures.

CONCLUSION

The proposed deep learning framework achieved comparable or superior target volume delineation accuracy, significantly accelerating the radiotherapy planning process.

Clinical evaluation of the efficacy of limbus artificial intelligence software to augment contouring for prostate and nodes radiotherapy

OBJECTIVES

To determine if Limbus, an artificial intelligence (AI) auto-contouring software, can offer meaningful time savings for prostate radiotherapy treatment planning.

METHODS

Three clinical oncologists recorded the time taken to contour prostate and seminal vesicles, lymph nodes, bladder, rectum, bowel, and femoral heads on CT scans for 30 prostate patients (15 prostate, 15 prostate and nodes). Limbus 1.6.0 was used to generate these contours on the 30 CT scans. The time taken by the oncologists to modify individual Limbus contours was noted and compared with manual contouring times. The geometric similarity of Limbus and expert contours was assessed using the Dice Similarity Coefficient (DSC), and the dosimetric impact of using un-edited Limbus organs at risk contours was studied.

RESULTS

Limbus reduced the time to produce clinically acceptable contours by 26 minutes for prostate and nodes patients and by 13 minutes for the prostate only patients. DSC values of greater than 0.7 were calculated for all contours, demonstrating good initial agreement. A dosimetric study showed that 5 of the 20 plans optimized using unmodified AI structures required unnecessary compromise of PTV coverage, highlighting the importance of expert review.

CONCLUSIONS

Limbus offers significant time saving and has become an essential part of our clinical practice.

ADVANCES IN KNOWLEDGE

This article is the first to include bowel and lymph nodes when assessing potential time savings using Limbus software. It demonstrates that Limbus can be used as an aid for prostate and node radiotherapy treatment planning.

Exploration of anatomical distribution of brain metastasis from breast cancer at first diagnosis assisted by artificial intelligence

Objectives

This study aimed to explore the spatial distribution of brain metastases (BMs) from breast cancer (BC) and to identify the high-risk sub-structures in BMs that are involved at first diagnosis.

Methods

Magnetic resonance imaging (MRI) scans were retrospectively reviewed at our centre. The brain was divided into eight regions according to its anatomy and function, and the volume of each region was calculated. The identification and volume calculation of metastatic brain lesions were accomplished using an automatically segmented 3D BUC-Net model. The observed and expected rates of BMs were compared using 2-tailed proportional hypothesis testing.

Results

A total of 250 patients with BC who presented with 1694 BMs were retrospectively identified. The overall observed incidences of the substructures were as follows: cerebellum, 42.1 %; frontal lobe, 20.1 %; occipital lobe, 9.7 %; temporal lobe, 8.0 %; parietal lobe, 13.1 %; thalamus, 4.7 %; brainstem, 0.9 %; and hippocampus, 1.3 %. Compared with the expected rate based on the volume of different brain regions, the cerebellum, occipital lobe, and thalamus were identified as higher risk regions for BMs (P value ≤ 5.6*10-3). Sub-group analysis according to the type of BC indicated that patients with triple-negative BC had a high risk of involvement of the hippocampus and brainstem.

Conclusions

Among patients with BC, the cerebellum, occipital lobe and thalamus were identified as higher-risk regions than expected for BMs. The brainstem and hippocampus were high-risk areas of the BMs in triple negative breast cancer. However, further validation of this conclusion requires a larger sample size.

Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation

Precise delineation of multiple organs or abnormal regions in the human body from medical images plays an essential role in computer-aided diagnosis, surgical simulation, image-guided interventions, and especially in radiotherapy treatment planning. Thus, it is of great significance to explore automatic segmentation approaches, among which deep learning-based approaches have evolved rapidly and witnessed remarkable progress in multi-organ segmentation. However, obtaining an appropriately sized and fine-grained annotated dataset of multiple organs is extremely hard and expensive. Such scarce annotation limits the development of high-performance multi-organ segmentation models but promotes many annotation-efficient learning paradigms. Among these, studies on transfer learning leveraging external datasets, semi-supervised learning including unannotated datasets and partially-supervised learning integrating partially-labeled datasets have led the dominant way to break such dilemmas in multi-organ segmentation. We first review the fully supervised method, then present a comprehensive and systematic elaboration of the 3 abovementioned learning paradigms in the context of multi-organ segmentation from both technical and methodological perspectives, and finally summarize their challenges and future trends.

Accurate object localization facilitates automatic esophagus segmentation in deep learning

BACKGROUND

Currently, automatic esophagus segmentation remains a challenging task due to its small size, low contrast, and large shape variation. We aimed to improve the performance of esophagus segmentation in deep learning by applying a strategy that involves locating the object first and then performing the segmentation task.

METHODS

A total of 100 cases with thoracic computed tomography scans from two publicly available datasets were used in this study. A modified CenterNet, an object location network, was employed to locate the center of the esophagus for each slice. Subsequently, the 3D U-net and 2D U-net_coarse models were trained to segment the esophagus based on the predicted object center. A 2D U-net_fine model was trained based on the updated object center according to the 3D U-net model. The dice similarity coefficient and the 95% Hausdorff distance were used as quantitative evaluation indexes for the delineation performance. The characteristics of the automatically delineated esophageal contours by the 2D U-net and 3D U-net models were summarized. Additionally, the impact of the accuracy of object localization on the delineation performance was analyzed. Finally, the delineation performance in different segments of the esophagus was also summarized.

RESULTS

The mean dice coefficient of the 3D U-net, 2D U-net_coarse, and 2D U-net_fine models were 0.77, 0.81, and 0.82, respectively. The 95% Hausdorff distance for the above models was 6.55, 3.57, and 3.76, respectively. Compared with the 2D U-net, the 3D U-net has a lower incidence of delineating wrong objects and a higher incidence of missing objects. After using the fine object center, the average dice coefficient was improved by 5.5% in the cases with a dice coefficient less than 0.75, while that value was only 0.3% in the cases with a dice coefficient greater than 0.75. The dice coefficients were lower for the esophagus between the orifice of the inferior and the pulmonary bifurcation compared with the other regions.

CONCLUSION

The 3D U-net model tended to delineate fewer incorrect objects but also miss more objects. Two-stage strategy with accurate object location could enhance the robustness of the segmentation model and significantly improve the esophageal delineation performance, especially for cases with poor delineation results.

Radiomics analysis of pancreas based on dual-energy computed tomography for the detection of type 2 diabetes mellitus

Objective

To utilize radiomics analysis on dual-energy CT images of the pancreas to establish a quantitative imaging biomarker for type 2 diabetes mellitus.

Materials and methods

In this retrospective study, 78 participants (45 with type 2 diabetes mellitus, 33 without) underwent a dual energy CT exam. Pancreas regions were segmented automatically using a deep learning algorithm. From these regions, radiomics features were extracted. Additionally, 24 clinical features were collected for each patient. Both radiomics and clinical features were then selected using the least absolute shrinkage and selection operator (LASSO) technique and then build classifies with random forest (RF), support vector machines (SVM) and Logistic. Three models were built: one using radiomics features, one using clinical features, and a combined model.

Results

Seven radiomic features were selected from the segmented pancreas regions, while eight clinical features were chosen from a pool of 24 using the LASSO method. These features were used to build a combined model, and its performance was evaluated using five-fold cross-validation. The best classifier type is Logistic and the reported area under the curve (AUC) values on the test dataset were 0.887 (0.73-1), 0.881 (0.715-1), and 0.922 (0.804-1) for the respective models.

Conclusion

Radiomics analysis of the pancreas on dual-energy CT images offers potential as a quantitative imaging biomarker in the detection of type 2 diabetes mellitus.

Emerging pharmacotherapy trends in preventing and managing oral mucositis induced by chemoradiotherapy and targeted agents

INTRODUCTION

The introduction of targeted therapy and immunotherapy has tremendously changed the clinical outcomes and prognosis of cancer patients. Despite innovative pharmacological therapies and improved radiotherapy (RT) techniques, patients continue to suffer from side effects, of which oral mucositis (OM) is still the most impactful, especially for quality of life.

AREAS COVERED

We provide an overview of current advances in cancer pharmacotherapy and RT, in relation to their potential to cause OM, and of the less explored and more recent literature reports related to the best management of OM. We have analyzed natural/antioxidant agents, probiotics, mucosal protectants and healing coadjuvants, pharmacotherapies, immunomodulatory and anticancer agents, photobiomodulation and the impact of technology.

EXPERT OPINION

The discovery of more precise pathophysiologic mechanisms of CT and RT-induced OM has outlined that OM has a multifactorial origin, including direct effects, oxidative damage, upregulation of immunologic factors, and effects on oral flora. A persistent upregulated immune response, associated with factors related to patients' characteristics, may contribute to more severe and long-lasting OM. The goal is strategies to conjugate individual patient, disease, and therapy-related factors to guide OM prevention or treatment. Despite further high-quality research is warranted, the issue of prevention is paramount in future strategies.

Deep-Learning for Rapid Estimation of the Out-of-Field Dose in External Beam Photon Radiation Therapy - A Proof of Concept

PURPOSE

The dose deposited outside of the treatment field during external photon beam radiation therapy treatment, also known as out-of-field dose, is the subject of extensive study as it may be associated with a higher risk of developing a second cancer and could have deleterious effects on the immune system that compromise the efficiency of combined radio-immunotherapy treatments. Out-of-field dose estimation tools developed today in research, including Monte Carlo simulations and analytical methods, are not suited to the requirements of clinical implementation because of their lack of versatility and their cumbersome application. We propose a proof of concept based on deep learning for out-of-field dose map estimation that addresses these limitations.

METHODS AND MATERIALS

For this purpose, a 3D U-Net, considering as inputs the in-field dose, as computed by the treatment planning system, and the patient's anatomy, was trained to predict out-of-field dose maps. The cohort used for learning and performance evaluation included 3151 pediatric patients from the FCCSS database, treated in 5 clinical centers, whose whole-body dose maps were previously estimated with an empirical analytical method. The test set, composed of 433 patients, was split into 5 subdata sets, each containing patients treated with devices unseen during the training phase. Root mean square deviation evaluated only on nonzero voxels located in the out-of-field areas was computed as performance metric.

RESULTS

Root mean square deviations of 0.28 and 0.41 cGy/Gy were obtained for the training and validation data sets, respectively. Values of 0.27, 0.26, 0.28, 0.30, and 0.45 cGy/Gy were achieved for the 6 MV linear accelerator, 16 MV linear accelerator, Alcyon cobalt irradiator, Mobiletron cobalt irradiator, and betatron device test sets, respectively.

CONCLUSIONS

This proof-of-concept approach using a convolutional neural network has demonstrated unprecedented generalizability for this task, although it remains limited, and brings us closer to an implementation compatible with clinical routine.

Rethinking automatic segmentation of gross target volume from a decoupling perspective

Accurate and reliable segmentation of Gross Target Volume (GTV) is critical in cancer Radiation Therapy (RT) planning, but manual delineation is time-consuming and subject to inter-observer variations. Recently, deep learning methods have achieved remarkable success in medical image segmentation. However, due to the low image contrast and extreme pixel imbalance between GTV and adjacent tissues, most existing methods usually obtained limited performance on automatic GTV segmentation. In this paper, we propose a Heterogeneous Cascade Framework (HCF) from a decoupling perspective, which decomposes the GTV segmentation into independent recognition and segmentation subtasks. The former aims to screen out the abnormal slices containing GTV, while the latter performs pixel-wise segmentation of these slices. With the decoupled two-stage framework, we can efficiently filter normal slices to reduce false positives. To further improve the segmentation performance, we design a multi-level Spatial Alignment Network (SANet) based on the feature pyramid structure, which introduces a spatial alignment module into the decoder to compensate for the information loss caused by downsampling. Moreover, we propose a Combined Regularization (CR) loss and Balance-Sampling Strategy (BSS) to alleviate the pixel imbalance problem and improve network convergence. Extensive experiments on two public datasets of StructSeg2019 challenge demonstrate that our method outperforms state-of-the-art methods, especially with significant advantages in reducing false positives and accurately segmenting small objects. The code is available at https://github.com/shijun18/GTV_AutoSeg.

Clinical acceptability of automatically generated lymph node levels and structures of deglutition and mastication for head and neck radiation therapy

Background and Purpose

Auto-contouring of complex anatomy in computed tomography (CT) scans is a highly anticipated solution to many problems in radiotherapy. In this study, artificial intelligence (AI)-based auto-contouring models were clinically validated for lymph node levels and structures of swallowing and chewing in the head and neck.

Materials and Methods

CT scans of 145 head and neck radiotherapy patients were retrospectively curated. One cohort (n = 47) was used to analyze seven lymph node levels and the other (n = 98) used to analyze 17 swallowing and chewing structures. Separate nnUnet models were trained and validated using the separate cohorts. For the lymph node levels, preference and clinical acceptability of AI vs human contours were scored. For the swallowing and chewing structures, clinical acceptability was scored. Quantitative analyses of the test sets were performed for AI vs human contours for all structures using overlap and distance metrics.

Results

Median Dice Similarity Coefficient ranged from 0.77 to 0.89 for lymph node levels and 0.86 to 0.96 for chewing and swallowing structures. The AI contours were superior to or equally preferred to the manual contours at rates ranging from 75% to 91%; there was not a significant difference in clinical acceptability for nodal levels I-V for manual versus AI contours. Across all AI-generated lymph node level contours, 92% were rated as usable with stylistic to no edits. Of the 340 contours in the chewing and swallowing cohort, 4% required minor edits.

Conclusions

An accurate approach was developed to auto-contour lymph node levels and chewing and swallowing structures on CT images for patients with intact nodal anatomy. Only a small portion of test set auto-contours required minor edits.

An open-source nnU-net algorithm for automatic segmentation of MRI scans in the male pelvis for adaptive radiotherapy

Background

Adaptive MRI-guided radiotherapy (MRIgRT) requires accurate and efficient segmentation of organs and targets on MRI scans. Manual segmentation is time-consuming and variable, while deformable image registration (DIR)-based contour propagation may not account for large anatomical changes. Therefore, we developed and evaluated an automatic segmentation method using the nnU-net framework.

Methods

The network was trained on 38 patients (76 scans) with localized prostate cancer and tested on 30 patients (60 scans) with localized prostate, metastatic prostate, or bladder cancer treated at a 1.5 T MRI-linac at our institution. The performance of the network was compared with the current clinical workflow based on DIR. The segmentation accuracy was evaluated using the Dice similarity coefficient (DSC), mean surface distance (MSD), and Hausdorff distance (HD) metrics.

Results

The trained network successfully segmented all 600 structures in the test set. High similarity was obtained for most structures, with 90% of the contours having a DSC above 0.9 and 86% having an MSD below 1 mm. The largest discrepancies were found in the sigmoid and colon structures. Stratified analysis on cancer type showed that the best performance was seen in the same type of patients that the model was trained on (localized prostate). Especially in patients with bladder cancer, the performance was lower for the bladder and the surrounding organs. A complete automatic delineation workflow took approximately 1 minute. Compared with contour transfer based on the clinically used DIR algorithm, the nnU-net performed statistically better across all organs, with the most significant gain in using the nnU-net seen for organs subject to more considerable volumetric changes due to variation in the filling of the rectum, bladder, bowel, and sigmoid.

Conclusion

We successfully trained and tested a network for automatically segmenting organs and targets for MRIgRT in the male pelvis region. Good test results were seen for the trained nnU-net, with test results outperforming the current clinical practice using DIR-based contour propagation at the 1.5 T MRI-linac. The trained network is sufficiently fast and accurate for clinical use in an online setting for MRIgRT. The model is provided as open-source.

Implementation and evaluation of an iterative-based algorithm for automatic beam angle optimization in breast cancer treatment planning

INTRODUCTION

A beam angle optimization (BAO) algorithm was developed to evaluate its clinical feasibility and investigate the impact of varying BAO constraints on breast cancer treatment plans.

MATERIALS AND METHODS

A two-part study was designed. In part 1, we retrospectively selected 20 patients treated with radiotherapy after breast-conserving surgery. For each patient, BAO plans were designed using beam angles optimized by the BAO algorithm and the same optimization constraints as manual plans. Dosimetric indices were compared between BAO and manual plans. In part 2, fifteen patients with left breast cancer were included. For each patient, three distinct cardiac constraints (mean heart dose < 5 Gy, 3 Gy or 1 Gy) were established during the BAO process to obtain three optimized beam sets which were marked as BAO_H1, BAO_H3, BAO_H5, respectively. These sets of beams were then utilized under identical IMRT constraints for planning. Comparative analysis was conducted among the three groups of plans.

RESULTS

For part 1, no significant differences were observed between BAO plans and manual plans in all dosimetric indices, except for ipsilateral lung V5, where BAO plans performed slightly better than manual plans (35.5% ± 5.6% vs 36.9% ± 4.3%, p = 0.034). For part 2, Stricter BAO heart constraints resulted in more perpendicular beams. However, there was no significant difference between BAO_H1, BAO_H3 and BAO_H5 with the same IMRT constraint in the heart dose. Meanwhile, the left lung dose was increased while the right breast and lung doses were decreased with stricter heart constraints in BAO. When mean heart dose < 5 Gy in IMRT constraint, the mean dose to the right lung was decreased from 0.46 Gy for BAO_H5 to 0.33 Gy for BAO_H1 (p = 0.027).

CONCLUSIONS

The BAO algorithm can achieve quality plans comparable to manual plans. IMRT constraints dominate the final plan dose, while varying BAO constraints alter the trade-off among structures, providing an additional degree of freedom in planning design.

Adaptive Region-Specific Loss for Improved Medical Image Segmentation

Defining the loss function is an important part of neural network design and critically determines the success of deep learning modeling. A significant shortcoming of the conventional loss functions is that they weight all regions in the input image volume equally, despite the fact that the system is known to be heterogeneous (i.e., some regions can achieve high prediction performance more easily than others). Here, we introduce a region-specific loss to lift the implicit assumption of homogeneous weighting for better learning. We divide the entire volume into multiple sub-regions, each with an individualized loss constructed for optimal local performance. Effectively, this scheme imposes higher weightings on the sub-regions that are more difficult to segment, and vice versa. Furthermore, the regional false positive and false negative errors are computed for each input image during a training step and the regional penalty is adjusted accordingly to enhance the overall accuracy of the prediction. Using different public and in-house medical image datasets, we demonstrate that the proposed regionally adaptive loss paradigm outperforms conventional methods in the multi-organ segmentations, without any modification to the neural network architecture or additional data preparation.

Federated Multi-Organ Segmentation With Inconsistent Labels

Federated learning is an emerging paradigm allowing large-scale decentralized learning without sharing data across different data owners, which helps address the concern of data privacy in medical image analysis. However, the requirement for label consistency across clients by the existing methods largely narrows its application scope. In practice, each clinical site may only annotate certain organs of interest with partial or no overlap with other sites. Incorporating such partially labeled data into a unified federation is an unexplored problem with clinical significance and urgency. This work tackles the challenge by using a novel federated multi-encoding U-Net (Fed-MENU) method for multi-organ segmentation. In our method, a multi-encoding U-Net (MENU-Net) is proposed to extract organ-specific features through different encoding sub-networks. Each sub-network can be seen as an expert of a specific organ and trained for that client. Moreover, to encourage the organ-specific features extracted by different sub-networks to be informative and distinctive, we regularize the training of the MENU-Net by designing an auxiliary generic decoder (AGD). Extensive experiments on six public abdominal CT datasets show that our Fed-MENU method can effectively obtain a federated learning model using the partially labeled datasets with superior performance to other models trained by either localized or centralized learning methods. Source code is publicly available at https://github.com/DIAL-RPI/Fed-MENU.

A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy

Purpose/objectives

Auto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.

Methods and materials

The organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.

Results

There are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.

Conclusions

All five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.

SDPN: A Slight Dual-Path Network With Local-Global Attention Guided for Medical Image Segmentation

Accurate identification of lesions is a key step in surgical planning. However, this task mainly exists two challenges: 1) Due to the complex anatomical shapes of different lesions, most segmentation methods only achieve outstanding performance for a specific structure, rather than other lesions with location differences. 2) The huge number of parameters limits existing transformer-based segmentation models. To overcome these problems, we propose a novel slight dual-path network (SDPN) to segment variable location lesions or organs with significant differences accurately. First, we design a dual-path module to integrate local with global features without obvious memory consumption. Second, a novel Multi-spectrum attention module is proposed to pay further attention to detailed information, which can automatically adapt to the variable segmentation target. Then, the compression module based on tensor ring decomposition is designed to compress convolutional and transformer structures. In the experiment, four datasets, including three benchmark datasets and a clinical dataset, are used to evaluate SDPN. Results of the experiments show that SDPN performs better than other start-of-the-art methods for brain tumor, liver tumor, endometrial tumor and cardiac segmentation. To ensure the generalizability, we train the network on Kvasir-SEG and test on CVC-ClinicDB which collected from a different institution. The quantitative analysis shows that the clinical evaluation results are consistent with the experts. Therefore, this model may be a potential candidate for the segmentation of lesions and organs segmentation with variable locations in clinical applications.

Analytical models for external photon beam radiotherapy out-of-field dose calculation: a scoping review

A growing body of scientific evidence indicates that exposure to low dose ionizing radiation (< 2 Gy) is associated with a higher risk of developing radio-induced cancer. Additionally, it has been shown to have significant impacts on both innate and adaptive immune responses. As a result, the evaluation of the low doses inevitably delivered outside the treatment fields (out-of-field dose) in photon radiotherapy is a topic that is regaining interest at a pivotal moment in radiotherapy. In this work, we proposed a scoping review in order to identify evidence of strengths and limitations of available analytical models for out-of-field dose calculation in external photon beam radiotherapy for the purpose of implementation in clinical routine. Papers published between 1988 and 2022 proposing a novel analytical model that estimated at least one component of the out-of-field dose for photon external radiotherapy were included. Models focusing on electrons, protons and Monte-Carlo methods were excluded. The methodological quality and potential limitations of each model were analyzed to assess their generalizability. Twenty-one published papers were selected for analysis, of which 14 proposed multi-compartment models, demonstrating that research efforts are directed towards an increasingly detailed description of the underlying physical phenomena. Our synthesis revealed great inhomogeneities in practices, in particular in the acquisition of experimental data and the standardization of measurements, in the choice of metrics used for the evaluation of model performance and even in the definition of regions considered out-of-the-field, which makes quantitative comparisons impossible. We therefore propose to clarify some key concepts. The analytical methods do not seem to be easily suitable for massive use in clinical routine, due to the inevitable cumbersome nature of their implementation. Currently, there is no consensus on a mathematical formalism that comprehensively describes the out-of-field dose in external photon radiotherapy, partly due to the complex interactions between a large number of influencing factors. Out-of-field dose calculation models based on neural networks could be promising tools to overcome these limitations and thus favor a transfer to the clinic, but the lack of sufficiently large and heterogeneous data sets is the main obstacle.

uRP: An integrated research platform for one-stop analysis of medical images

Introduction

Medical image analysis is of tremendous importance in serving clinical diagnosis, treatment planning, as well as prognosis assessment. However, the image analysis process usually involves multiple modality-specific software and relies on rigorous manual operations, which is time-consuming and potentially low reproducible.

Methods

We present an integrated platform - uAI Research Portal (uRP), to achieve one-stop analyses of multimodal images such as CT, MRI, and PET for clinical research applications. The proposed uRP adopts a modularized architecture to be multifunctional, extensible, and customizable.

Results and Discussion

The uRP shows 3 advantages, as it 1) spans a wealth of algorithms for image processing including semi-automatic delineation, automatic segmentation, registration, classification, quantitative analysis, and image visualization, to realize a one-stop analytic pipeline, 2) integrates a variety of functional modules, which can be directly applied, combined, or customized for specific application domains, such as brain, pneumonia, and knee joint analyses, 3) enables full-stack analysis of one disease, including diagnosis, treatment planning, and prognosis assessment, as well as full-spectrum coverage for multiple disease applications. With the continuous development and inclusion of advanced algorithms, we expect this platform to largely simplify the clinical scientific research process and promote more and better discoveries.

Extended reality for biomedicine

Extended reality (XR) refers to an umbrella of methods that allows users to be immersed in a three-dimensional (3D) or a 4D (spatial + temporal) virtual environment to different extents, including virtual reality (VR), augmented reality (AR), and mixed reality (MR). While VR allows a user to be fully immersed in a virtual environment, AR and MR overlay virtual objects over the real physical world. The immersion and interaction of XR provide unparalleled opportunities to extend our world beyond conventional lifestyles. While XR has extensive applications in fields such as entertainment and education, its numerous applications in biomedicine create transformative opportunities in both fundamental research and healthcare. This Primer outlines XR technology from instrumentation to software computation methods, delineating the biomedical applications that have been advanced by state-of-the-art techniques. We further describe the technical advances overcoming current limitations in XR and its applications, providing an entry point for professionals and trainees to thrive in this emerging field.