Addressing the Contrast Media Recognition Challenge: A Fully Automated Machine Learning Approach for Predicting Contrast Phases in CT Imaging

OBJECTIVES

Accurately acquiring and assigning different contrast-enhanced phases in computed tomography (CT) is relevant for clinicians and for artificial intelligence orchestration to select the most appropriate series for analysis. However, this information is commonly extracted from the CT metadata, which is often wrong. This study aimed at developing an automatic pipeline for classifying intravenous (IV) contrast phases and additionally for identifying contrast media in the gastrointestinal tract (GIT).

MATERIALS AND METHODS

This retrospective study used 1200 CT scans collected at the investigating institution between January 4, 2016 and September 12, 2022, and 240 CT scans from multiple centers from The Cancer Imaging Archive for external validation. The open-source segmentation algorithm TotalSegmentator was used to identify regions of interest (pulmonary artery, aorta, stomach, portal/splenic vein, liver, portal vein/hepatic veins, inferior vena cava, duodenum, small bowel, colon, left/right kidney, urinary bladder), and machine learning classifiers were trained with 5-fold cross-validation to classify IV contrast phases (noncontrast, pulmonary arterial, arterial, venous, and urographic) and GIT contrast enhancement. The performance of the ensembles was evaluated using the receiver operating characteristic area under the curve (AUC) and 95% confidence intervals (CIs).

RESULTS

For the IV phase classification task, the following AUC scores were obtained for the internal test set: 99.59% [95% CI, 99.58-99.63] for the noncontrast phase, 99.50% [95% CI, 99.49-99.52] for the pulmonary-arterial phase, 99.13% [95% CI, 99.10-99.15] for the arterial phase, 99.8% [95% CI, 99.79-99.81] for the venous phase, and 99.7% [95% CI, 99.68-99.7] for the urographic phase. For the external dataset, a mean AUC of 97.33% [95% CI, 97.27-97.35] and 97.38% [95% CI, 97.34-97.41] was achieved for all contrast phases for the first and second annotators, respectively. Contrast media in the GIT could be identified with an AUC of 99.90% [95% CI, 99.89-99.9] in the internal dataset, whereas in the external dataset, an AUC of 99.73% [95% CI, 99.71-99.73] and 99.31% [95% CI, 99.27-99.33] was achieved with the first and second annotator, respectively.

CONCLUSIONS

The integration of open-source segmentation networks and classifiers effectively classified contrast phases and identified GIT contrast enhancement using anatomical landmarks.

From admission to discharge: a systematic review of clinical natural language processing along the patient journey

BACKGROUND

Medical text, as part of an electronic health record, is an essential information source in healthcare. Although natural language processing (NLP) techniques for medical text are developing fast, successful transfer into clinical practice has been rare. Especially the hospital domain offers great potential while facing several challenges including many documents per patient, multiple departments and complex interrelated processes.

METHODS

In this work, we survey relevant literature to identify and classify approaches which exploit NLP in the clinical context. Our contribution involves a systematic mapping of related research onto a prototypical patient journey in the hospital, along which medical documents are created, processed and consumed by hospital staff and patients themselves. Specifically, we reviewed which dataset types, dataset languages, model architectures and tasks are researched in current clinical NLP research. Additionally, we extract and analyze major obstacles during development and implementation. We discuss options to address them and argue for a focus on bias mitigation and model explainability.

RESULTS

While a patient's hospital journey produces a significant amount of structured and unstructured documents, certain steps and documents receive more research attention than others. Diagnosis, Admission and Discharge are clinical patient steps that are researched often across the surveyed paper. In contrast, our findings reveal significant under-researched areas such as Treatment, Billing, After Care, and Smart Home. Leveraging NLP in these stages can greatly enhance clinical decision-making and patient outcomes. Additionally, clinical NLP models are mostly based on radiology reports, discharge letters and admission notes, even though we have shown that many other documents are produced throughout the patient journey. There is a significant opportunity in analyzing a wider range of medical documents produced throughout the patient journey to improve the applicability and impact of NLP in healthcare.

CONCLUSIONS

Our findings suggest that there is a significant opportunity to leverage NLP approaches to advance clinical decision-making systems, as there remains a considerable understudied potential for the analysis of patient journey data.

Automated 3D-Body Composition Analysis as a Predictor of Survival in Patients With Idiopathic Pulmonary Fibrosis

PURPOSE

Idiopathic pulmonary fibrosis (IPF) is the most common interstitial lung disease, with a median survival time of 2 to 5 years. The focus of this study is to establish a novel imaging biomarker.

MATERIALS AND METHODS

In this study, 79 patients (19% female) with a median age of 70 years were studied retrospectively. Fully automated body composition analysis (BCA) features (bone, muscle, total adipose tissue, intermuscular, and intramuscular adipose tissue) were combined into Sarcopenia, Fat, and Myosteatosis indices and compared between patients with a survival of more or less than 2 years. In addition, we divided the cohort at the median (high=≥ median, low=

RESULTS

A high Sarcopenia and Fat index and low Myosteatosis index were associated with longer median survival (35 vs. 16 mo for high vs. low Sarcopenia index, P=0.066; 44 vs. 14 mo for high vs. low Fat index, P<0.001; and 33 vs. 14 mo for low vs. high Myosteatosis index, P=0.0056) and better 5-year survival rates (34.0% vs. 23.6% for high vs. low Sarcopenia index; 47.3% vs. 9.2% for high vs. low Fat index; and 11.2% vs. 42.7% for high vs. low Myosteatosis index). Adjusted multivariate Cox regression showed a significant impact of the Fat (HR=0.71, P=0.01) and Myosteatosis (HR=1.12, P=0.005) on overall survival.

CONCLUSION

The fully automated BCA provides biomarkers with a predictive value for the overall survival in patients with IPF.

Collapse

Key Words Collapse

MESH Headings Collapse

Grants Collapse

Establishing Medical Intelligence - Leveraging FHIR to Improve Clinical Management: a retrospective cohort and clinical implementation study

BACKGROUND

FHIR (Fast Healthcare Interoperability Resources) has been proposed to enable health data interoperability. So far, its applicability has been demonstrated for selected research projects with limited data.

OBJECTIVE

Here, we designed and implemented a conceptual medical intelligence framework to leverage real-world care data for clinical decision-making.

METHODS

A Python package for the utilization of multimodal FHIR data (FHIRPACK) was developed and pioneered in five real-world clinical use cases, i.e., myocardial infarction (MI), stroke, diabetes, sepsis, and prostate cancer (PC). Patients were identified based on ICD-10 codes, and outcomes were derived from laboratory tests, prescriptions, procedures, and diagnostic reports. Results were provided as browser-based dashboards.

RESULTS

For 2022, 1,302,988 patient encounters were analyzed. MI: In 72.7% of cases (N=261) medication regimens fulfilled guideline recommendations. Stroke: Out of 1,277 patients, 165 patients received thrombolysis and 108 thrombectomy. Diabetes: In 443,866 serum glucose and 16,180 HbA1c measurements from 35,494 unique patients, the prevalence of dysglycemic findings was 39% (N=13,887). Among those with dysglycemia, diagnosis was coded in 44.2% (N=6,138) of the patients. Sepsis: In 1,803 patients, Staphylococcus epidermidis was the primarily isolated pathogen (N=773, 28.9%) and piperacillin/tazobactam was the primarily prescribed antibiotic (N=593, 37.2%). PC: Three out of 54 patients who received radical prostatectomy were identified as cases with PSA persistence or biochemical recurrence.

CONCLUSIONS

Leveraging FHIR data through large-scale analytics can enhance healthcare quality and improve patient outcomes across five clinical specialties. We identified i) sepsis patients requiring less broad antibiotic therapy, ii) patients with myocardial infarction who could benefit from statin and antiplatelet therapy, iii) stroke patients with longer than recommended times to intervention, iv) patients with hyperglycemia who could benefit from specialist referral and v) PC patients with early increases in cancer markers.

CLINICALTRIAL

AI-derived body composition parameters as prognostic factors in patients with HCC undergoing TACE in a multicenter study

Background & Aims

Body composition assessment (BCA) parameters have recently been identified as relevant prognostic factors for patients with hepatocellular carcinoma (HCC). Herein, we aimed to investigate the role of BCA parameters for prognosis prediction in patients with HCC undergoing transarterial chemoembolization (TACE).

Methods

This retrospective multicenter study included a total of 754 treatment-naïve patients with HCC who underwent TACE at six tertiary care centers between 2010-2020. Fully automated artificial intelligence-based quantitative 3D volumetry of abdominal cavity tissue composition was performed to assess skeletal muscle volume (SM), total adipose tissue (TAT), intra- and intermuscular adipose tissue, visceral adipose tissue, and subcutaneous adipose tissue (SAT) on pre-intervention computed tomography scans. BCA parameters were normalized to the slice number of the abdominal cavity. We assessed the influence of BCA parameters on median overall survival and performed multivariate analysis including established estimates of survival.

Results

Univariate survival analysis revealed that impaired median overall survival was predicted by low SM (p <0.001), high TAT volume (p = 0.013), and high SAT volume (p = 0.006). In multivariate survival analysis, SM remained an independent prognostic factor (p = 0.039), while TAT and SAT volumes no longer showed predictive ability. This predictive role of SM was confirmed in a subgroup analysis of patients with BCLC stage B.

Conclusions

SM is an independent prognostic factor for survival prediction. Thus, the integration of SM into novel scoring systems could potentially improve survival prediction and clinical decision-making. Fully automated approaches are needed to foster the implementation of this imaging biomarker into daily routine.

Impact and implications

Body composition assessment parameters, especially skeletal muscle volume, have been identified as relevant prognostic factors for many diseases and treatments. In this study, skeletal muscle volume has been identified as an independent prognostic factor for patients with hepatocellular carcinoma undergoing transarterial chemoembolization. Therefore, skeletal muscle volume as a metaparameter could play a role as an opportunistic biomarker in holistic patient assessment and be integrated into decision support systems. Workflow integration with artificial intelligence is essential for automated, quantitative body composition assessment, enabling broad availability in multidisciplinary case discussions.

A proof of concept for microcirculation monitoring using machine learning based hyperspectral imaging in critically ill patients: a monocentric observational study

BACKGROUND

Impaired microcirculation is a cornerstone of sepsis development and leads to reduced tissue oxygenation, influenced by fluid and catecholamine administration during treatment. Hyperspectral imaging (HSI) is a non-invasive bedside technology for visualizing physicochemical tissue characteristics. Machine learning (ML) for skin HSI might offer an automated approach for bedside microcirculation assessment, providing an individualized tissue fingerprint of critically ill patients in intensive care. The study aimed to determine if machine learning could be utilized to automatically identify regions of interest (ROIs) in the hand, thereby distinguishing between healthy individuals and critically ill patients with sepsis using HSI.

METHODS

HSI raw data from 75 critically ill sepsis patients and from 30 healthy controls were recorded using TIVITA® Tissue System and analyzed using an automated ML approach. Additionally, patients were divided into two groups based on their SOFA scores for further subanalysis: less severely ill (SOFA ≤ 5) and severely ill (SOFA > 5). The analysis of the HSI raw data was fully-automated using MediaPipe for ROI detection (palm and fingertips) and feature extraction. HSI Features were statistically analyzed to highlight relevant wavelength combinations using Mann-Whitney-U test and Benjamini, Krieger, and Yekutieli (BKY) correction. In addition, Random Forest models were trained using bootstrapping, and feature importances were determined to gain insights regarding the wavelength importance for a model decision.

RESULTS

An automated pipeline for generating ROIs and HSI feature extraction was successfully established. HSI raw data analysis accurately distinguished healthy controls from sepsis patients. Wavelengths at the fingertips differed in the ranges of 575-695 nm and 840-1000 nm. For the palm, significant differences were observed in the range of 925-1000 nm. Feature importance plots indicated relevant information in the same wavelength ranges. Combining palm and fingertip analysis provided the highest reliability, with an AUC of 0.92 to distinguish between sepsis patients and healthy controls.

CONCLUSION

Based on this proof of concept, the integration of automated and standardized ROIs along with automated skin HSI analyzes, was able to differentiate between healthy individuals and patients with sepsis. This approach offers a reliable and objective assessment of skin microcirculation, facilitating the rapid identification of critically ill patients.

ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Dataset

Automated medical image analysis systems often require large amounts of training data with high quality labels, which are difficult and time consuming to generate. This paper introduces Radiology Object in COntext version 2 (ROCOv2), a multimodal dataset consisting of radiological images and associated medical concepts and captions extracted from the PMC Open Access subset. It is an updated version of the ROCO dataset published in 2018, and adds 35,705 new images added to PMC since 2018. It further provides manually curated concepts for imaging modalities with additional anatomical and directional concepts for X-rays. The dataset consists of 79,789 images and has been used, with minor modifications, in the concept detection and caption prediction tasks of ImageCLEFmedical Caption 2023. The dataset is suitable for training image annotation models based on image-caption pairs, or for multi-label image classification using Unified Medical Language System (UMLS) concepts provided with each image. In addition, it can serve for pre-training of medical domain models, and evaluation of deep learning models for multi-task learning.

Early Metabolic Response by PET Predicts Sensitivity to Next-Line Targeted Therapy in EGFR-Mutated Lung Cancer with Unknown Mechanism of Acquired Resistance

Targeted therapy with epidermal growth factor receptor (EGFR) tyrosine kinase inhibitors (TKIs) has established the precision oncology paradigm in lung cancer. Most patients with EGFR-mutated lung cancer respond but eventually acquire resistance. Methods: Patients exhibiting the EGFR p.T790M resistance biomarker benefit from sequenced targeted therapy with osimertinib. We hypothesized that metabolic response as detected by 18F-FDG PET after short-course osimertinib identifies additional patients susceptible to sequenced therapy. Results: Fourteen patients with EGFR-mutated lung cancer and resistance to first- or second-generation EGFR TKI testing negatively for EGFR p.T790M were enrolled in a phase II study. Five patients (36%) achieved a metabolic 18F-FDG PET response and continued osimertinib. In those, the median duration of treatment was not reached (95% CI, 24 mo to not estimable), median progression-free survival was 18.7 mo (95% CI, 14.6 mo to not estimable), and median overall survival was 41.5 mo. Conclusion: Connecting theranostic osimertinib treatment with early metabolic response assessment by PET enables early identification of patients with unknown mechanisms of TKI resistance who derive dramatic clinical benefit from sequenced osimertinib. This defines a novel paradigm for personalization of targeted therapies in patients with lung cancer dependent on a tractable driver oncogene.

BOA: A CT-Based Body and Organ Analysis for Radiologists at the Point of Care

PURPOSE

The study aimed to develop the open-source body and organ analysis (BOA), a comprehensive computed tomography (CT) image segmentation algorithm with a focus on workflow integration.

METHODS

The BOA combines 2 segmentation algorithms: body composition analysis (BCA) and TotalSegmentator. The BCA was trained with the nnU-Net framework using a dataset including 300 CT examinations. The CTs were manually annotated with 11 semantic body regions: subcutaneous tissue, muscle, bone, abdominal cavity, thoracic cavity, glands, mediastinum, pericardium, breast implant, brain, and spinal cord. The models were trained using 5-fold cross-validation, and at inference time, an ensemble was used. Afterward, the segmentation efficiency was evaluated on a separate test set comprising 60 CT scans. In a postprocessing step, a tissue segmentation (muscle, subcutaneous adipose tissue, visceral adipose tissue, intermuscular adipose tissue, epicardial adipose tissue, and paracardial adipose tissue) is created by subclassifying the body regions. The BOA combines this algorithm and the open-source segmentation software TotalSegmentator to have an all-in-one comprehensive selection of segmentations. In addition, it integrates into clinical workflows as a DICOM node-triggered service using the open-source Orthanc research PACS (Picture Archiving and Communication System) server to make the automated segmentation algorithms available to clinicians. The BCA model's performance was evaluated using the Sørensen-Dice score. Finally, the segmentations from the 3 different tools (BCA, TotalSegmentator, and BOA) were compared by assessing the overall percentage of the segmented human body on a separate cohort of 150 whole-body CT scans.

RESULTS

The results showed that the BCA outperformed the previous publication, achieving a higher Sørensen-Dice score for the previously existing classes, including subcutaneous tissue (0.971 vs 0.962), muscle (0.959 vs 0.933), abdominal cavity (0.983 vs 0.973), thoracic cavity (0.982 vs 0.965), bone (0.961 vs 0.942), and an overall good segmentation efficiency for newly introduced classes: brain (0.985), breast implant (0.943), glands (0.766), mediastinum (0.880), pericardium (0.964), and spinal cord (0.896). All in all, it achieved a 0.935 average Sørensen-Dice score, which is comparable to the one of the TotalSegmentator (0.94). The TotalSegmentator had a mean voxel body coverage of 31% ± 6%, whereas BCA had a coverage of 75% ± 6% and BOA achieved 93% ± 2%.

CONCLUSIONS

The open-source BOA merges different segmentation algorithms with a focus on workflow integration through DICOM node integration, offering a comprehensive body segmentation in CT images with a high coverage of the body volume.

SAROS: A dataset for whole-body region and organ segmentation in CT imaging

The Sparsely Annotated Region and Organ Segmentation (SAROS) dataset was created using data from The Cancer Imaging Archive (TCIA) to provide a large open-access CT dataset with high-quality annotations of body landmarks. In-house segmentation models were employed to generate annotation proposals on randomly selected cases from TCIA. The dataset includes 13 semantic body region labels (abdominal/thoracic cavity, bones, brain, breast implant, mediastinum, muscle, parotid/submandibular/thyroid glands, pericardium, spinal cord, subcutaneous tissue) and six body part labels (left/right arm/leg, head, torso). Case selection was based on the DICOM series description, gender, and imaging protocol, resulting in 882 patients (438 female) for a total of 900 CTs. Manual review and correction of proposals were conducted in a continuous quality control cycle. Only every fifth axial slice was annotated, yielding 20150 annotated slices from 28 data collections. For the reproducibility on downstream tasks, five cross-validation folds and a test set were pre-defined. The SAROS dataset serves as an open-access resource for training and evaluating novel segmentation models, covering various scanner vendors and diseases.

Elexacaftor/tezacaftor/ivacaftor influences body composition in adults with cystic fibrosis: a fully automated CT-based analysis

A poor nutritional status is associated with worse pulmonary function and survival in people with cystic fibrosis (pwCF). CF transmembrane conductance regulator modulators can improve pulmonary function and body weight, but more data is needed to evaluate its effects on body composition. In this retrospective study, a pre-trained deep-learning network was used to perform a fully automated body composition analysis on chest CTs from 66 adult pwCF before and after receiving elexacaftor/tezacaftor/ivacaftor (ETI) therapy. Muscle and adipose tissues were quantified and divided by bone volume to obtain body size-adjusted ratios. After receiving ETI therapy, marked increases were observed in all adipose tissue ratios among pwCF, including the total adipose tissue ratio (+ 46.21%, p < 0.001). In contrast, only small, but statistically significant increases of the muscle ratio were measured in the overall study population (+ 1.63%, p = 0.008). Study participants who were initially categorized as underweight experienced more pronounced effects on total adipose tissue ratio (p = 0.002), while gains in muscle ratio were equally distributed across BMI categories (p = 0.832). Our findings suggest that ETI therapy primarily affects adipose tissues, not muscle tissue, in adults with CF. These effects are primarily observed among pwCF who were initially underweight. Our findings may have implications for the future nutritional management of pwCF.

Body composition impacts outcome of bronchoscopic lung volume reduction in patients with severe emphysema: a fully automated CT-based analysis

Chronic Obstructive Pulmonary Disease (COPD) is characterized by progressive and irreversible airflow limitation, with individual body composition influencing disease severity. Severe emphysema worsens symptoms through hyperinflation, which can be relieved by bronchoscopic lung volume reduction (BLVR). To investigate how body composition, assessed through CT scans, impacts outcomes in emphysema patients undergoing BLVR. Fully automated CT-based body composition analysis (BCA) was performed in patients with end-stage emphysema receiving BLVR with valves. Post-interventional muscle and adipose tissues were quantified, body size-adjusted, and compared to baseline parameters. Between January 2015 and December 2022, 300 patients with severe emphysema underwent endobronchial valve treatment. Significant improvements were seen in outcome parameters, which were defined as changes in pulmonary function, physical performance, and quality of life (QoL) post-treatment. Muscle volume remained stable (1.632 vs. 1.635 for muscle bone adjusted ratio (BAR) at baseline and after 6 months respectively), while bone adjusted adipose tissue volumes, especially total and pericardial adipose tissue, showed significant increase (2.86 vs. 3.00 and 0.16 vs. 0.17, respectively). Moderate to strong correlations between bone adjusted muscle volume and weaker correlations between adipose tissue volumes and outcome parameters (pulmonary function, QoL and physical performance) were observed. Particularly after 6-month, bone adjusted muscle volume changes positively corresponded to improved outcomes (ΔForced expiratory volume in 1 s [FEV1], r = 0.440; ΔInspiratory vital capacity [IVC], r = 0.397; Δ6Minute walking distance [6MWD], r = 0.509 and ΔCOPD assessment test [CAT], r = -0.324; all p < 0.001). Group stratification by bone adjusted muscle volume changes revealed that groups with substantial muscle gain experienced a greater clinical benefit in pulmonary function improvements, QoL and physical performance (ΔFEV1%, 5.5 vs. 39.5; ΔIVC%, 4.3 vs. 28.4; Δ6MWDm, 14 vs. 110; ΔCATpts, -2 vs. -3.5 for groups with ΔMuscle, BAR% < -10 vs. > 10, respectively). BCA results among patients divided by the minimal clinically important difference for forced expiratory volume of the first second (FEV1) showed significant differences in bone-adjusted muscle and intramuscular adipose tissue (IMAT) volumes and their respective changes after 6 months (ΔMuscle, BAR% -5 vs. 3.4 and ΔIMAT, BAR% -0.62 vs. 0.60 for groups with ΔFEV1 ≤ 100 mL vs > 100 mL). Altered body composition, especially increased muscle volume, is associated with functional improvements in BLVR-treated patients.

Radiomics Analysis of Multiparametric PET/MRI for N- and M-Staging in Patients with Primary Cervical Cancer

PURPOSE

 The aim of this study was to investigate the potential of multiparametric 18F-FDG PET/MR imaging as a platform for radiomics analysis and machine learning algorithms based on primary cervical cancers to predict N- and M-stage in patients.

MATERIALS AND METHODS

 A total of 30 patients with histopathological confirmation of primary and untreated cervical cancer were prospectively enrolled for a multiparametric 18F-FDG PET/MR examination, comprising a dedicated protocol for imaging of the female pelvis. The primary tumor in the uterine cervix was manually segmented on post-contrast T1-weighted images. Quantitative features were extracted from the segmented tumors using the Radiomic Image Processing Toolbox for the R software environment for statistical computing and graphics. 45 different image features were calculated from non-enhanced as well as post-contrast T1-weighted TSE images, T2-weighted TSE images, the ADC map, the parametric Ktrans, Kep, Ve and iAUC maps and PET images, respectively. Statistical analysis and modeling was performed using Python 3.5 and the scikit-learn software machine learning library for the Python programming language.

RESULTS

 Prediction of M-stage was superior when compared to N-stage. Prediction of M-stage using SVM with SVM-RFE as feature selection obtained the highest performance providing sensitivity of 91 % and specificity of 92 %. Using receiver operating characteristic (ROC) analysis of the pooled predictions, the area under the curve (AUC) was 0.97. Prediction of N-stage using RBF-SVM with MIFS as feature selection reached sensitivity of 83 %, specificity of 67 % and an AUC of 0.82.

CONCLUSION

 M- and N-stage can be predicted based on isolated radiomics analyses of the primary tumor in cervical cancers, thus serving as a template for noninvasive tumor phenotyping and patient stratification using high-dimensional feature vectors extracted from multiparametric PET/MRI data.

KEY POINTS

  · Radiomics analysis based on multiparametric PET/MRI enables prediction of the metastatic status of cervical cancers. · Prediction of M-stage is superior to N-stage. · Multiparametric PET/MRI displays a valuable platform for radiomics analyses .

AI as a New Frontier in Contrast Media Research: Bridging the Gap Between Contrast Media Reduction, the Contrast-Free Question and New Application Discoveries

ABSTRACT

Artificial intelligence (AI) techniques are currently harnessed to revolutionize the domain of medical imaging. This review investigates 3 major AI-driven approaches for contrast agent management: new frontiers in contrast agent dose reduction, the contrast-free question, and new applications. By examining recent studies that use AI as a new frontier in contrast media research, we synthesize the current state of the field and provide a comprehensive understanding of the potential and limitations of AI in this context. In doing so, we show the dose limits of reducing the amount of contrast agents and demonstrate why it might not be possible to completely eliminate contrast agents in the future. In addition, we highlight potential new applications to further increase the radiologist's sensitivity at normal doses. At the same time, this review shows which network architectures provide promising approaches and reveals possible artifacts of a paired image-to-image conversion. Furthermore, current US Food and Drug Administration regulatory guidelines regarding AI/machine learning-enabled medical devices are highlighted.

Closing the loop for AI-ready radiology

BACKGROUND

 In recent years, AI has made significant advancements in medical diagnosis and prognosis. However, the incorporation of AI into clinical practice is still challenging and under-appreciated. We aim to demonstrate a possible vertical integration approach to close the loop for AI-ready radiology.

METHOD

 This study highlights the importance of two-way communication for AI-assisted radiology. As a key part of the methodology, it demonstrates the integration of AI systems into clinical practice with structured reports and AI visualization, giving more insight into the AI system. By integrating cooperative lifelong learning into the AI system, we ensure the long-term effectiveness of the AI system, while keeping the radiologist in the loop.  RESULTS:  We demonstrate the use of lifelong learning for AI systems by incorporating AI visualization and structured reports. We evaluate Memory Aware-Synapses and Rehearsal approach and find that both approaches work in practice. Furthermore, we see the advantage of lifelong learning algorithms that do not require the storing or maintaining of samples from previous datasets.

CONCLUSION

 In conclusion, incorporating AI into the clinical routine of radiology requires a two-way communication approach and seamless integration of the AI system, which we achieve with structured reports and visualization of the insight gained by the model. Closing the loop for radiology leads to successful integration, enabling lifelong learning for the AI system, which is crucial for sustainable long-term performance.

KEY POINTS

  · The integration of AI systems into the clinical routine with structured reports and AI visualization.. · Two-way communication between AI and radiologists is necessary to enable AI that keeps the radiologist in the loop.. · Closing the loop enables lifelong learning, which is crucial for long-term, high-performing AI in radiology..

The importance of educational tools and a new software solution for visualizing and quantifying report correction in radiology training

A novel software, DiffTool, was developed in-house to keep track of changes made by board-certified radiologists to preliminary reports created by residents and evaluate its impact on radiological hands-on training. Before (t0) and after (t2-4) the deployment of the software, 18 residents (median age: 29 years; 33% female) completed a standardized questionnaire on professional training. At t2-4 the participants were also requested to respond to three additional questions to evaluate the software. Responses were recorded via a six-point Likert scale ranging from 1 ("strongly agree") to 6 ("strongly disagree"). Prior to the release of the software, 39% (7/18) of the residents strongly agreed with the statement that they manually tracked changes made by board-certified radiologists to each of their radiological reports while 61% were less inclined to agree with that statement. At t2-4, 61% (11/18) stated that they used DiffTool to track differences. Furthermore, we observed an increase from 33% (6/18) to 44% (8/18) of residents who agreed to the statement "I profit from every corrected report". The DiffTool was well accepted among residents with a regular user base of 72% (13/18), while 78% (14/18) considered it a relevant improvement to their training. The results of this study demonstrate the importance of providing a time-efficient way to analyze changes made to preliminary reports as an additive for professional training.

Information extraction from weakly structured radiological reports with natural language queries

OBJECTIVES

Provide physicians and researchers an efficient way to extract information from weakly structured radiology reports with natural language processing (NLP) machine learning models.

METHODS

We evaluate seven different German bidirectional encoder representations from transformers (BERT) models on a dataset of 857,783 unlabeled radiology reports and an annotated reading comprehension dataset in the format of SQuAD 2.0 based on 1223 additional reports.

RESULTS

Continued pre-training of a BERT model on the radiology dataset and a medical online encyclopedia resulted in the most accurate model with an F1-score of 83.97% and an exact match score of 71.63% for answerable questions and 96.01% accuracy in detecting unanswerable questions. Fine-tuning a non-medical model without further pre-training led to the lowest-performing model. The final model proved stable against variation in the formulations of questions and in dealing with questions on topics excluded from the training set.

CONCLUSIONS

General domain BERT models further pre-trained on radiological data achieve high accuracy in answering questions on radiology reports. We propose to integrate our approach into the workflow of medical practitioners and researchers to extract information from radiology reports.

CLINICAL RELEVANCE STATEMENT

By reducing the need for manual searches of radiology reports, radiologists' resources are freed up, which indirectly benefits patients.

KEY POINTS

• BERT models pre-trained on general domain datasets and radiology reports achieve high accuracy (83.97% F1-score) on question-answering for radiology reports. • The best performing model achieves an F1-score of 83.97% for answerable questions and 96.01% accuracy for questions without an answer. • Additional radiology-specific pretraining of all investigated BERT models improves their performance.

k-strip: A novel segmentation algorithm in k-space for the application of skull stripping

BACKGROUND AND OBJECTIVE

We present a novel deep learning-based skull stripping algorithm for magnetic resonance imaging (MRI) that works directly in the information rich complex valued k-space.

METHODS

Using four datasets from different institutions with a total of around 200,000 MRI slices, we show that our network can perform skull-stripping on the raw data of MRIs while preserving the phase information which no other skull stripping algorithm is able to work with. For two of the datasets, skull stripping performed by HD-BET (Brain Extraction Tool) in the image domain is used as the ground truth, whereas the third and fourth dataset comes with per-hand annotated brain segmentations.

RESULTS

All four datasets were very similar to the ground truth (DICE scores of 92 %-99 % and Hausdorff distances of under 5.5 pixel). Results on slices above the eye-region reach DICE scores of up to 99 %, whereas the accuracy drops in regions around the eyes and below, with partially blurred output. The output of k-Strip often has smoothed edges at the demarcation to the skull. Binary masks are created with an appropriate threshold.

CONCLUSION

With this proof-of-concept study, we were able to show the feasibility of working in the k-space frequency domain, preserving phase information, with consistent results. Besides preserving valuable information for further diagnostics, this approach makes an immediate anonymization of patient data possible, already before being transformed into the image domain. Future research should be dedicated to discovering additional ways the k-space can be used for innovative image analysis and further workflows.

Fully automated MR-based virtual biopsy of primary CNS lymphomas

Background

Primary central nervous system lymphomas (PCNSL) pose a challenge as they may mimic gliomas on magnetic resonance imaging (MRI) imaging, compelling precise differentiation for appropriate treatment. This study focuses on developing an automated MRI-based workflow to distinguish between PCNSL and gliomas.

Methods

MRI examinations of 240 therapy-naive patients (141 males and 99 females, mean age: 55.16 years) with cerebral gliomas and PCNSLs (216 gliomas and 24 PCNSLs), each comprising a non-contrast T1-weighted, fluid-attenuated inversion recovery (FLAIR), and contrast-enhanced T1-weighted sequence were included in the study. HD-GLIO, a pre-trained segmentation network, was used to generate segmentations automatically. To validate the segmentation efficiency, 237 manual segmentations were prepared (213 gliomas and 24 PCNSLs). Subsequently, radiomics features were extracted following feature selection and training of an XGBoost algorithm for classification.

Results

The segmentation models for gliomas and PCNSLs achieved a mean Sørensen-Dice coefficient of 0.82 and 0.80 for whole tumors, respectively. Three classification models were developed in this study to differentiate gliomas from PCNSLs. The first model differentiated PCNSLs from gliomas, with an area under the curve (AUC) of 0.99 (F1-score: 0.75). The second model discriminated between high-grade gliomas and PCNSLs with an AUC of 0.91 (F1-score: 0.6), and the third model differentiated between low-grade gliomas and PCNSLs with an AUC of 0.95 (F1-score: 0.89).

Conclusions

This study serves as a pilot investigation presenting an automated virtual biopsy workflow that distinguishes PCNSLs from cerebral gliomas. Prior to clinical use, it is necessary to validate the results in a prospective multicenter setting with a larger number of PCNSL patients.

Prognostic value of deep learning-derived body composition in advanced pancreatic cancer-a retrospective multicenter study

BACKGROUND

Despite the prognostic relevance of cachexia in pancreatic cancer, individual body composition has not been routinely integrated into treatment planning. In this multicenter study, we investigated the prognostic value of sarcopenia and myosteatosis automatically extracted from routine computed tomography (CT) scans of patients with advanced pancreatic ductal adenocarcinoma (PDAC).

PATIENTS AND METHODS

We retrospectively analyzed clinical imaging data of 601 patients from three German cancer centers. We applied a deep learning approach to assess sarcopenia by the abdominal muscle-to-bone ratio (MBR) and myosteatosis by the ratio of abdominal inter- and intramuscular fat to muscle volume. In the pooled cohort, univariable and multivariable analyses were carried out to analyze the association between body composition markers and overall survival (OS). We analyzed the relationship between body composition markers and laboratory values during the first year of therapy in a subgroup using linear regression analysis adjusted for age, sex, and American Joint Committee on Cancer (AJCC) stage.

RESULTS

Deep learning-derived MBR [hazard ratio (HR) 0.60, 95% confidence interval (CI) 0.47-0.77, P < 0.005] and myosteatosis (HR 3.73, 95% CI 1.66-8.39, P < 0.005) were significantly associated with OS in univariable analysis. In multivariable analysis, MBR (P = 0.019) and myosteatosis (P = 0.02) were associated with OS independent of age, sex, and AJCC stage. In a subgroup, MBR and myosteatosis were associated with albumin and C-reactive protein levels after initiation of therapy. Additionally, MBR was also associated with hemoglobin and total protein levels.

CONCLUSIONS

Our work demonstrates that deep learning can be applied across cancer centers to automatically assess sarcopenia and myosteatosis from routine CT scans. We highlight the prognostic role of our proposed markers and show a strong relationship with protein levels, inflammation, and anemia. In clinical practice, automated body composition analysis holds the potential to further personalize cancer treatment.

Optimizing platelet transfusion through a personalized deep learning risk assessment system for demand management

ABSTRACT

Platelet demand management (PDM) is a resource-consuming task for physicians and transfusion managers of large hospitals. Inpatient numbers and institutional standards play significant roles in PDM. However, reliance on these factors alone commonly results in platelet shortages. Using data from multiple sources, we developed, validated, tested, and implemented a patient-specific approach to support PDM that uses a deep learning-based risk score to forecast platelet transfusions for each hospitalized patient in the next 24 hours. The models were developed using retrospective electronic health record data of 34 809 patients treated between 2017 and 2022. Static and time-dependent features included demographics, diagnoses, procedures, blood counts, past transfusions, hematotoxic medications, and hospitalization duration. Using an expanding window approach, we created a training and live-prediction pipeline with a 30-day input and 24-hour forecast. Hyperparameter tuning determined the best validation area under the precision-recall curve (AUC-PR) score for long short-term memory deep learning models, which were then tested on independent data sets from the same hospital. The model tailored for hematology and oncology patients exhibited the best performance (AUC-PR, 0.84; area under the receiver operating characteristic curve [ROC-AUC], 0.98), followed by a multispecialty model covering all other patients (AUC-PR, 0.73). The model specific to cardiothoracic surgery had the lowest performance (AUC-PR, 0.42), likely because of unexpected intrasurgery bleedings. To our knowledge, this is the first deep learning-based platelet transfusion predictor enabling individualized 24-hour risk assessments at high AUC-PR. Implemented as a decision-support system, deep-learning forecasts might improve patient care by detecting platelet demand earlier and preventing critical transfusion shortages.

A Systematic Comparison of Task Adaptation Techniques for Digital Histopathology

Due to an insufficient amount of image annotation, artificial intelligence in computational histopathology usually relies on fine-tuning pre-trained neural networks. While vanilla fine-tuning has shown to be effective, research on computer vision has recently proposed improved algorithms, promising better accuracy. While initial studies have demonstrated the benefits of these algorithms for medical AI, in particular for radiology, there is no empirical evidence for improved accuracy in histopathology. Therefore, based on the ConvNeXt architecture, our study performs a systematic comparison of nine task adaptation techniques, namely, DELTA, L2-SP, MARS-PGM, Bi-Tuning, BSS, MultiTune, SpotTune, Co-Tuning, and vanilla fine-tuning, on five histopathological classification tasks using eight datasets. The results are based on external testing and statistical validation and reveal a multifaceted picture: some techniques are better suited for histopathology than others, but depending on the classification task, a significant relative improvement in accuracy was observed for five advanced task adaptation techniques over the control method, i.e., vanilla fine-tuning (e.g., Co-Tuning: P(≫) = 0.942, d = 2.623). Furthermore, we studied the classification accuracy for three of the nine methods with respect to the training set size (e.g., Co-Tuning: P(≫) = 0.951, γ = 0.748). Overall, our results show that the performance of advanced task adaptation techniques in histopathology is affected by influencing factors such as the specific classification task or the size of the training dataset.

Falling off the screening grid-Predictors for postponed utilization of psycho-oncological support in cancer patients and its implications for distress assessment and management

OBJECTIVE

Distress assessment of cancer patients is considered state-of-the-art. In addition to distress scores, individual care needs are an important factor for the initiation of psycho-oncological interventions. In a mono-centric, observational study, we aimed for characterization of patients indicating a subjective need but declining to utilize support services immediately to facilitate implementation of adapted screenings.

METHODS

This study analyzed retrospective data from routine distress screening and associated data from hospital records. Descriptive, variance and regression analyses were used to assess characteristics of postponed support utilization in patients with mixed cancer diagnoses in different treatment settings.

RESULTS

Of the total sample (N = 1863), 13% indicated a subjective need but postponed support utilization. This subgroup presented as being as burdened by symptoms of depression (p < 0.001), anxiety (p < 0.001) and distress (p < 0.001) as subjectively distressed patients with intent to directly utilize support. Time periods since diagnosis were shorter (p = 0.007) and patients were more often inpatients (p = 0.045).

CONCLUSIONS

Despite high heterogeneity among the subgroups, this study identified distress-related factors and time since diagnosis as possible predictors for postponed utilization of psycho-oncological interventions. Results suggest the necessity for time-individualized support which may improve utilization by distressed patients.

Preoperative Computed Tomography Volumetry and Graft Weight Estimation of Left Lateral Segment in Pediatric Living Donor Liver Transplant

OBJECTIVES

Liver volumetry based on a computed tomography scan is widely used to estimate liver volume before any liver resection, especially before living donorliver donation. The 1-to-1 conversion rule for liver volume to liver weight has been widely adopted; however, debate continues regarding this approach. Therefore, we analyzed the relationship between the left-lateral lobe liver graft volume and actual graft weight.

MATERIALS AND METHODS

This study retrospectively included consecutive donors who underwent left lateral hepatectomy for pediatric living donor liver transplant from December 2008 to September 2020. All donors were healthy adults who met the evaluation criteria for pediatric living donor liver transplant and underwent a preoperative contrast-enhanced computed tomography scan. Manual segmentation of the leftlateral liverlobe for graft volume estimation and intraoperative measurement of an actual graft weight were performed. The relationship between estimated graft volume and actual graft weight was analyzed.

RESULTS

Ninety-four living liver donors were included in the study. The mean actual graft weight was ~283.4 ± 68.5 g, and the mean graft volume was 244.9 ± 63.86 mL. A strong correlation was shown between graft volume and actual graft weight (r = 0.804; P < .001). Bland-Altman analysis revealed an interobserver agreement of 38.0 ± 97.25, and intraclass correlation coefficient showed almost perfect agreement(r = 0.840; P < .001). The conversion formula for calculating graft weight based on computed tomography volumetry was determined based on regression analysis: 0.88 × graft volume + 41.63.

CONCLUSIONS

The estimation of left liver graft weight using only the 1-to-1 rule is subject to measurable variability in calculated graft weights and tends to underestimate the true graft weight. Instead, a different, improved conversion formula should be used to calculate graft weight to more accurately determine donor graft weight-to-recipient body weightratio and reduce the risk of underestimation of liver graft weightin the donor selection process before pediatric living donor liver transplant.