A DICOM Framework for Machine Learning and Processing Pipelines Against Real-time Radiology Images

Visualizing Clinical Data Retrieval and Curation in Multimodal Healthcare AI Research: A Technical Note on RIL-workflow

Curating and integrating data from sources are bottlenecks to procuring robust training datasets for artificial intelligence (AI) models in healthcare. While numerous applications can process discrete types of clinical data, it is still time-consuming to integrate heterogenous data types. Therefore, there exists a need for more efficient retrieval and storage of curated patient data from dissimilar sources, such as biobanks, health records, and sensors. We describe a customizable, modular data retrieval application (RIL-workflow), which integrates clinical notes, images, and prescription data, and show its feasibility applied to research at our institution. It uses the workflow automation platform Camunda (Camunda Services GmbH, Berlin, Germany) to collect internal data from Fast Healthcare Interoperability Resources (FHIR) and Digital Imaging and Communications in Medicine (DICOM) sources. Using the web-based graphical user interface (GUI), the workflow runs tasks to completion according to visual representation, retrieving and storing results for patients meeting study inclusion criteria while segregating errors for human review. We showcase RIL-workflow with its library of ready-to-use modules, enabling researchers to specify human input or automation at fixed steps. We validated our workflow by demonstrating its capability to aggregate, curate, and handle errors related to data from multiple sources to generate a multimodal database for clinical AI research. Further, we solicited user feedback to highlight the pros and cons associated with RIL-workflow. The source code is available at github.com/magnooj/RIL-workflow.

IODeep: An IOD for the introduction of deep learning in the DICOM standard

BACKGROUND AND OBJECTIVE

In recent years, Artificial Intelligence (AI) and in particular Deep Neural Networks (DNN) became a relevant research topic in biomedical image segmentation due to the availability of more and more data sets along with the establishment of well known competitions. Despite the popularity of DNN based segmentation on the research side, these techniques are almost unused in the daily clinical practice even if they could support effectively the physician during the diagnostic process. Apart from the issues related to the explainability of the predictions of a neural model, such systems are not integrated in the diagnostic workflow, and a standardization of their use is needed to achieve this goal.

METHODS

This paper presents IODeep a new DICOM Information Object Definition (IOD) aimed at storing both the weights and the architecture of a DNN already trained on a particular image dataset that is labeled as regards the acquisition modality, the anatomical region, and the disease under investigation.

RESULTS

The IOD architecture is presented along with a DNN selection algorithm from the PACS server based on the labels outlined above, and a simple PACS viewer purposely designed for demonstrating the effectiveness of the DICOM integration, while no modifications are required on the PACS server side. Also a service based architecture in support of the entire workflow has been implemented.

CONCLUSION

IODeep ensures full integration of a trained AI model in a DICOM infrastructure, and it is also enables a scenario where a trained model can be either fine-tuned with hospital data or trained in a federated learning scheme shared by different hospitals. In this way AI models can be tailored to the real data produced by a Radiology ward thus improving the physician decision making process. Source code is freely available at https://github.com/CHILab1/IODeep.git.

Opportunistic screening for coronary artery calcium deposition using chest radiographs - a multi-objective models with multi-modal data fusion

Background

To create an opportunistic screening strategy by multitask deep learning methods to stratify prediction for coronary artery calcium (CAC) and associated cardiovascular risk with frontal chest x-rays (CXR) and minimal data from electronic health records (EHR).

Methods

In this retrospective study, 2,121 patients with available computed tomography (CT) scans and corresponding CXR images were collected internally (Mayo Enterprise) with calculated CAC scores binned into 3 categories (0, 1-99, and 100+) as ground truths for model training. Results from the internal training were tested on multiple external datasets (domestic (EUH) and foreign (VGHTPE)) with significant racial and ethnic differences and classification performance was compared.

Findings

Classification performance between 0, 1-99, and 100+ CAC scores performed moderately on both the internal test and external datasets, reaching average f1-score of 0.66 for Mayo, 0.62 for EUH and 0.61 for VGHTPE. For the clinically relevant binary task of 0 vs 400+ CAC classification, the performance of our model on the internal test and external datasets reached an average AUCROC of 0.84.

Interpretation

The fusion model trained on CXR performed better (0.84 average AUROC on internal and external dataset) than existing state-of-the-art models on predicting CAC scores only on internal (0.73 AUROC), with robust performance on external datasets. Thus, our proposed model may be used as a robust, first-pass opportunistic screening method for cardiovascular risk from regular chest radiographs. For community use, trained model and the inference code can be downloaded with an academic open-source license from https://github.com/jeong-jasonji/MTL_CAC_classification .

Funding

The study was partially supported by National Institute of Health 1R01HL155410-01A1 award.

Using Deep Learning to Predict Treatment Response in Patients with Hepatocellular Carcinoma Treated with Y90 Radiation Segmentectomy

Treatment of hepatocellular carcinoma (HCC) with Y90 radioembolization segmentectomy (Y90-RE) demonstrates a tumor dose-response threshold, where dose estimates are highly dependent on accurate SPECT/CT acquisition, registration, and reconstruction. Any error can result in distorted absorbed dose distributions and inaccurate estimates of treatment success. This study improves upon the voxel-based dosimetry model, one of the most accurate methods available clinically, by using a deep convolutional network ensemble to account for the spatially variable uptake of Y90 within a treated lesion. A retrospective analysis was conducted in patients with HCC who received Y90-RE at a single institution. Seventy-seven patients with 103 lesions met the inclusion criteria: three or fewer tumors, pre- and post treatment MRI, and no prior Y90-RE. Lesions were labeled as complete (n = 57) or incomplete response (n = 46) based on 3-month post treatment MRI and divided by medical record number into a 20% hold-out test set and 80% training set with 5-fold cross-validation. Slice-wise predictions were made from an average ensemble of models and thresholds from the highest accuracy epochs across all five folds. Lesion predictions were made by thresholding all slice predictions through the lesion. When compared to the voxel-based dosimetry model, our model had a higher F1-score (0.72 vs. 0.2), higher accuracy (0.65 vs. 0.60), and higher sensitivity (1.0 vs. 0.11) at predicting complete treatment response. This algorithm has the potential to identify patients with treatment failure who may benefit from earlier follow-up or additional treatment.

The EMory BrEast imaging Dataset (EMBED): A Racially Diverse, Granular Dataset of 3.4 Million Screening and Diagnostic Mammographic Images

Supplemental material is available for this article. Keywords: Mammography, Breast, Machine Learning © RSNA, 2023.

Corner Detection of the Computer VR Microscope Image Based on the 3D Reconstruction Algorithm

In order to solve the problem of multisolution and ill-formedness of the 3D reconstruction method of a single image (purpose), the author proposes a microscope image segmentation algorithm based on the Harris multiscale corner detection. Separating complex engineering images into several simple basic geometric shapes, rebuild them separately to avoid the ill-conditioned solution problem of directly recovering depth information. In order to improve the registration accuracy of the corner-based image registration algorithm, the idea of multiresolution analysis was introduced into the classic Harris corner detection, and a gray intensity variation formula based on the wavelet transform was constructed, and a scale transformation characteristic was obtained so that the improved Harris corner detection algorithm is invariant to rotation, translation, and scale. Experimental results show that after reconstruction, the error between the length of the object measured based on the point cloud data and the actual length of the object is small, and both remain within the error range of 3 mm. The experiment verifies the fast, accurate, and stable characteristics of the improved algorithm.

Understanding Scanner Utilization With Real-Time DICOM Metadata Extraction

Understanding system performance metrics ensures better utilization of the radiology resources with more targeted interventions. The images produced by radiology scanners typically follow the DICOM (Digital Imaging and Communications in Medicine) standard format. The DICOM images consist of textual metadata that can be used to calculate key timing parameters, such as the exact study durations and scanner utilization. However, hospital networks lack the resources and capabilities to extract the metadata from the images quickly and automatically compute the scanner utilization properties. Thus, they resort to using data records from the Radiology Information Systems (RIS). However, data acquired from RIS are prone to human errors, rendering many derived key performance metrics inadequate and inaccurate. Hence, there is motivation to establish a real-time image transfer from the Picture Archiving and Communication Systems (PACS) to receive the DICOM images from the scanners to research clusters to conduct such metadata processing to evaluate scanner utilization metrics efficiently and quickly. This paper analyzes the scanners' utilization by developing a real-time monitoring framework that retrieves radiology images into a research cluster using the DICOM networking protocol and then extracts and processes the metadata from the images. Our proposed approach facilitates a better understanding of scanner utilization across a vast healthcare network by observing properties such as study duration, the interval between the encounters, and the series count of studies. Benchmarks against using the RIS data indicate that our proposed framework based on real-time PACS data estimates the scanner utilization more accurately. Furthermore, our framework has been running stable and performing its computation for more than two years on our extensive healthcare network in pseudo real-time.