Imaging Registries and Single-Center Series

Improved Robustness for Deep Learning-based Segmentation of Multi-Center Myocardial Perfusion MRI Datasets Using Data Adaptive Uncertainty-guided Space-time Analysis

BACKGROUND

Fully automatic analysis of myocardial perfusion MRI datasets enables rapid and objective reporting of stress/rest studies in patients with suspected ischemic heart disease. Developing deep learning techniques that can analyze multi-center datasets despite limited training data and variations in software (pulse sequence) and hardware (scanner vendor) is an ongoing challenge.

METHODS

Datasets from 3 medical centers acquired at 3T (n = 150 subjects; 21,150 first-pass images) were included: an internal dataset (inD; n = 95) and two external datasets (exDs; n = 55) used for evaluating the robustness of the trained deep neural network (DNN) models against differences in pulse sequence (exD-1) and scanner vendor (exD-2). A subset of inD (n = 85) was used for training/validation of a pool of DNNs for segmentation, all using the same spatiotemporal U-Net architecture and hyperparameters but with different parameter initializations. We employed a space-time sliding-patch analysis approach that automatically yields a pixel-wise "uncertainty map" as a byproduct of the segmentation process. In our approach, dubbed Data Adaptive Uncertainty-Guided Space-time (DAUGS) analysis, a given test case is segmented by all members of the DNN pool and the resulting uncertainty maps are leveraged to automatically select the "best" one among the pool of solutions. For comparison, we also trained a DNN using the established approach with the same settings (hyperparameters, data augmentation, etc.).

RESULTS

The proposed DAUGS analysis approach performed similarly to the established approach on the internal dataset (Dice score for the testing subset of inD: 0.896 ± 0.050 vs. 0.890 ± 0.049; p = n.s.) whereas it significantly outperformed on the external datasets (Dice for exD-1: 0.885 ± 0.040 vs. 0.849 ± 0.065, p < 0.005; Dice for exD-2: 0.811 ± 0.070 vs. 0.728 ± 0.149, p < 0.005). Moreover, the number of image series with "failed" segmentation (defined as having myocardial contours that include bloodpool or are noncontiguous in ≥1 segment) was significantly lower for the proposed vs. the established approach (4.3% vs. 17.1%, p < 0.0005).

CONCLUSIONS

The proposed DAUGS analysis approach has the potential to improve the robustness of deep learning methods for segmentation of multi-center stress perfusion datasets with variations in the choice of pulse sequence, site location or scanner vendor.

The Society for Cardiovascular Magnetic Resonance Registry at 150,000

BACKGROUND

Cardiovascular magnetic resonance (CMR) is increasingly utilized to evaluate expanding cardiovascular conditions. The Society for Cardiovascular Magnetic Resonance (SCMR) Registry is a central repository for real-world clinical data to support cardiovascular research, including those relating to outcomes, quality improvement, and machine learning. The SCMR Registry is built on a regulatory-compliant, cloud-based infrastructure that houses searchable content and Digital Imaging and Communications in Medicine images. The goal of this study is to summarize the status of the SCMR Registry at 150,000 exams.

METHODS

The processes for data security, data submission, and research access are outlined. We interrogated the Registry and presented a summary of its contents.

RESULTS

Data were compiled from 154,458 CMR scans across 20 United States sites, containing 299,622,066 total images (∼100 terabytes of storage). Across reported values, the human subjects had an average age of 58 years (range 1 month to >90 years old), were 44% (63,070/145,275) female, 72% (69,766/98,008) Caucasian, and had a mortality rate of 8% (9,962/132,979). The most common indication was cardiomyopathy (35,369/131,581, 27%), and most frequently used current procedural terminology code was 75561 (57,195/162,901, 35%). Macrocyclic gadolinium-based contrast agents represented 89% (83,089/93,884) of contrast utilization after 2015. Short-axis cines were performed in 99% (76,859/77,871) of tagged scans, short-axis late gadolinium enhancement (LGE) in 66% (51,591/77,871), and stress perfusion sequences in 30% (23,241/77,871). Mortality data demonstrated increased mortality in patients with left ventricular ejection fraction <35%, the presence of wall motion abnormalities, stress perfusion defects, and infarct LGE, compared to those without these markers. There were 456,678 patient-years of all-cause mortality follow-up, with a median follow-up time of 3.6 years.

CONCLUSION

The vision of the SCMR Registry is to promote evidence-based utilization of CMR through a collaborative effort by providing a web mechanism for centers to securely upload de-identified data and images for research, education, and quality control. The Registry quantifies changing practice over time and supports large-scale real-world multicenter observational studies of prognostic utility.

Deep learning to automatically classify very large sets of preoperative and postoperative shoulder arthroplasty radiographs

BACKGROUND

Joint arthroplasty registries usually lack information on medical imaging owing to the laborious process of observing and recording, as well as the lack of standard methods to transfer the imaging information to the registries, which can limit the investigation of various research questions. Artificial intelligence (AI) algorithms can automate imaging-feature identification with high accuracy and efficiency. With the purpose of enriching shoulder arthroplasty registries with organized imaging information, it was hypothesized that an automated AI algorithm could be developed to classify and organize preoperative and postoperative radiographs from shoulder arthroplasty patients according to laterality, radiographic projection, and implant type.

METHODS

This study used a cohort of 2303 shoulder radiographs from 1724 shoulder arthroplasty patients. Two observers manually labeled all radiographs according to (1) laterality (left or right), (2) projection (anteroposterior, axillary, or lateral), and (3) whether the radiograph was a preoperative radiograph or showed an anatomic total shoulder arthroplasty or a reverse shoulder arthroplasty. All these labeled radiographs were randomly split into developmental and testing sets at the patient level and based on stratification. By use of 10-fold cross-validation, a 3-task deep-learning algorithm was trained on the developmental set to classify the 3 aforementioned characteristics. The trained algorithm was then evaluated on the testing set using quantitative metrics and visual evaluation techniques.

RESULTS

The trained algorithm perfectly classified laterality (F1 scores [harmonic mean values of precision and sensitivity] of 100% on the testing set). When classifying the imaging projection, the algorithm achieved F1 scores of 99.2%, 100%, and 100% on anteroposterior, axillary, and lateral views, respectively. When classifying the implant type, the model achieved F1 scores of 100%, 95.2%, and 100% on preoperative radiographs, anatomic total shoulder arthroplasty radiographs, and reverse shoulder arthroplasty radiographs, respectively. Visual evaluation using integrated maps showed that the algorithm focused on the relevant patient body and prosthesis parts for classification. It took the algorithm 20.3 seconds to analyze 502 images.

CONCLUSIONS

We developed an efficient, accurate, and reliable AI algorithm to automatically identify key imaging features of laterality, imaging view, and implant type in shoulder radiographs. This algorithm represents the first step to automatically classify and organize shoulder radiographs on a large scale in very little time, which will profoundly enrich shoulder arthroplasty registries.

A practical guide to the development and deployment of deep learning models for the orthopaedic surgeon: Part III, focus on registry creation, diagnosis, and data privacy

Deep learning is a subset of artificial intelligence (AI) with enormous potential to transform orthopaedic surgery. As has already become evident with the deployment of Large Language Models (LLMs) like ChatGPT (OpenAI Inc.), deep learning can rapidly enter clinical and surgical practices. As such, it is imperative that orthopaedic surgeons acquire a deeper understanding of the technical terminology, capabilities and limitations associated with deep learning models. The focus of this series thus far has been providing surgeons with an overview of the steps needed to implement a deep learning-based pipeline, emphasizing some of the important technical details for surgeons to understand as they encounter, evaluate or lead deep learning projects. However, this series would be remiss without providing practical examples of how deep learning models have begun to be deployed and highlighting the areas where the authors feel deep learning may have the most profound potential. While computer vision applications of deep learning were the focus of Parts I and II, due to the enormous impact that natural language processing (NLP) has had in recent months, NLP-based deep learning models are also discussed in this final part of the series. In this review, three applications that the authors believe can be impacted the most by deep learning but with which many surgeons may not be familiar are discussed: (1) registry construction, (2) diagnostic AI and (3) data privacy. Deep learning-based registry construction will be essential for the development of more impactful clinical applications, with diagnostic AI being one of those applications likely to augment clinical decision-making in the near future. As the applications of deep learning continue to grow, the protection of patient information will become increasingly essential; as such, applications of deep learning to enhance data privacy are likely to become more important than ever before. Level of Evidence: Level IV.

Medical Imaging Applications Developed Using Artificial Intelligence Demonstrate High Internal Validity Yet Are Limited in Scope and Lack External Validation

PURPOSE

To (1) review definitions and concepts necessary to interpret applications of deep learning (DL; a domain of artificial intelligence that leverages neural networks to make predictions on media inputs such as images) and (2) identify knowledge and translational gaps in the literature to provide insight into specific areas for improvement as adoption of this technology continues.

METHODS

A comprehensive search of the literature was performed in December 2023 for articles regarding the use of DL in sports medicine. For each study, information regarding the joint of focus, specific anatomic structure/pathology to which DL was applied, imaging modality utilized, source of images used for model training and testing, data set size, model performance, and whether the DL model was externally validated was recorded. A numerical scale was used to rate each DL model's clinical impact, with 1 corresponding to proof-of-concept studies with little to no direct clinical impact and 5 corresponding to practice-changing clinical impact and ready for clinical deployment.

RESULTS

Fifty-five studies were identified, all of which were published within the past 5 years, while 82% were published within the past 3 years. Of the DL models identified, 84% were developed for classification tasks, 9% for automated measurements, and 7% for segmentation. A total of 62% of studies utilized magnetic resonance imaging as the imaging modality, 25% radiographs, and 7% ultrasound, while 1 study each used computed tomography, arthroscopic images, or arthroscopic video. Sixty-five percent of studies focused on the detection of tears (anterior cruciate ligament [ACL], rotator cuff [RC], and meniscus). The diagnostic performance of ACL tears, as determined by the area under the receiver operator curve (AUROC), ranged from 0.81 to 0.99 for ACL tears (excellent to near perfect), 0.83 to 0.94 for RC tears (excellent), and from 0.75 to 0.96 for meniscus tears (acceptable to excellent). In addition, 3 studies focused on detection of cartilage lesions had AUROC ranging from 0.90 to 0.92 (excellent performance). However, only 4 (7%) studies externally validated their models, suggesting that they may not be generalizable or may not perform well when applied to populations other than that used to develop the model. Finally, the mean clinical impact score was 2 (range, 1-3) on scale of 1 to 5, corresponding to limited clinical applicability.

CONCLUSIONS

DL models in orthopaedic sports medicine show generally excellent performance (high internal validity) but require external validation to facilitate clinical deployment. In addition, current models have low clinical applicability and fail to advance the field due to a focus on routine tasks and a narrow conceptual framework.

LEVEL OF EVIDENCE

Level IV, scoping review of Level I to IV studies.

Guidelines of the International Headache Society for Clinic-Based Headache Registries, 1st edition

Clinic-based headache registries collect data for a wide variety of purposes including delineating disease characteristics, longitudinal natural disease courses, headache management approaches, quality of care, treatment safety and effectiveness, factors that predict treatment response, health care resource utilization, clinician adherence to guidelines, and cost-effectiveness. Registry data are valuable for numerous stakeholders, including individuals with headache disorders and their caregivers, healthcare providers, scientists, healthcare systems, regulatory authorities, pharmaceutical companies, employers, and policymakers. This International Headache Society document may serve as guidance for developing clinic-based headache registries. Use of registry data requires a formal research protocol that includes: 1) research aims; 2) methods for data collection, harmonization, analysis, privacy, and protection; 3) methods for human subject protection; and 4) publication and dissemination plans. Depending upon their objectives, headache registries should include validated headache-specific questionnaires, patient reported outcome measures, data elements that are used consistently across studies (i.e., "common data elements"), and medical record data. Amongst other data types, registries may be linked to healthcare and pharmacy claims data, biospecimens, and neuroimaging data. Headache diagnoses should be made according to the International Classification of Headache Disorders diagnostic criteria. The data from well-designed headache registries can provide wide-ranging and novel insights into the characteristics, burden, and treatment of headache disorders and ultimately lead to improvements in the management of patients with headache.

Applying Deep Learning to Establish a Total Hip Arthroplasty Radiography Registry: A Stepwise Approach

BACKGROUND

Establishing imaging registries for large patient cohorts is challenging because manual labeling is tedious and relying solely on DICOM (digital imaging and communications in medicine) metadata can result in errors. We endeavored to establish an automated hip and pelvic radiography registry of total hip arthroplasty (THA) patients by utilizing deep-learning pipelines. The aims of the study were (1) to utilize these automated pipelines to identify all pelvic and hip radiographs with appropriate annotation of laterality and presence or absence of implants, and (2) to automatically measure acetabular component inclination and version for THA images.

METHODS

We retrospectively retrieved 846,988 hip and pelvic radiography DICOM files from 20,378 patients who underwent primary or revision THA performed at our institution from 2000 to 2020. Metadata for the files were screened followed by extraction of imaging data. Two deep-learning algorithms (an EfficientNetB3 classifier and a YOLOv5 object detector) were developed to automatically determine the radiographic appearance of all files. Additional deep-learning algorithms were utilized to automatically measure the acetabular angles on anteroposterior pelvic and lateral hip radiographs. Algorithm performance was compared with that of human annotators on a random test sample of 5,000 radiographs.

RESULTS

Deep-learning algorithms enabled appropriate exclusion of 209,332 DICOM files (24.7%) as misclassified non-hip/pelvic radiographs or having corrupted pixel data. The final registry was automatically curated and annotated in <8 hours and included 168,551 anteroposterior pelvic, 176,890 anteroposterior hip, 174,637 lateral hip, and 117,578 oblique hip radiographs. The algorithms achieved 99.9% accuracy, 99.6% precision, 99.5% recall, and a 99.6% F1 score in determining the radiograph appearance.

CONCLUSIONS

We developed a highly accurate series of deep-learning algorithms to rapidly curate and annotate THA patient radiographs. This efficient pipeline can be utilized by other institutions or registries to construct radiography databases for patient care, longitudinal surveillance, and large-scale research. The stepwise approach for establishing a radiography registry can further be utilized as a workflow guide for other anatomic areas.

LEVEL OF EVIDENCE

Diagnostic Level IV . See Instructions for Authors for a complete description of levels of evidence.

The American Registry for Migraine Research: Research Methods and Baseline Data for an Initial Patient Cohort

BACKGROUND

The American Registry for Migraine Research (ARMR) is a multicenter, prospective, longitudinal patient registry, biorepository, and neuroimaging repository that collects clinical data, electronic health record (EHR) data, blood samples, and brain imaging data from individuals with migraine or other headache types. In this manuscript, we outline ARMR research methods and report baseline data describing an initial cohort of ARMR participants.

METHODS

Adults with any International Classification of Headache Disorders (ICHD) diagnosis were prospectively enrolled from one of the 8 participating headache specialty centers. At baseline, ARMR participants complete web-based questionnaires, clinicians enter the participant's ICHD diagnoses, an optional blood specimen is collected, and neuroimaging data are uploaded to the ARMR neuroimaging repository. Participants maintain the ARMR daily headache diary longitudinally and follow-up questionnaires are completed by participants every 3 months. EHR data are integrated into the ARMR database from a subset of ARMR sites. Herein, we describe the ARMR methodology and report the summary data from ARMR participants who had, from February 2016 to May 2019, completed at least 1 baseline questionnaire from which data are reported in this manuscript. Descriptive statistics are used to provide an overview of patient's sociodemographics, headache diagnoses, headache characteristics, most bothersome symptoms other than headache, headache-related disability, comorbidities, and treatments.

RESULTS

Data were available from 996 ARMR participants, enrolled from Mayo Clinic Arizona, Dartmouth-Hitchcock Medical Center, University of Utah, University of Colorado, Thomas Jefferson University, University of Texas Health Science Center at Houston, Georgetown University Medical Center, and DENT Neurological Institute. Among ARMR participants, 86.7% (n = 864) were female and the mean age at the time of enrollment was 48.6 years (±13.9; range 18-84). The most common provider-reported diagnosis was chronic migraine (n = 622), followed by migraine without aura (n = 327), migraine with aura (n = 196), and medication overuse headache (n = 65). Average headache frequency was 19.1 ± 9.2 days per month (n = 751), with 68% reporting at least 15 headache days per month. Sensitivity to light was the most frequent (n = 222) most bothersome symptom overall, other than headache, but when present, cognitive dysfunction was most frequently (n = 157) the most bothersome symptom other than headache. Average migraine disability assessment (MIDAS) score was 52 ± 49 (n = 760), (very severe headache-related disability); however, 17% of the ARMR population had MIDAS scores suggesting "no" or "mild" disability. The most common non-headache health issues were allergies (n = 364), back pain (n = 296), neck pain (n = 296), depression (n = 292), and anxiety (n = 278). Nearly 85% (n = 695) of patients were using preventive medications and 24.7% were using non-medication preventive therapy (eg, vitamins and neuromodulation). The most common preventive medication classes were neurotoxins, anticonvulsants, antidepressants, vitamins/supplements, and anticalcitonin gene-related peptide ligand or receptor-targeted monoclonal antibodies. Nearly 90% (n = 734) of ARMR participants was taking medications to treat migraine attacks, with the most common classes being triptans, non-steroidal anti-inflammatory drugs, antiemetics, acetaminophen, and combination analgesics.

CONCLUSIONS

ARMR is a source of real-world patient data, biospecimens, and brain neuroimaging data that provides comprehensive insight into patients with migraine and other headache types being seen in headache specialty clinics in the United States. ARMR data will allow for longitudinal and advanced analytics that are expected to lead to a better characterization of patient heterogeneity, healthcare resource utilization, identification of endophenotypes, factors that predict treatment outcomes and clinical course, and ultimately advance the field toward precision headache medicine.