Generative models: an upcoming innovation in musculoskeletal radiology? A preliminary test in spine imaging

Conversion of T2-Weighted Magnetic Resonance Images of Cervical Spine Trauma to Short T1 Inversion Recovery (STIR) Images by Generative Adversarial Network

INTRODUCTION

The short T1 inversion recovery (STIR) sequence is advantageous for visualizing ligamentous injuries, but the STIR sequence may be missing in some cases. The purpose of this study was to generate synthetic STIR images from MRI T2-weighted images (T2WI) of patients with cervical spine trauma using a generative adversarial network (GAN).  Methods: A total of 969 pairs of T2WI and STIR images were extracted from 79 patients with cervical spine trauma. The synthetic model was trained 100 times, and the performance of the model was evaluated with five-fold cross-validation.  Results: As for quantitative validation, the structural similarity score was 0.519±0.1 and the peak signal-to-noise ratio score was 19.37±1.9 dB. As for qualitative validation, the incorporation of synthetic STIR images generated by a GAN alongside T2WI substantially enhances sensitivity in the detection of interspinous ligament injuries, outperforming assessments reliant solely on T2WI.

CONCLUSION

The GAN model can generate synthetic STIRs from T2 images of cervical spine trauma using image-to-image conversion techniques. The use of a combination of synthetic STIR images generated by a GAN and T2WI improves sensitivity in detecting interspinous ligament injuries compared to assessments that use only T2WI.

AI applications in musculoskeletal imaging: a narrative review

This narrative review focuses on clinical applications of artificial intelligence (AI) in musculoskeletal imaging. A range of musculoskeletal disorders are discussed using a clinical-based approach, including trauma, bone age estimation, osteoarthritis, bone and soft-tissue tumors, and orthopedic implant-related pathology. Several AI algorithms have been applied to fracture detection and classification, which are potentially helpful tools for radiologists and clinicians. In bone age assessment, AI methods have been applied to assist radiologists by automatizing workflow, thus reducing workload and inter-observer variability. AI may potentially aid radiologists in identifying and grading abnormal findings of osteoarthritis as well as predicting the onset or progression of this disease. Either alone or combined with radiomics, AI algorithms may potentially improve diagnosis and outcome prediction of bone and soft-tissue tumors. Finally, information regarding appropriate positioning of orthopedic implants and related complications may be obtained using AI algorithms. In conclusion, rather than replacing radiologists, the use of AI should instead help them to optimize workflow, augment diagnostic performance, and keep up with ever-increasing workload.Relevance statement This narrative review provides an overview of AI applications in musculoskeletal imaging. As the number of AI technologies continues to increase, it will be crucial for radiologists to play a role in their selection and application as well as to fully understand their potential value in clinical practice. Key points • AI may potentially assist musculoskeletal radiologists in several interpretative tasks.• AI applications to trauma, age estimation, osteoarthritis, tumors, and orthopedic implants are discussed.• AI should help radiologists to optimize workflow and augment diagnostic performance.

Generative adversarial networks for spine imaging: A critical review of current applications

PURPOSE

In recent years, the field of medical imaging has witnessed remarkable advancements, with innovative technologies which revolutionized the visualization and analysis of the human spine. Among the groundbreaking developments in medical imaging, Generative Adversarial Networks (GANs) have emerged as a transformative tool, offering unprecedented possibilities in enhancing spinal imaging techniques and diagnostic outcomes. This review paper aims to provide a comprehensive overview of the use of GANs in spinal imaging, and to emphasize their potential to improve the diagnosis and treatment of spine-related disorders. A specific review focusing on Generative Adversarial Networks (GANs) in the context of medical spine imaging is needed to provide a comprehensive and specialized analysis of the unique challenges, applications, and advancements within this specific domain, which might not be fully addressed in broader reviews covering GANs in general medical imaging. Such a review can offer insights into the tailored solutions and innovations that GANs bring to the field of spinal medical imaging.

METHODS

An extensive literature search from 2017 until July 2023, was conducted using the most important search engines and identified studies that used GANs in spinal imaging.

RESULTS

The implementations include generating fat suppressed T2-weighted (fsT2W) images from T1 and T2-weighted sequences, to reduce scan time. The generated images had a significantly better image quality than true fsT2W images and could improve diagnostic accuracy for certain pathologies. GANs were also utilized in generating virtual thin-slice images of intervertebral spaces, creating digital twins of human vertebrae, and predicting fracture response. Lastly, they could be applied to convert CT to MRI images, with the potential to generate near-MR images from CT without MRI.

CONCLUSIONS

GANs have promising applications in personalized medicine, image augmentation, and improved diagnostic accuracy. However, limitations such as small databases and misalignment in CT-MRI pairs, must be considered.

Denoising diffusion-based MRI to CT image translation enables automated spinal segmentation

BACKGROUND

Automated segmentation of spinal magnetic resonance imaging (MRI) plays a vital role both scientifically and clinically. However, accurately delineating posterior spine structures is challenging.

METHODS

This retrospective study, approved by the ethical committee, involved translating T1-weighted and T2-weighted images into computed tomography (CT) images in a total of 263 pairs of CT/MR series. Landmark-based registration was performed to align image pairs. We compared two-dimensional (2D) paired - Pix2Pix, denoising diffusion implicit models (DDIM) image mode, DDIM noise mode - and unpaired (SynDiff, contrastive unpaired translation) image-to-image translation using "peak signal-to-noise ratio" as quality measure. A publicly available segmentation network segmented the synthesized CT datasets, and Dice similarity coefficients (DSC) were evaluated on in-house test sets and the "MRSpineSeg Challenge" volumes. The 2D findings were extended to three-dimensional (3D) Pix2Pix and DDIM.

RESULTS

2D paired methods and SynDiff exhibited similar translation performance and DCS on paired data. DDIM image mode achieved the highest image quality. SynDiff, Pix2Pix, and DDIM image mode demonstrated similar DSC (0.77). For craniocaudal axis rotations, at least two landmarks per vertebra were required for registration. The 3D translation outperformed the 2D approach, resulting in improved DSC (0.80) and anatomically accurate segmentations with higher spatial resolution than that of the original MRI series.

CONCLUSIONS

Two landmarks per vertebra registration enabled paired image-to-image translation from MRI to CT and outperformed all unpaired approaches. The 3D techniques provided anatomically correct segmentations, avoiding underprediction of small structures like the spinous process.

RELEVANCE STATEMENT

This study addresses the unresolved issue of translating spinal MRI to CT, making CT-based tools usable for MRI data. It generates whole spine segmentation, previously unavailable in MRI, a prerequisite for biomechanical modeling and feature extraction for clinical applications.

KEY POINTS

• Unpaired image translation lacks in converting spine MRI to CT effectively. • Paired translation needs registration with two landmarks per vertebra at least. • Paired image-to-image enables segmentation transfer to other domains. • 3D translation enables super resolution from MRI to CT. • 3D translation prevents underprediction of small structures.

[Deep learning method for magnetic resonance imaging fluid-attenuated inversion recovery image synthesis]

Magnetic resonance imaging(MRI) can obtain multi-modal images with different contrast, which provides rich information for clinical diagnosis. However, some contrast images are not scanned or the quality of the acquired images cannot meet the diagnostic requirements due to the difficulty of patient's cooperation or the limitation of scanning conditions. Image synthesis techniques have become a method to compensate for such image deficiencies. In recent years, deep learning has been widely used in the field of MRI synthesis. In this paper, a synthesis network based on multi-modal fusion is proposed, which firstly uses a feature encoder to encode the features of multiple unimodal images separately, and then fuses the features of different modal images through a feature fusion module, and finally generates the target modal image. The similarity measure between the target image and the predicted image in the network is improved by introducing a dynamic weighted combined loss function based on the spatial domain and K-space domain. After experimental validation and quantitative comparison, the multi-modal fusion deep learning network proposed in this paper can effectively synthesize high-quality MRI fluid-attenuated inversion recovery (FLAIR) images. In summary, the method proposed in this paper can reduce MRI scanning time of the patient, as well as solve the clinical problem of missing FLAIR images or image quality that is difficult to meet diagnostic requirements.

The promise and limitations of artificial intelligence in musculoskeletal imaging

With the recent developments in deep learning and the rapid growth of convolutional neural networks, artificial intelligence has shown promise as a tool that can transform several aspects of the musculoskeletal imaging cycle. Its applications can involve both interpretive and non-interpretive tasks such as the ordering of imaging, scheduling, protocoling, image acquisition, report generation and communication of findings. However, artificial intelligence tools still face a number of challenges that can hinder effective implementation into clinical practice. The purpose of this review is to explore both the successes and limitations of artificial intelligence applications throughout the muscuskeletal imaging cycle and to highlight how these applications can help enhance the service radiologists deliver to their patients, resulting in increased efficiency as well as improved patient and provider satisfaction.

Variational Autoencoders for Generating Synthetic Tractography-Based Bundle Templates in a Low-Data Setting

White matter tracts generated from whole brain tractography are often processed using automatic segmentation methods with standard atlases. Atlases are generated from hundreds of subjects, which becomes time-consuming to create and difficult to apply to all populations. In this study, we extended our prior work on using a deep generative model - a Convolutional Variational Autoencoder - to map complex and data-intensive streamlines to a low-dimensional latent space given a limited sample size of 50 subjects from the ADNI3 dataset, to generate synthetic population-specific bundle templates using Kernel Density Estimation (KDE) on streamline embeddings. We conducted a quantitative shape analysis by calculating bundle shape metrics, and found that our bundle templates better capture the shape distribution of the bundles than the atlas data used in the original segmentation derived from young healthy adults. We further demonstrated the use of our framework for direct bundle segmentation from whole-brain tractograms.

Improving accelerated 3D imaging in MRI-guided radiotherapy for prostate cancer using a deep learning method

PURPOSE

This study was to improve image quality for high-speed MR imaging using a deep learning method for online adaptive radiotherapy in prostate cancer. We then evaluated its benefits on image registration.

METHODS

Sixty pairs of 1.5 T MR images acquired with an MR-linac were enrolled. The data included low-speed, high-quality (LSHQ), and high-speed low-quality (HSLQ) MR images. We proposed a CycleGAN, which is based on the data augmentation technique, to learn the mapping between the HSLQ and LSHQ images and then generate synthetic LSHQ (synLSHQ) images from the HSLQ images. Five-fold cross-validation was employed to test the CycleGAN model. The normalized mean absolute error (nMAE), peak signal-to-noise ratio (PSNR), structural similarity index measurement (SSIM), and edge keeping index (EKI) were calculated to determine image quality. The Jacobian determinant value (JDV), Dice similarity coefficient (DSC), and mean distance to agreement (MDA) were used to analyze deformable registration.

RESULTS

Compared with the LSHQ, the proposed synLSHQ achieved comparable image quality and reduced imaging time by ~ 66%. Compared with the HSLQ, the synLSHQ had better image quality with improvement of 57%, 3.4%, 26.9%, and 3.6% for nMAE, SSIM, PSNR, and EKI, respectively. Furthermore, the synLSHQ enhanced registration accuracy with a superior mean JDV (6%) and preferable DSC and MDA values compared with HSLQ.

CONCLUSION

The proposed method can generate high-quality images from high-speed scanning sequences. As a result, it shows potential to shorten the scan time while ensuring the accuracy of radiotherapy.

Variational Autoencoders for Generating Synthetic Tractography-Based Bundle Templates in a Low-Data Setting

White matter tracts generated from whole brain tractography are often processed using automatic segmentation methods with standard atlases. Atlases are generated from hundreds of subjects, which becomes time-consuming to create and difficult to apply to all populations. In this study, we extended our prior work on using a deep generative model a Convolutional Variational Autoencoder - to map complex and data-intensive streamlines to a low-dimensional latent space given a limited sample size of 50 subjects from the ADNI3 dataset, to generate synthetic population-specific bundle templates using Kernel Density Estimation (KDE) on streamline embeddings. We conducted a quantitative shape analysis by calculating bundle shape metrics, and found that our bundle templates better capture the shape distribution of the bundles than the atlas data used in the original segmentation derived from young healthy adults. We further demonstrated the use of our framework for direct bundle segmentation from whole-brain tractograms.

A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II

Deep learning has the potential to be one of the most transformative technologies to impact orthopedic surgery. Substantial innovation in this area has occurred over the past 5 years, but clinically meaningful advancements remain limited by a disconnect between clinical and technical experts. That is, it is likely that few orthopedic surgeons possess both the clinical knowledge necessary to identify orthopedic problems, and the technical knowledge needed to implement deep learning-based solutions. To maximize the utilization of rapidly advancing technologies derived from deep learning models, orthopedic surgeons should understand the steps needed to design, organize, implement, and evaluate a deep learning project and its workflow. Equipping surgeons with this knowledge is the objective of this three-part editorial review. Part I described the processes involved in defining the problem, team building, data acquisition, curation, labeling, and establishing the ground truth. Building on that, this review (Part II) provides guidance on pre-processing and augmenting the data, making use of open-source libraries/toolkits, and selecting the required hardware to implement the pipeline. Special considerations regarding model training and evaluation unique to deep learning models relative to "shallow" machine learning models are also reviewed. Finally, guidance pertaining to the clinical deployment of deep learning models in the real world is provided. As in Part I, the focus is on applications of deep learning for computer vision and imaging.

Artificial intelligence and treatment algorithms in spine surgery

Artificial intelligence (AI) is a set of theories and techniques in which machines are used to simulate human intelligence with complex computer programs. The various machine learning (ML) methods are a subtype of AI. They originate from computer science and use algorithms established from analyzing a database to accomplish certain tasks. Among these methods are decision trees or random forests, support vector machines along with artificial neural networks. Convolutive neural networks were inspired from the visual cortex; they process combinations of information used in image or voice recognition. Deep learning (DL) groups together a set of ML methods and is useful for modeling complex relationships with a high degree of abstraction by using multiple layers of artificial neurons. ML techniques have a growing role in spine surgery. The main applications are the segmentation of intraoperative images for surgical navigation or robotics used for pedicle screw placement, the interpretation of images of intervertebral discs or full spine radiographs, which can be automated using ML algorithms. ML techniques can also be used as aids for surgical decision-making in complex fields, such as preoperative evaluation of adult spinal deformity. ML algorithms "learn" from large clinical databases. They make it possible to establish the intraoperative risk level and make a prognosis on how the postoperative functional scores will change over time as a function of the patient profile. These applications open a new path relative to standard statistical analyses. They make it possible to explore more complex relationships with multiple indirect interactions. In the future, AI algorithms could have a greater role in clinical research, evaluating clinical and surgical practices, and conducting health economics analyses.

Synthetic Contrasts in Musculoskeletal MRI: A Review

ABSTRACT

This review summarizes the existing techniques and methods used to generate synthetic contrasts from magnetic resonance imaging data focusing on musculoskeletal magnetic resonance imaging. To that end, the different approaches were categorized into 3 different methodological groups: mathematical image transformation, physics-based, and data-driven approaches. Each group is characterized, followed by examples and a brief overview of their clinical validation, if present. Finally, we will discuss the advantages, disadvantages, and caveats of synthetic contrasts, focusing on the preservation of image information, validation, and aspects of the clinical workflow.

Convolutional Neural Networks in Spinal Magnetic Resonance Imaging: A Systematic Review

OBJECTIVE

Convolutional neural networks (CNNs) are being increasingly used in the medical field, especially for image recognition in high-resolution, large-volume data sets. The study represents the current state of research on the application of CNNs in image segmentation and pathology detection in spine magnetic resonance imaging.

METHODS

For this systematic literature review, the authors performed a systematic initial search of the PubMed/Medline and Web of Science (Core collection) databases for eligible investigations. The authors limited the search to observational studies. Outcome parameters were analyzed according to the inclusion criteria and assigned to 3 groups: 1) segmentation of anatomical structures, 2) segmentation and evaluation of pathologic structures, and 3) specific implementation of CNNs.

RESULTS

Twenty-four retrospectively designed articles met the inclusion criteria. Publication dates ranged from 2017 to 2021. In total, 14,065 patients with 113,110 analyzed images were included. Most authors trained their network with a training-to-testing ratio of 80/20, while all but 2 articles used 5- to 10-fold cross-validation. Nine articles compared their performance results with other neural networks and algorithms, and all 24 articles described outcomes as positive.

CONCLUSIONS

State-of-the-art CNNs can detect and segment-specific anatomical landmarks and pathologies across a wide range, comparable to the skills of radiologists and experienced clinicians. With rapidly evolving network architectures and growing medical image databases, the future is likely to show growth in the development and refinement of these capable networks. However, the aid of automated segmentation and classification by neural networks cannot and should not be expected to replace clinical experts.

Artificial Intelligence in Spinal Imaging: Current Status and Future Directions

Spinal maladies are among the most common causes of pain and disability worldwide. Imaging represents an important diagnostic procedure in spinal care. Imaging investigations can provide information and insights that are not visible through ordinary visual inspection. Multiscale in vivo interrogation has the potential to improve the assessment and monitoring of pathologies thanks to the convergence of imaging, artificial intelligence (AI), and radiomic techniques. AI is revolutionizing computer vision, autonomous driving, natural language processing, and speech recognition. These revolutionary technologies are already impacting radiology, diagnostics, and other fields, where automated solutions can increase precision and reproducibility. In the first section of this narrative review, we provide a brief explanation of the many approaches currently being developed, with a particular emphasis on those employed in spinal imaging studies. The previously documented uses of AI for challenges involving spinal imaging, including imaging appropriateness and protocoling, image acquisition and reconstruction, image presentation, image interpretation, and quantitative image analysis, are then detailed. Finally, the future applications of AI to imaging of the spine are discussed. AI has the potential to significantly affect every step in spinal imaging. AI can make images of the spine more useful to patients and doctors by improving image quality, imaging efficiency, and diagnostic accuracy.

The stability of oncologic MRI radiomic features and the potential role of deep learning: a review

The use of MRI radiomic models for the diagnosis, prognosis and treatment response prediction of tumors has been increasingly reported in literature. However, its widespread adoption in clinics is hampered by issues related to features stability. In the MRI radiomic workflow, the main factors that affect radiomic features computation can be found in the image acquisition and reconstruction phase, in the image pre-processing steps, and in the segmentation of the region of interest on which radiomic indices are extracted. Deep Neural Networks (DNNs), having shown their potentiality in the medical image processing and analysis field, can be seen as an attractive strategy to partially overcome the issues related to radiomic stability and mitigate their impact. In fact, DNN approaches can be prospectively integrated in the MRI radiomic workflow to improve image quality, obtain accurate and reproducible segmentations and generate standardized images. In this review, DNN methods that can be included in the image processing steps of the radiomic workflow are described and discussed, in the light of a detailed analysis of the literature in the context of MRI radiomic reliability.

Artificial Intelligence Technologies in Neurosurgery: a Systematic Literature Review Using Topic Modeling. Part II: Research Objectives and Perspectives

The current increase in the number of publications on the use of artificial intelligence (AI) technologies in neurosurgery indicates a new trend in clinical neuroscience.

The aim of the study was to conduct a systematic literature review to highlight the main directions and trends in the use of AI in neurosurgery.

Deep generative models for automated muscle segmentation in computed tomography scanning

Accurate gluteus medius (GMd) volume evaluation may aid in the analysis of muscular atrophy states and help gain an improved understanding of patient recovery via rehabilitation. However, the segmentation of muscle regions in GMd images for cubic muscle volume assessment is time-consuming and labor-intensive. This study automated GMd-region segmentation from the computed tomography (CT) images of patients diagnosed with hip osteoarthritis using deep learning and evaluated the segmentation accuracy. To this end, 5250 augmented pairs of training data were obtained from five participants, and a conditional generative adversarial network was used to identify the relationships between the image pairs. Using the preserved test datasets, the results of automatic segmentation with the trained deep learning model were compared to those of manual segmentation in terms of the dice similarity coefficient (DSC), volume similarity (VS), and shape similarity (MS). As observed, the average DSC values for automatic and manual segmentations were 0.748 and 0.812, respectively, with a significant difference (p < 0.0001); the average VS values were 0.247 and 0.203, respectively, with no significant difference (p = 0.069); and the average MS values were 1.394 and 1.156, respectively, with no significant difference (p = 0.308). The GMd volumes obtained by automatic and manual segmentation were 246.2 cm³ and 282.9 cm³, respectively. The noninferiority of the DSC obtained by automatic segmentation was verified against that obtained by manual segmentation. Accordingly, the proposed GAN-based automatic GMd-segmentation technique is confirmed to be noninferior to manual segmentation. Therefore, the findings of this research confirm that the proposed method not only reduces time and effort but also facilitates accurate assessment of the cubic muscle volume.

Utility of machine learning algorithms in degenerative cervical and lumbar spine disease: a systematic review

Machine learning is a rapidly evolving field that offers physicians an innovative and comprehensive mechanism to examine various aspects of patient data. Cervical and lumbar degenerative spine disorders are commonly age-related disease processes that can utilize machine learning to improve patient outcomes with careful patient selection and intervention. The aim of this study is to examine the current applications of machine learning in cervical and lumbar degenerative spine disease. A systematic review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A search of PubMed, Embase, Medline, and Cochrane was conducted through May 31st, 2020, using the following terms: "artificial intelligence" OR "machine learning" AND "neurosurgery" AND "spine." Studies were included if original research on machine learning was utilized in patient care for degenerative spine disease, including radiographic machine learning applications. Studies focusing on robotic applications in neurosurgery, navigation, or stereotactic radiosurgery were excluded. The literature search identified 296 papers, with 35 articles meeting inclusion criteria. There were nine studies involving cervical degenerative spine disease and 26 studies on lumbar degenerative spine disease. The majority of studies for both cervical and lumbar spines utilized machine learning for the prediction of postoperative outcomes, with 5 (55.6%) and 15 (61.5%) studies, respectively. Machine learning applications focusing on degenerative lumbar spine greatly outnumber the current volume of cervical spine studies. The current research in lumbar spine also demonstrates more advanced clinical applications of radiographic, diagnostic, and predictive machine learning models.

Synthesizing Quantitative T2 Maps in Right Lateral Knee Femoral Condyles from Multicontrast Anatomic Data with a Conditional Generative Adversarial Network

PURPOSE

To develop a proof-of-concept convolutional neural network (CNN) to synthesize T2 maps in right lateral femoral condyle articular cartilage from anatomic MR images by using a conditional generative adversarial network (cGAN).

MATERIALS AND METHODS

In this retrospective study, anatomic images (from turbo spin-echo and double-echo in steady-state scans) of the right knee of 4621 patients included in the 2004-2006 Osteoarthritis Initiative were used as input to a cGAN-based CNN, and a predicted CNN T2 was generated as output. These patients included men and women of all ethnicities, aged 45-79 years, with or at high risk for knee osteoarthritis incidence or progression who were recruited at four separate centers in the United States. These data were split into 3703 (80%) for training, 462 (10%) for validation, and 456 (10%) for testing. Linear regression analysis was performed between the multiecho spin-echo (MESE) and CNN T2 in the test dataset. A more detailed analysis was performed in 30 randomly selected patients by means of evaluation by two musculoskeletal radiologists and quantification of cartilage subregions. Radiologist assessments were compared by using two-sided t tests.

RESULTS

The readers were moderately accurate in distinguishing CNN T2 from MESE T2, with one reader having random-chance categorization. CNN T2 values were correlated to the MESE values in the subregions of 30 patients and in the bulk analysis of all patients, with best-fit line slopes between 0.55 and 0.83.

CONCLUSION

With use of a neural network-based cGAN approach, it is feasible to synthesize T2 maps in femoral cartilage from anatomic MRI sequences, giving good agreement with MESE scans.See also commentary by Yi and Fritz in this issue.Keywords: Cartilage Imaging, Knee, Experimental Investigations, Quantification, Vision, Application Domain, Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms© RSNA, 2021.

Generating Virtual Short Tau Inversion Recovery (STIR) Images from T1- and T2-Weighted Images Using a Conditional Generative Adversarial Network in Spine Imaging

Short tau inversion recovery (STIR) sequences are frequently used in magnetic resonance imaging (MRI) of the spine. However, STIR sequences require a significant amount of scanning time. The purpose of the present study was to generate virtual STIR (vSTIR) images from non-contrast, non-fat-suppressed T1- and T2-weighted images using a conditional generative adversarial network (cGAN). The training dataset comprised 612 studies from 514 patients, and the validation dataset comprised 141 studies from 133 patients. For validation, 100 original STIR and respective vSTIR series were presented to six senior radiologists (blinded for the STIR type) in independent A/B-testing sessions. Additionally, for 141 real or vSTIR sequences, the testers were required to produce a structured report of 15 different findings. In the A/B-test, most testers could not reliably identify the real STIR (mean error of tester 1-6: 41%; 44%; 58%; 48%; 39%; 45%). In the evaluation of the structured reports, vSTIR was equivalent to real STIR in 13 of 15 categories. In the category of the number of STIR hyperintense vertebral bodies (p = 0.08) and in the diagnosis of bone metastases (p = 0.055), the vSTIR was only slightly insignificantly equivalent. By virtually generating STIR images of diagnostic quality from T1- and T2-weighted images using a cGAN, one can shorten examination times and increase throughput.

Deep Generative Adversarial Networks: Applications in Musculoskeletal Imaging

In recent years, deep learning techniques have been applied in musculoskeletal radiology to increase the diagnostic potential of acquired images. Generative adversarial networks (GANs), which are deep neural networks that can generate or transform images, have the potential to aid in faster imaging by generating images with a high level of realism across multiple contrast and modalities from existing imaging protocols. This review introduces the key architectures of GANs as well as their technical background and challenges. Key research trends are highlighted, including: (a) reconstruction of high-resolution MRI; (b) image synthesis with different modalities and contrasts; (c) image enhancement that efficiently preserves high-frequency information suitable for human interpretation; (d) pixel-level segmentation with annotation sharing between domains; and (e) applications to different musculoskeletal anatomies. In addition, an overview is provided of the key issues wherein clinical applicability is challenging to capture with conventional performance metrics and expert evaluation. When clinically validated, GANs have the potential to improve musculoskeletal imaging. Keywords: Adults and Pediatrics, Computer Aided Diagnosis (CAD), Computer Applications-General (Informatics), Informatics, Skeletal-Appendicular, Skeletal-Axial, Soft Tissues/Skin © RSNA, 2021.

Radiomic Machine Learning Classifiers in Spine Bone Tumors: A Multi-Software, Multi-Scanner Study

PURPOSE

Spinal lesion differential diagnosis remains challenging even in MRI. Radiomics and machine learning (ML) have proven useful even in absence of a standardized data mining pipeline. We aimed to assess ML diagnostic performance in spinal lesion differential diagnosis, employing radiomic data extracted by different software.

METHODS

Patients undergoing MRI for a vertebral lesion were retrospectively analyzed (n = 146, 67 males, 79 females; mean age 63 ± 16 years, range 8-89 years) and constituted the train (n = 100) and internal test cohorts (n = 46). Part of the latter had additional prior exams which constituted a multi-scanner, external test cohort (n = 35). Lesions were labeled as benign or malignant (2-label classification), and benign, primary malignant or metastases (3-label classification) for classification analyses. Features extracted via 3D Slicer heterogeneityCAD module (hCAD) and PyRadiomics were independently used to compare different combinations of feature selection methods and ML classifiers (n = 19).

RESULTS

In total, 90 and 1548 features were extracted by hCAD and PyRadiomics, respectively. The best feature selection method-ML algorithm combination was selected by 10 iterations of 10-fold cross-validation in the training data. For the 2-label classification ML obtained 94% accuracy in the internal test cohort, using hCAD data, and 86% in the external one. For the 3-label classification, PyRadiomics data allowed for 80% and 69% accuracy in the internal and external test sets, respectively.

CONCLUSIONS

MRI radiomics combined with ML may be useful in spinal lesion assessment. More robust pre-processing led to better consistency despite scanner and protocol heterogeneity.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

Automated interpretation of the coronary angioscopy with deep convolutional neural networks

BACKGROUND

Coronary angioscopy (CAS) is a useful modality to assess atherosclerotic changes, but interpretation of the images requires expert knowledge. Deep convolutional neural networks (DCNN) can be used for diagnostic prediction and image synthesis.

METHODS

107 images from 47 patients, who underwent CAS in our hospital between 2014 and 2017, and 864 images, selected from 142 MEDLINE-indexed articles published between 2000 and 2019, were analysed. First, we developed a prediction model for the angioscopic findings. Next, we made a generative adversarial networks (GAN) model to simulate the CAS images. Finally, we tried to control the output images according to the angioscopic findings with conditional GAN architecture.

RESULTS

For both yellow colour (YC) grade and neointimal coverage (NC) grade, we could observe strong correlations between the true grades and the predicted values (YC grade, average r=0.80±0.02, p<0.001; NC grade, average r=0.73±0.02, p<0.001). The binary classification model for the red thrombus yielded 0.71±0.03 F1-score and the area under the receiver operator characteristic curve was 0.91±0.02. The standard GAN model could generate realistic CAS images (average Inception score=3.57±0.06). GAN-based data augmentation improved the performance of the prediction models. In the conditional GAN model, there were significant correlations between given values and the expert's diagnosis in YC grade but not in NC grade.

CONCLUSION

DCNN is useful in both predictive and generative modelling that can help develop the diagnostic support system for CAS.

Current applications and future directions of deep learning in musculoskeletal radiology

Deep learning with convolutional neural networks (CNN) is a rapidly advancing subset of artificial intelligence that is ideally suited to solving image-based problems. There are an increasing number of musculoskeletal applications of deep learning, which can be conceptually divided into the categories of lesion detection, classification, segmentation, and non-interpretive tasks. Numerous examples of deep learning achieving expert-level performance in specific tasks in all four categories have been demonstrated in the past few years, although comprehensive interpretation of imaging examinations has not yet been achieved. It is important for the practicing musculoskeletal radiologist to understand the current scope of deep learning as it relates to musculoskeletal radiology. Interest in deep learning from researchers, radiology leadership, and industry continues to increase, and it is likely that these developments will impact the daily practice of musculoskeletal radiology in the near future.

[Radiomics-AI-based image analysis]

Radiomics deals with the statistical analysis of radiologic image data. In this article, radiomics is introduced and some of its applications are presented. In particular, an example is used to demonstrate that pathology and radiology can work together for better diagnoses. There is no denying that artificial intelligence will find its place in radiology (and pathology). Deep learning in particular will increasingly find applications. However, the impact on clinical routine is more long term and probably gradual, so AI will initially only be used in the form of specialized tools to support everyday clinical practice until methods and programs improve to the extent that AI can also take on more general diagnoses. However, this will not replace pathologists and radiologists in the long term, but rather turn them into "information specialists" who interpret the results obtained and integrate them into clinical contours.

Imaging in Spine Surgery: Current Concepts and Future Directions

OBJECTIVE

To review and highlight the historical and recent advances of imaging in spine surgery and to discuss current applications and future directions.

METHODS

A PubMed review of the current literature was performed on all relevant articles that examined historical and recent imaging techniques used in spine surgery. Studies were examined for their thoroughness in description of various modalities and applications in current and future management.

RESULTS

We reviewed 97 articles that discussed past, present, and future applications for imaging in spine surgery. Although most historical approaches relied heavily upon basic radiography, more recent advances have begun to expand upon advanced modalities, including the integration of more sophisticated equipment and artificial intelligence.

CONCLUSIONS

Since the days of conventional radiography, various modalities have emerged and become integral components of the spinal surgeon's diagnostic armamentarium. As such, it behooves the practitioner to remain informed on the current trends and potential developments in spinal imaging, as rapid adoption and interpretation of new techniques may make significant differences in patient management and outcomes. Future directions will likely become increasingly sophisticated as the implementation of machine learning, and artificial intelligence has become more commonplace in clinical practice.