Translating AI to Clinical Practice: Overcoming Data Shift with Explainability

To translate artificial intelligence (AI) algorithms into clinical practice requires generalizability of models to real-world data. One of the main obstacles to generalizability is data shift, a data distribution mismatch between model training and real environments. Explainable AI techniques offer tools to detect and mitigate the data shift problem and develop reliable AI for clinical practice. Most medical AI is trained with datasets gathered from limited environments, such as restricted disease populations and center-dependent acquisition conditions. The data shift that commonly exists in the limited training set often causes a significant performance decrease in the deployment environment. To develop a medical application, it is important to detect potential data shift and its impact on clinical translation. During AI training stages, from premodel analysis to in-model and post hoc explanations, explainability can play a key role in detecting model susceptibility to data shift, which is otherwise hidden because the test data have the same biased distribution as the training data. Performance-based model assessments cannot effectively distinguish the model overfitting to training data bias without enriched test sets from external environments. In the absence of such external data, explainability techniques can aid in translating AI to clinical practice as a tool to detect and mitigate potential failures due to data shift. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Building a high-resolution T2-weighted MR-based probabilistic model of tumor occurrence in the prostate

PURPOSE

We present a method for generating a T2 MR-based probabilistic model of tumor occurrence in the prostate to guide the selection of anatomical sites for targeted biopsies and serve as a diagnostic tool to aid radiological evaluation of prostate cancer.

MATERIALS AND METHODS

In our study, the prostate and any radiological findings within were segmented retrospectively on 3D T2-weighted MR images of 266 subjects who underwent radical prostatectomy. Subsequent histopathological analysis determined both the ground truth and the Gleason grade of the tumors. A randomly chosen subset of 19 subjects was used to generate a multi-subject-derived prostate template. Subsequently, a cascading registration algorithm involving both affine and non-rigid B-spline transforms was used to register the prostate of every subject to the template. Corresponding transformation of radiological findings yielded a population-based probabilistic model of tumor occurrence. The quality of our probabilistic model building approach was statistically evaluated by measuring the proportion of correct placements of tumors in the prostate template, i.e., the number of tumors that maintained their anatomical location within the prostate after their transformation into the prostate template space.

RESULTS

Probabilistic model built with tumors deemed clinically significant demonstrated a heterogeneous distribution of tumors, with higher likelihood of tumor occurrence at the mid-gland anterior transition zone and the base-to-mid-gland posterior peripheral zones. Of 250 MR lesions analyzed, 248 maintained their original anatomical location with respect to the prostate zones after transformation to the prostate.

CONCLUSION

We present a robust method for generating a probabilistic model of tumor occurrence in the prostate that could aid clinical decision making, such as selection of anatomical sites for MR-guided prostate biopsies.

Deep transfer learning for characterizing chondrocyte patterns in phase contrast X-Ray computed tomography images of the human patellar cartilage

Phase contrast X-ray computed tomography (PCI-CT) has been demonstrated to be effective for visualization of the human cartilage matrix at micrometer resolution, thereby capturing osteoarthritis induced changes to chondrocyte organization. This study aims to systematically assess the efficacy of deep transfer learning methods for classifying between healthy and diseased tissue patterns. We extracted features from two different convolutional neural network architectures, CaffeNet and Inception-v3 for characterizing such patterns. These features were quantitatively evaluated in a classification task measured by the area (AUC) under the Receiver Operating Characteristic (ROC) curve as well as qualitative visualization through a dimension reduction approach t-Distributed Stochastic Neighbor Embedding (t-SNE). The best classification performance, for CaffeNet, was observed when using features from the last convolutional layer and the last fully connected layer (AUCs >0.91). Meanwhile, off-the-shelf features from Inception-v3 produced similar classification performance (AUC >0.95). Visualization of features from these layers further confirmed adequate characterization of chondrocyte patterns for reliably distinguishing between healthy and osteoarthritic tissue classes. Such techniques, can be potentially used for detecting the presence of osteoarthritis related changes in the human patellar cartilage.

Alteration of brain network topology in HIV-associated neurocognitive disorder: A novel functional connectivity perspective

HIV is capable of invading the brain soon after seroconversion. This ultimately can lead to deficits in multiple cognitive domains commonly referred to as HIV-associated neurocognitive disorders (HAND). Clinical diagnosis of such deficits requires detailed neuropsychological assessment but clinical signs may be difficult to detect during asymptomatic injury of the central nervous system (CNS). Therefore neuroimaging biomarkers are of particular interest in HAND. In this study, we constructed brain connectivity profiles of 40 subjects (20 HIV positive subjects and 20 age-matched seronegative controls) using two different methods: a non-linear mutual connectivity analysis approach and a conventional method based on Pearson's correlation. These profiles were then summarized using graph-theoretic methods characterizing their topological network properties. Standard clinical and laboratory assessments were performed and a battery of neuropsychological (NP) tests was administered for all participating subjects. Based on NP testing, 14 of the seropositive subjects exhibited mild neurologic impairment. Subsequently, we analyzed associations between the network derived measures and neuropsychological assessment scores as well as common clinical laboratory plasma markers (CD4 cell count, HIV RNA) after adjusting for age and gender. Mutual connectivity analysis derived graph-theoretic measures, Modularity and Small Worldness, were significantly (p < 0.05, FDR adjusted) associated with the Executive as well as Overall z-score of NP performance. In contrast, network measures derived from conventional correlation-based connectivity did not yield any significant results. Thus, changes in connectivity can be captured using advanced time-series analysis techniques. The demonstrated associations between imaging-derived graph-theoretic properties of brain networks with neuropsychological performance, provides opportunities to further investigate the evolution of HAND in larger, longitudinal studies. Our analysis approach, involving non-linear time-series analysis in conjunction with graph theory, is promising and it may prove to be useful not only in HAND but also in other neurodegenerative disorders.

•

Currently, cognitive impairment in HIV positive individuals is detected using detailed neuropsychological testing.

•

Analysis of fMRI data using MCA-GRBF method revealed significant associations with current clinical standards.

•

In contrast, functional connectivity analysis using conventional correlation analysis does not produce any such associations.

•

Nonlinear analysis using MCA-GRBF method can potentially capture relevant information when compared to conventional methods.

Mutual Connectivity Analysis (MCA) Using Generalized Radial Basis Function Neural Networks for Nonlinear Functional Connectivity Network Recovery in Resting-State Functional MRI

We investigate the applicability of a computational framework, called mutual connectivity analysis (MCA), for directed functional connectivity analysis in both synthetic and resting-state functional MRI data. This framework comprises of first evaluating non-linear cross-predictability between every pair of time series prior to recovering the underlying network structure using community detection algorithms. We obtain the non-linear cross-prediction score between time series using Generalized Radial Basis Functions (GRBF) neural networks. These cross-prediction scores characterize the underlying functionally connected networks within the resting brain, which can be extracted using non-metric clustering approaches, such as the Louvain method. We first test our approach on synthetic models with known directional influence and network structure. Our method is able to capture the directional relationships between time series (with an area under the ROC curve = 0.92 ± 0.037) as well as the underlying network structure (Rand index = 0.87 ± 0.063) with high accuracy. Furthermore, we test this method for network recovery on resting-state fMRI data, where results are compared to the motor cortex network recovered from a motor stimulation sequence, resulting in a strong agreement between the two (Dice coefficient = 0.45). We conclude that our MCA approach is effective in analyzing non-linear directed functional connectivity and in revealing underlying functional network structure in complex systems.

Detecting Altered connectivity patterns in HIV associated neurocognitive impairment using Mutual Connectivity Analysis

The use of functional Magnetic Resonance Imaging (fMRI) has provided interesting insights into our understanding of the brain. In clinical setups these scans have been used to detect and study changes in the brain network properties in various neurological disorders. A large percentage of subjects infected with HIV present cognitive deficits, which are known as HIV associated neurocognitive disorder (HAND). In this study we propose to use our novel technique named Mutual Connectivity Analysis (MCA) to detect differences in brain networks in subjects with and without HIV infection. Resting state functional MRI scans acquired from 10 subjects (5 HIV+ and 5 HIV-) were subject to standard pre-processing routines. Subsequently, the average time-series for each brain region of the Automated Anatomic Labeling (AAL) atlas are extracted and used with the MCA framework to obtain a graph characterizing the interactions between them. The network graphs obtained for different subjects are then compared using Network-Based Statistics (NBS), which is an approach to detect differences between graphs edges while controlling for the family-wise error rate when mass univariate testing is performed. Applying this approach on the graphs obtained yields a single network encompassing 42 nodes and 65 edges, which is significantly different between the two subject groups. Specifically connections to the regions in and around the basal ganglia are significantly decreased. Also some nodes corresponding to the posterior cingulate cortex are affected. These results are inline with our current understanding of pathophysiological mechanisms of HIV associated neurocognitive disease (HAND) and other HIV based fMRI connectivity studies. Hence, we illustrate the applicability of our novel approach with network-based statistics in a clinical case-control study to detect differences connectivity patterns.

Investigating Changes in Brain Network Properties in HIV-Associated Neurocognitive Disease (HAND) using Mutual Connectivity Analysis (MCA)

About 50% of subjects infected with HIV present deficits in cognitive domains, which are known collectively as HIV associated neurocognitive disorder (HAND). The underlying synaptodendritic damage can be captured using resting state functional MRI, as has been demonstrated by a few earlier studies. Such damage may induce topological changes of brain connectivity networks. We test this hypothesis by capturing the functional interdependence of 90 brain network nodes using a Mutual Connectivity Analysis (MCA) framework with non-linear time series modeling based on Generalized Radial Basis function (GRBF) neural networks. The network nodes are selected based on the regions defined in the Automated Anatomic Labeling (AAL) atlas. Each node is represented by the average time series of the voxels of that region. The resulting networks are then characterized using graph-theoretic measures that quantify various network topology properties at a global as well as at a local level. We tested for differences in these properties in network graphs obtained for 10 subjects (6 male and 4 female, 5 HIV+ and 5 HIV-). Global network properties captured some differences between these subject cohorts, though significant differences were seen only with the clustering coefficient measure. Local network properties, such as local efficiency and the degree of connections, captured significant differences in regions of the frontal lobe, precentral and cingulate cortex amongst a few others. These results suggest that our method can be used to effectively capture differences occurring in brain network connectivity properties revealed by resting-state functional MRI in neurological disease states, such as HAND.

Assessing vertebral fracture risk on volumetric quantitative computed tomography by geometric characterization of trabecular bone structure

The current clinical standard for measuring Bone Mineral Density (BMD) is dual X-ray absorptiometry, however more recently BMD derived from volumetric quantitative computed tomography has been shown to demonstrate a high association with spinal fracture susceptibility. In this study, we propose a method of fracture risk assessment using structural properties of trabecular bone in spinal vertebrae. Experimental data was acquired via axial multi-detector CT (MDCT) from 12 spinal vertebrae specimens using a whole-body 256-row CT scanner with a dedicated calibration phantom. Common image processing methods were used to annotate the trabecular compartment in the vertebral slices creating a circular region of interest (ROI) that excluded cortical bone for each slice. The pixels inside the ROI were converted to values indicative of BMD. High dimensional geometrical features were derived using the scaling index method (SIM) at different radii and scaling factors (SF). The mean BMD values within the ROI were then extracted and used in conjunction with a support vector machine to predict the failure load of the specimens. Prediction performance was measured using the root-mean-square error (RMSE) metric and determined that SIM combined with mean BMD features (RMSE = 0.82 ± 0.37) outperformed MDCT-measured mean BMD (RMSE = 1.11 ± 0.33) (p < 10^-4). These results demonstrate that biomechanical strength prediction in vertebrae can be significantly improved through the use of SIM-derived texture features from trabecular bone.

Volumetric quantitative characterization of human patellar cartilage with topological and geometrical features on phase-contrast X-ray computed tomography

Phase-contrast X-ray computed tomography (PCI-CT) has attracted significant interest in recent years for its ability to provide significantly improved image contrast in low absorbing materials such as soft biological tissue. In the research context of cartilage imaging, previous studies have demonstrated the ability of PCI-CT to visualize structural details of human patellar cartilage matrix and capture changes to chondrocyte organization induced by osteoarthritis. This study evaluates the use of geometrical and topological features for volumetric characterization of such chondrocyte patterns in the presence (or absence) of osteoarthritic damage. Geometrical features derived from the scaling index method (SIM) and topological features derived from Minkowski Functionals were extracted from 1392 volumes of interest (VOI) annotated on PCI-CT images of ex vivo human patellar cartilage specimens. These features were subsequently used in a machine learning task with support vector regression to classify VOIs as healthy or osteoarthritic; classification performance was evaluated using the area under the receiver operating characteristic curve (AUC). Our results show that the classification performance of SIM-derived geometrical features (AUC: 0.90 ± 0.09) is significantly better than Minkowski Functionals volume (AUC: 0.54 ± 0.02), surface (AUC: 0.72 ± 0.06), mean breadth (AUC: 0.74 ± 0.06) and Euler characteristic (AUC: 0.78 ± 0.04) (p < 10(-4)). These results suggest that such geometrical features can provide a detailed characterization of the chondrocyte organization in the cartilage matrix in an automated manner, while also enabling classification of cartilage as healthy or osteoarthritic with high accuracy. Such features could potentially serve as diagnostic imaging markers for evaluating osteoarthritis progression and its response to different therapeutic intervention strategies.

Volumetric Characterization of Human Patellar Cartilage Matrix on Phase Contrast X-Ray Computed Tomography

Phase contrast X-ray computed tomography (PCI-CT) has recently emerged as a novel imaging technique that allows visualization of cartilage soft tissue, subsequent examination of chondrocyte patterns, and their correlation to osteoarthritis. Previous studies have shown that 2D texture features are effective at distinguishing between healthy and osteoarthritic regions of interest annotated in the radial zone of cartilage matrix on PCI-CT images. In this study, we further extend the texture analysis to 3D and investigate the ability of volumetric texture features at characterizing chondrocyte patterns in the cartilage matrix for purposes of classification. Here, we extracted volumetric texture features derived from Minkowski Functionals and gray-level co-occurrence matrices (GLCM) from 496 volumes of interest (VOI) annotated on PCI-CT images of human patellar cartilage specimens. The extracted features were then used in a machine-learning task involving support vector regression to classify ROIs as healthy or osteoarthritic. Classification performance was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC). The best classification performance was observed with GLCM features correlation (AUC = 0.83 ± 0.06) and homogeneity (AUC = 0.82 ± 0.07), which significantly outperformed all Minkowski Functionals (p < 0.05). These results suggest that such quantitative analysis of chondrocyte patterns in human patellar cartilage matrix involving GLCM-derived statistical features can distinguish between healthy and osteoarthritic tissue with high accuracy.

Investigating the use of mutual information and non-metric clustering for functional connectivity analysis on resting-state functional MRI

Functional MRI (fMRI) is currently used to investigate structural and functional connectivity in human brain networks. To this end, previous studies have proposed computational methods that involve assumptions that can induce information loss, such as assumed linear coupling of the fMRI signals or requiring dimension reduction. This study presents a new computational framework for investigating the functional connectivity in the brain and recovering network structure while reducing the information loss inherent in previous methods. For this purpose, pair-wise mutual information (MI) was extracted from all pixel time series within the brain on resting-state fMRI data. Non-metric topographic mapping of proximity (TMP) data was subsequently applied to recover network structure from the pair-wise MI analysis. Our computational framework is demonstrated in the task of identifying regions of the primary motor cortex network on resting state fMRI data. For ground truth comparison, we also localized regions of the primary motor cortex associated with hand movement in a task-based fMRI sequence with a finger-tapping stimulus function. The similarity between our pair-wise MI clustering results and the ground truth is evaluated using the dice coefficient. Our results show that non-metric clustering with the TMP algorithm, as performed on pair-wise MI analysis, was able to detect the primary motor cortex network and achieved a dice coefficient of 0.53 in terms of overlap with the ground truth. Thus, we conclude that our computational framework can extract and visualize valuable information concerning the underlying network structure between different regions of the brain in resting state fMRI.

Characterizing Trabecular Bone structure for Assessing Vertebral Fracture Risk on Volumetric Quantitative Computed Tomography

While the proximal femur is preferred for measuring bone mineral density (BMD) in fracture risk estimation, the introduction of volumetric quantitative computed tomography has revealed stronger associations between BMD and spinal fracture status. In this study, we propose to capture properties of trabecular bone structure in spinal vertebrae with advanced second-order statistical features for purposes of fracture risk assessment. For this purpose, axial multi-detector CT (MDCT) images were acquired from 28 spinal vertebrae specimens using a whole-body 256-row CT scanner with a dedicated calibration phantom. A semi-automated method was used to annotate the trabecular compartment in the central vertebral slice with a circular region of interest (ROI) to exclude cortical bone; pixels within were converted to values indicative of BMD. Six second-order statistical features derived from gray-level co-occurrence matrices (GLCM) and the mean BMD within the ROI were then extracted and used in conjunction with a generalized radial basis functions (GRBF) neural network to predict the failure load of the specimens; true failure load was measured through biomechanical testing. Prediction performance was evaluated with a root-mean-square error (RMSE) metric. The best prediction performance was observed with GLCM feature 'correlation' (RMSE = 1.02 ± 0.18), which significantly outperformed all other GLCM features (p < 0.01). GLCM feature correlation also significantly outperformed MDCT-measured mean BMD (RMSE = 1.11 ± 0.17) (p < 10^-4). These results suggest that biomechanical strength prediction in spinal vertebrae can be significantly improved through characterization of trabecular bone structure with GLCM-derived texture features.

Functional Connectivity Analysis in Resting State fMRI with Echo-State Networks and Non-Metric Clustering for Network Structure Recovery

Echo state networks (ESN) are recurrent neural networks where the hidden layer is replaced with a fixed reservoir of neurons. Unlike feed-forward networks, neuron training in ESN is restricted to the output neurons alone thereby providing a computational advantage. We demonstrate the use of such ESNs in our mutual connectivity analysis (MCA) framework for recovering the primary motor cortex network associated with hand movement from resting state functional MRI (fMRI) data. Such a framework consists of two steps - (1) defining a pair-wise affinity matrix between different pixel time series within the brain to characterize network activity and (2) recovering network components from the affinity matrix with non-metric clustering. Here, ESNs are used to evaluate pair-wise cross-estimation performance between pixel time series to create the affinity matrix, which is subsequently subject to non-metric clustering with the Louvain method. For comparison, the ground truth of the motor cortex network structure is established with a task-based fMRI sequence. Overlap between the primary motor cortex network recovered with our model free MCA approach and the ground truth was measured with the Dice coefficient. Our results show that network recovery with our proposed MCA approach is in close agreement with the ground truth. Such network recovery is achieved without requiring low-pass filtering of the time series ensembles prior to analysis, an fMRI preprocessing step that has courted controversy in recent years. Thus, we conclude our MCA framework can allow recovery and visualization of the underlying functionally connected networks in the brain on resting state fMRI.

Integrating dimension reduction and out-of-sample extension in automated classification of ex vivo human patellar cartilage on phase contrast X-ray computed tomography

Phase contrast X-ray computed tomography (PCI-CT) has been demonstrated as a novel imaging technique that can visualize human cartilage with high spatial resolution and soft tissue contrast. Different textural approaches have been previously investigated for characterizing chondrocyte organization on PCI-CT to enable classification of healthy and osteoarthritic cartilage. However, the large size of feature sets extracted in such studies motivates an investigation into algorithmic feature reduction for computing efficient feature representations without compromising their discriminatory power. For this purpose, geometrical feature sets derived from the scaling index method (SIM) were extracted from 1392 volumes of interest (VOI) annotated on PCI-CT images of ex vivo human patellar cartilage specimens. The extracted feature sets were subject to linear and non-linear dimension reduction techniques as well as feature selection based on evaluation of mutual information criteria. The reduced feature set was subsequently used in a machine learning task with support vector regression to classify VOIs as healthy or osteoarthritic; classification performance was evaluated using the area under the receiver-operating characteristic (ROC) curve (AUC). Our results show that the classification performance achieved by 9-D SIM-derived geometric feature sets (AUC: 0.96 ± 0.02) can be maintained with 2-D representations computed from both dimension reduction and feature selection (AUC values as high as 0.97 ± 0.02). Thus, such feature reduction techniques can offer a high degree of compaction to large feature sets extracted from PCI-CT images while maintaining their ability to characterize the underlying chondrocyte patterns.

Nonlinear Functional Connectivity Network Recovery in the Human Brain with Mutual Connectivity Analysis (MCA): Convergent Cross-Mapping and Non-Metric Clustering

We explore a computational framework for functional connectivity analysis in resting-state functional MRI (fMRI) data acquired from the human brain for recovering the underlying network structure and understanding causality between network components. Termed mutual connectivity analysis (MCA), this framework involves two steps, the first of which is to evaluate the pair-wise cross-prediction performance between fMRI pixel time series within the brain. In a second step, the underlying network structure is subsequently recovered from the affinity matrix using non-metric network clustering approaches, such as the so-called Louvain method. Finally, we use convergent cross-mapping (CCM) to study causality between different network components. We demonstrate our MCA framework in the problem of recovering the motor cortex network associated with hand movement from resting state fMRI data. Results are compared with a ground truth of active motor cortex regions as identified by a task-based fMRI sequence involving a finger-tapping stimulation experiment. Our results regarding causation between regions of the motor cortex revealed a significant directional variability and were not readily interpretable in a consistent manner across subjects. However, our results on whole-slice fMRI analysis demonstrate that MCA-based model-free recovery of regions associated with the primary motor cortex and supplementary motor area are in close agreement with localization of similar regions achieved with a task-based fMRI acquisition. Thus, we conclude that our MCA methodology can extract and visualize valuable information concerning the underlying network structure between different regions of the brain in resting state fMRI.

Computer-aided diagnosis for phase-contrast X-ray computed tomography: quantitative characterization of human patellar cartilage with high-dimensional geometric features

Phase-contrast computed tomography (PCI-CT) has shown tremendous potential as an imaging modality for visualizing human cartilage with high spatial resolution. Previous studies have demonstrated the ability of PCI-CT to visualize (1) structural details of the human patellar cartilage matrix and (2) changes to chondrocyte organization induced by osteoarthritis. This study investigates the use of high-dimensional geometric features in characterizing such chondrocyte patterns in the presence or absence of osteoarthritic damage. Geometrical features derived from the scaling index method (SIM) and statistical features derived from gray-level co-occurrence matrices were extracted from 842 regions of interest (ROI) annotated on PCI-CT images of ex vivo human patellar cartilage specimens. These features were subsequently used in a machine learning task with support vector regression to classify ROIs as healthy or osteoarthritic; classification performance was evaluated using the area under the receiver-operating characteristic curve (AUC). SIM-derived geometrical features exhibited the best classification performance (AUC, 0.95 ± 0.06) and were most robust to changes in ROI size. These results suggest that such geometrical features can provide a detailed characterization of the chondrocyte organization in the cartilage matrix in an automated and non-subjective manner, while also enabling classification of cartilage as healthy or osteoarthritic with high accuracy. Such features could potentially serve as imaging markers for evaluating osteoarthritis progression and its response to different therapeutic intervention strategies.

Using Anisotropic 3D Minkowski Functionals for Trabecular Bone Characterization and Biomechanical Strength Prediction in Proximal Femur Specimens

The ability of Anisotropic Minkowski Functionals (AMFs) to capture local anisotropy while evaluating topological properties of the underlying gray-level structures has been previously demonstrated. We evaluate the ability of this approach to characterize local structure properties of trabecular bone micro-architecture in ex vivo proximal femur specimens, as visualized on multi-detector CT, for purposes of biomechanical bone strength prediction. To this end, volumetric AMFs were computed locally for each voxel of volumes of interest (VOI) extracted from the femoral head of 146 specimens. The local anisotropy captured by such AMFs was quantified using a fractional anisotropy measure; the magnitude and direction of anisotropy at every pixel was stored in histograms that served as a feature vectors that characterized the VOIs. A linear multi-regression analysis algorithm was used to predict the failure load (FL) from the feature sets; the predicted FL was compared to the true FL determined through biomechanical testing. The prediction performance was measured by the root mean square error (RMSE) for each feature set. The best prediction performance was obtained from the fractional anisotropy histogram of AMF Euler Characteristic (RMSE = 1.01 ± 0.13), which was significantly better than MDCT-derived mean BMD (RMSE = 1.12 ± 0.16, p<0.05). We conclude that such anisotropic Minkowski Functionals can capture valuable information regarding regional trabecular bone quality and contribute to improved bone strength prediction, which is important for improving the clinical assessment of osteoporotic fracture risk.

Investigating the use of texture features for analysis of breast lesions on contrast-enhanced cone beam CT

Cone beam computed tomography (CBCT) has found use in mammography for imaging the entire breast with sufficient spatial resolution at a radiation dose within the range of that of conventional mammography. Recently, enhancement of lesion tissue through the use of contrast agents has been proposed for cone beam CT. This study investigates whether the use of such contrast agents improves the ability of texture features to differentiate lesion texture from healthy tissue on CBCT in an automated manner. For this purpose, 9 lesions were annotated by an experienced radiologist on both regular and contrast-enhanced CBCT images using two-dimensional (2D) square ROIs. These lesions were then segmented, and each pixel within the lesion ROI was assigned a label - lesion or non-lesion, based on the segmentation mask. On both sets of CBCT images, four three-dimensional (3D) Minkowski Functionals were used to characterize the local topology at each pixel. The resulting feature vectors were then used in a machine learning task involving support vector regression with a linear kernel (SVRlin) to classify each pixel as belonging to the lesion or non-lesion region of the ROI. Classification performance was assessed using the area under the receiver-operating characteristic (ROC) curve (AUC). Minkowski Functionals derived from contrast-enhanced CBCT images were found to exhibit significantly better performance at distinguishing between lesion and non-lesion areas within the ROI when compared to those extracted from CBCT images without contrast enhancement (p < 0.05). Thus, contrast enhancement in CBCT can improve the ability of texture features to distinguish lesions from surrounding healthy tissue.

Predicting the Biomechanical Strength of Proximal Femur Specimens with Minkowski Functionals and Support Vector Regression

Regional trabecular bone quality estimation for purposes of femoral bone strength prediction is important for improving the clinical assessment of osteoporotic fracture risk. In this study, we explore the ability of 3D Minkowski Functionals derived from multi-detector computed tomography (MDCT) images of proximal femur specimens in predicting their corresponding biomechanical strength. MDCT scans were acquired for 50 proximal femur specimens harvested from human cadavers. An automated volume of interest (VOI)-fitting algorithm was used to define a consistent volume in the femoral head of each specimen. In these VOIs, the trabecular bone micro-architecture was characterized by statistical moments of its BMD distribution and by topological features derived from Minkowski Functionals. A linear multi-regression analysis and a support vector regression (SVR) algorithm with a linear kernel were used to predict the failure load (FL) from the feature sets; the predicted FL was compared to the true FL determined through biomechanical testing. The prediction performance was measured by the root mean square error (RMSE) for each feature set. The best prediction result was obtained from the Minkowski Functional surface used in combination with SVR, which had the lowest prediction error (RMSE = 0.939 ± 0.345) and which was significantly lower than mean BMD (RMSE = 1.075 ± 0.279, p<0.005). Our results indicate that the biomechanical strength prediction can be significantly improved in proximal femur specimens with Minkowski Functionals extracted from on MDCT images used in conjunction with support vector regression.

Pair-wise Clustering of Large Scale Granger Causality Index Matrices for Revealing Communities

The analysis of large ensembles of time series is a fundamental challenge in different domains of biomedical image processing applications, specifically in the area of functional MRI data processing. An important aspect of such analysis is the ability to reconstruct community network structures based on interactive behavior between different nodes of the network which are captured in such time series. In this study, we start with a previously proposed novel approach that applies the linear Granger Causality concept to very high-dimensional time series. This approach is based on integrating dimensionality reduction into a multivariate time series model. If residuals of dimensionality reduced models can be transformed back into the original space, prediction errors in the high-dimensional space may be computed, and a large scale Granger Causality Index (lsGCI) is properly defined. The primary goal of this study was then to present an approach for recovering network structure from such lsGCI interactions through the application of pair-wise clustering. We specifically focus on a clustering approach based on topographic mapping of proximity data (TMP) for this purpose. We demonstrate our approach with a simulated network composed of five pair-wise different internal networks with varying strengths of community structure (based on the number of inter-network vertices). Our results suggest that such pair-wise clustering with TMP is capable of reconstructing the structure of the original network from lsGCI matrices that record the interactions between different nodes of the network when there is sufficient disparity between the intra- and inter-network vertices.

Improving bone strength prediction in human proximal femur specimens through geometrical characterization of trabecular bone microarchitecture and support vector regression

We investigate the use of different trabecular bone descriptors and advanced machine learning tech niques to complement standard bone mineral density (BMD) measures derived from dual-energy x-ray absorptiometry (DXA) for improving clinical assessment of osteoporotic fracture risk. For this purpose, volumes of interest were extracted from the head, neck, and trochanter of 146 ex vivo proximal femur specimens on multidetector computer tomography. The trabecular bone captured was characterized with (1) statistical moments of the BMD distribution, (2) geometrical features derived from the scaling index method (SIM), and (3) morphometric parameters, such as bone fraction, trabecular thickness, etc. Feature sets comprising DXA BMD and such supplemental features were used to predict the failure load (FL) of the specimens, previously determined through biomechanical testing, with multiregression and support vector regression. Prediction performance was measured by the root mean square error (RMSE); correlation with measured FL was evaluated using the coefficient of determination R2. The best prediction performance was achieved by a combination of DXA BMD and SIM-derived geometric features derived from the femoral head (RMSE: 0.869 ± 0.121, R2: 0.68 ± 0.079), which was significantly better than DXA BMD alone (RMSE: 0.948 ± 0.119, R2: 0.61 ± 0.101) (p < 10-4). For multivariate feature sets, SVR outperformed multiregression (p < 0.05). These results suggest that supplementing standard DXA BMD measurements with sophisticated femoral trabecular bone characterization and supervised learning techniques can significantly improve biomechanical strength prediction in proximal femur specimens.

Phase contrast imaging X-ray computed tomography: Quantitative characterization of human patellar cartilage matrix with topological and geometrical features

Current assessment of cartilage is primarily based on identification of indirect markers such as joint space narrowing and increased subchondral bone density on x-ray images. In this context, phase contrast CT imaging (PCI-CT) has recently emerged as a novel imaging technique that allows a direct examination of chondrocyte patterns and their correlation to osteoarthritis through visualization of cartilage soft tissue. This study investigates the use of topological and geometrical approaches for characterizing chondrocyte patterns in the radial zone of the knee cartilage matrix in the presence and absence of osteoarthritic damage. For this purpose, topological features derived from Minkowski Functionals and geometric features derived from the Scaling Index Method (SIM) were extracted from 842 regions of interest (ROI) annotated on PCI-CT images of healthy and osteoarthritic specimens of human patellar cartilage. The extracted features were then used in a machine learning task involving support vector regression to classify ROIs as healthy or osteoarthritic. Classification performance was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC). The best classification performance was observed with high-dimensional geometrical feature vectors derived from SIM (0.95 ± 0.06) which outperformed all Minkowski Functionals (p < 0.001). These results suggest that such quantitative analysis of chondrocyte patterns in human patellar cartilage matrix involving SIM-derived geometrical features can distinguish between healthy and osteoarthritic tissue with high accuracy.

Classification of small lesions in dynamic breast MRI: Eliminating the need for precise lesion segmentation through spatio-temporal analysis of contrast enhancement over time

Characterizing the dignity of breast lesions as benign or malignant is specifically difficult for small lesions; they don't exhibit typical characteristics of malignancy and are harder to segment since margins are harder to visualize. Previous attempts at using dynamic or morphologic criteria to classify small lesions (mean lesion diameter of about 1 cm) have not yielded satisfactory results. The goal of this work was to improve the classification performance in such small diagnostically challenging lesions while concurrently eliminating the need for precise lesion segmentation. To this end, we introduce a method for topological characterization of lesion enhancement patterns over time. Three Minkowski Functionals were extracted from all five post-contrast images of sixty annotated lesions on dynamic breast MRI exams. For each Minkowski Functional, topological features extracted from each post-contrast image of the lesions were combined into a high-dimensional texture feature vector. These feature vectors were classified in a machine learning task with support vector regression. For comparison, conventional Haralick texture features derived from gray-level co-occurrence matrices (GLCM) were also used. A new method for extracting thresholded GLCM features was also introduced and investigated here. The best classification performance was observed with Minkowski Functionals area and perimeter, thresholded GLCM features f8 and f9, and conventional GLCM features f4 and f6. However, both Minkowski Functionals and thresholded GLCM achieved such results without lesion segmentation while the performance of GLCM features significantly deteriorated when lesions were not segmented (p < 0.05). This suggests that such advanced spatio-temporal characterization can improve the classification performance achieved in such small lesions, while simultaneously eliminating the need for precise segmentation.

Computer-aided diagnosis in phase contrast imaging X-ray computed tomography for quantitative characterization of ex vivo human patellar cartilage

Visualization of ex vivo human patellar cartilage matrix through the phase contrast imaging X-ray computed tomography (PCI-CT) has been previously demonstrated. Such studies revealed osteoarthritis-induced changes to chondrocyte organization in the radial zone. This study investigates the application of texture analysis to characterizing such chondrocyte patterns in the presence and absence of osteoarthritic damage. Texture features derived from Minkowski functionals (MF) and gray-level co-occurrence matrices (GLCM) were extracted from 842 regions of interest (ROI) annotated on PCI-CT images of ex vivo human patellar cartilage specimens. These texture features were subsequently used in a machine learning task with support vector regression to classify ROIs as healthy or osteoarthritic; classification performance was evaluated using the area under the receiver operating characteristic curve (AUC). The best classification performance was observed with the MF features perimeter (AUC: 0.94 ±0.08 ) and "Euler characteristic" (AUC: 0.94 ±0.07 ), and GLCM-derived feature "Correlation" (AUC: 0.93 ±0.07). These results suggest that such texture features can provide a detailed characterization of the chondrocyte organization in the cartilage matrix, enabling classification of cartilage as healthy or osteoarthritic with high accuracy.

Introducing Anisotropic Minkowski Functionals and Quantitative Anisotropy Measures for Local Structure Analysis in Biomedical Imaging

The ability of Minkowski Functionals to characterize local structure in different biological tissue types has been demonstrated in a variety of medical image processing tasks. We introduce anisotropic Minkowski Functionals (AMFs) as a novel variant that captures the inherent anisotropy of the underlying gray-level structures. To quantify the anisotropy characterized by our approach, we further introduce a method to compute a quantitative measure motivated by a technique utilized in MR diffusion tensor imaging, namely fractional anisotropy. We showcase the applicability of our method in the research context of characterizing the local structure properties of trabecular bone micro-architecture in the proximal femur as visualized on multi-detector CT. To this end, AMFs were computed locally for each pixel of ROIs extracted from the head, neck and trochanter regions. Fractional anisotropy was then used to quantify the local anisotropy of the trabecular structures found in these ROIs and to compare its distribution in different anatomical regions. Our results suggest a significantly greater concentration of anisotropic trabecular structures in the head and neck regions when compared to the trochanter region (p < 10^-4 ). We also evaluated the ability of such AMFs to predict bone strength in the femoral head of proximal femur specimens obtained from 50 donors. Our results suggest that such AMFs, when used in conjunction with multi-regression models, can outperform more conventional features such as BMD in predicting failure load. We conclude that such anisotropic Minkowski Functionals can capture valuable information regarding directional attributes of local structure, which may be useful in a wide scope of biomedical imaging applications.

Predicting the Biomechanical Strength of Proximal Femur Specimens through High Dimensional Geometric Features and Support Vector Regression

Estimating local trabecular bone quality for purposes of femoral bone strength prediction is important for improving the clinical assessment of osteoporotic hip fracture risk. In this study, we explore the ability of geometric features derived from the Scaling Index Method (SIM) in predicting the biomechanical strength of proximal femur specimens as visualized on multi-detector computed tomography (MDCT) images. MDCT scans were acquired for 50 proximal femur specimens harvested from human cadavers. An automated volume of interest (VOI)-fitting algorithm was used to define a consistent volume in the femoral head of each specimen. In these VOIs, the non-linear micro-structure of the trabecular bone was characterized by statistical moments of its BMD distribution and by local scaling properties derived from SIM. Linear multi-regression analysis and support vector regression with a linear kernel (SVRlin) were used to predict the failure load (FL) from the feature sets; the predicted FL was compared to the FL values determined through biomechanical testing. The prediction performance was measured by the root mean square error (RMSE) for each image feature on independent test set. The best prediction result was obtained from the SIM feature set with SVRlin, which had the lowest prediction error (RMSE = 0.842 ± 0.209) and which was significantly lower than the conventionally used mean BMD (RMSE = 1.103 ± 0.262,, p<0.005). Our results indicate that the biomechanical strength prediction can be significantly improved in proximal femur specimens on MDCT images by using high-dimensional geometric features derived from SIM with support vector regression.

Classification of Small Lesions in Breast MRI: Evaluating The Role of Dynamically Extracted Texture Features Through Feature Selection

Dynamic texture quantification, i.e., extracting texture features from the lesion enhancement pattern in all available post-contrast images, has not been evaluated in terms of its ability to classify small lesions. This study investigates the classification performance achieved with texture features extracted from all five post-contrast images of lesions (mean lesion diameter of 1.1 cm) annotated in dynamic breast magnetic resonance imaging exams. Sixty lesions are characterized dynamically using Haralick texture features. The texture features are then used in a classification task with support vector regression and a fuzzy k-nearest neighbor classifier; free parameters of these classifiers are optimized using random sub-sampling cross-validation. Classifier performance is determined through receiver-operator characteristic (ROC) analysis, specifically through computation of the area under the ROC curve (AUC). Mutual information is used to evaluate the contribution of texture features extracted from different post-contrast stages to classifier performance. Significant improvements (p < 0.05) are observed for six of the thirteen texture features when the lesion enhancement pattern is quantified using the proposed approach of dynamic texture quantification. The highest AUC value observed (0.82) is achieved with texture features responsible for capturing aspects of lesion heterogeneity. Mutual information analysis reveals that texture features extracted from the third and fourth post-contrast images contributed most to the observed improvement in classifier performance. These results show that the performance of automated character classification with small lesions can be significantly improved through dynamic texture quantification of the lesion enhancement pattern.

Characterizing healthy and osteoarthritic knee cartilage on phase contrast CT with geometric texture features

The current approach to evaluating cartilage degeneration at the knee joint requires visualization of the joint space on radiographic images where indirect cues such as joint space narrowing serve as markers for osteoarthritis. A recent novel approach to visualizing the knee cartilage matrix using phase contrast imaging (PCI) with computed tomography (CT) was shown to allow direct examination of chondrocyte patterns and their subsequent correlation to osteoarthritis. This study aims to characterize chondrocyte cell patterns in the radial zone of the knee cartilage matrix in the presence and absence of osteoarthritic damage through texture analysis. Statistical features derived from gray-level co-occurrence matrices (GLCM) and geometric features derived from the Scaling Index Method (SIM) were extracted from 404 regions of interest (ROI) annotated on PCI images of healthy and osteoarthritic specimens of knee cartilage. These texture features were then used in a machine learning task to classify ROIs as healthy or osteoarthritic. A fuzzy k-nearest neighbor classifier was used and its performance was evaluated using the area under the Receiver Operating Characteristic (ROC) curve (AUC). The best classification performance was observed with high-dimensional geometrical feature vectors derived from SIM and GLCM correlation features. With the experimental conditions used in this study, both SIM and GLCM achieved a high classification performance (AUC value of 0.98) in the task of distinguishing between healthy and osteoarthritic ROIs. These results show that such quantitative analysis of chondrocyte patterns in the knee cartilage matrix can distinguish between healthy and osteoarthritic tissue with high accuracy.

Texture feature ranking with relevance learning to classify interstitial lung disease patterns

OBJECTIVE

The generalized matrix learning vector quantization (GMLVQ) is used to estimate the relevance of texture features in their ability to classify interstitial lung disease patterns in high-resolution computed tomography images.

METHODOLOGY

After a stochastic gradient descent, the GMLVQ algorithm provides a discriminative distance measure of relevance factors, which can account for pairwise correlations between different texture features and their importance for the classification of healthy and diseased patterns. 65 texture features were extracted from gray-level co-occurrence matrices (GLCMs). These features were ranked and selected according to their relevance obtained by GMLVQ and, for comparison, to a mutual information (MI) criteria. The classification performance for different feature subsets was calculated for a k-nearest-neighbor (kNN) and a random forests classifier (RanForest), and support vector machines with a linear and a radial basis function kernel (SVMlin and SVMrbf).

RESULTS

For all classifiers, feature sets selected by the relevance ranking assessed by GMLVQ had a significantly better classification performance (p<0.05) for many texture feature sets compared to the MI approach. For kNN, RanForest, and SVMrbf, some of these feature subsets had a significantly better classification performance when compared to the set consisting of all features (p<0.05).

CONCLUSION

While this approach estimates the relevance of single features, future considerations of GMLVQ should include the pairwise correlation for the feature ranking, e.g. to reduce the redundancy of two equally relevant features.

Prediction of biomechanical properties of trabecular bone in MR images with geometric features and support vector regression

Whole knee joint MR image datasets were used to compare the performance of geometric trabecular bone features and advanced machine learning techniques in predicting biomechanical strength properties measured on the corresponding ex vivo specimens. Changes of trabecular bone structure throughout the proximal tibia are indicative of several musculoskeletal disorders involving changes in the bone quality and the surrounding soft tissue. Recent studies have shown that MR imaging also allows non-invasive 3-D characterization of bone microstructure. Sophisticated features like the scaling index method (SIM) can estimate local structural and geometric properties of the trabecular bone and may improve the ability of MR imaging to determine local bone quality in vivo. A set of 67 bone cubes was extracted from knee specimens and their biomechanical strength estimated by the yield stress (YS) [in MPa] was determined through mechanical testing. The regional apparent bone volume fraction (BVF) and SIM derived features were calculated for each bone cube. A linear multiregression analysis (MultiReg) and a optimized support vector regression (SVR) algorithm were used to predict the YS from the image features. The prediction accuracy was measured by the root mean square error (RMSE) for each image feature on independent test sets. The best prediction result with the lowest prediction error of RMSE = 1.021 MPa was obtained with a combination of BVF and SIM features and by using SVR. The prediction accuracy with only SIM features and SVR (RMSE = 1.023 MPa) was still significantly better than BVF alone and MultiReg (RMSE = 1.073 MPa). The current study demonstrates that the combination of sophisticated bone structure features and supervised learning techniques can improve MR-based determination of trabecular bone quality.