Automating Ground Truth Annotations for Gland Segmentation Through Immunohistochemistry

Microscopic evaluation of glands in the colon is of utmost importance in the diagnosis of inflammatory bowel disease and cancer. When properly trained, deep learning pipelines can provide a systematic, reproducible, and quantitative assessment of disease-related changes in glandular tissue architecture. The training and testing of deep learning models require large amounts of manual annotations, which are difficult, time-consuming, and expensive to obtain. Here, we propose a method for automated generation of ground truth in digital hematoxylin and eosin (H&E)-stained slides using immunohistochemistry (IHC) labels. The image processing pipeline generates annotations of glands in H&E histopathology images from colon biopsy specimens by transfer of gland masks from KRT8/18, CDX2, or EPCAM IHC. The IHC gland outlines are transferred to coregistered H&E images for training of deep learning models. We compared the performance of the deep learning models to that of manual annotations using an internal held-out set of biopsy specimens as well as 2 public data sets. Our results show that EPCAM IHC provides gland outlines that closely match manual gland annotations (Dice = 0.89) and are resilient to damage by inflammation. In addition, we propose a simple data sampling technique that allows models trained on data from several sources to be adapted to a new data source using just a few newly annotated samples. The best performing models achieved average Dice scores of 0.902 and 0.89 on Gland Segmentation and Colorectal Adenocarcinoma Gland colon cancer public data sets, respectively, when trained with only 10% of annotated cases from either public cohort. Altogether, the performances of our models indicate that automated annotations using cell type-specific IHC markers can safely replace manual annotations. Automated IHC labels from single-institution cohorts can be combined with small numbers of hand-annotated cases from multi-institutional cohorts to train models that generalize well to diverse data sources.

A Correspondence-Based Network Approach for Groupwise Analysis of Patient-Specific Spatiotemporal Data

Methods for statistically analyzing patient-specific data that vary both spatially and over time are currently either limited to summary statistics or require elaborate surface registration. We propose a new method, called correspondence-based network analysis, which leverages particle-based shape modeling to establish correspondence across a population and preserve patient-specific measurements and predictions through statistical analysis. Herein, we evaluated this method using three published datasets of the hip describing cortical bone thickness of the proximal femur, cartilage contact stress, and dynamic joint space between control and patient cohorts to evaluate activity- and group-based differences, as applicable, using traditional statistical parametric mapping (SPM) and our proposed spatially considerate correspondence-based network analysis approach. The network approach was insensitive to correspondence density, while the traditional application of SPM showed decreasing area of the region of significance with increasing correspondence density. In comparison to SPM, the network approach identified broader and more connected regions of significance for all three datasets. The correspondence-based network analysis approach identified differences between groups and activities without loss of subject and spatial specificity which could improve clinical interpretation of results.

Particle-Based Shape Modeling for Arbitrary Regions-of-Interest

Statistical Shape Modeling (SSM) is a quantitative method for analyzing morphological variations in anatomical structures. These analyses often necessitate building models on targeted anatomical regions of interest to focus on specific morphological features. We propose an extension to particle-based shape modeling (PSM), a widely used SSM framework, to allow shape modeling to arbitrary regions of interest. Existing methods to define regions of interest are computationally expensive and have topological limitations. To address these shortcomings, we use mesh fields to define free-form constraints, which allow for delimiting arbitrary regions of interest on shape surfaces. Furthermore, we add a quadratic penalty method to the model optimization to enable computationally efficient enforcement of any combination of cutting-plane and free-form constraints. We demonstrate the effectiveness of this method on a challenging synthetic dataset and two medical datasets.

Statistical multi-level shape models for scalable modeling of multi-organ anatomies

Statistical shape modeling is an indispensable tool in the quantitative analysis of anatomies. Particle-based shape modeling (PSM) is a state-of-the-art approach that enables the learning of population-level shape representation from medical imaging data (e.g., CT, MRI) and the associated 3D models of anatomy generated from them. PSM optimizes the placement of a dense set of landmarks (i.e., correspondence points) on a given shape cohort. PSM supports multi-organ modeling as a particular case of the conventional single-organ framework via a global statistical model, where multi-structure anatomy is considered as a single structure. However, global multi-organ models are not scalable for many organs, induce anatomical inconsistencies, and result in entangled shape statistics where modes of shape variation reflect both within- and between-organ variations. Hence, there is a need for an efficient modeling approach that can capture the inter-organ relations (i.e., pose variations) of the complex anatomy while simultaneously optimizing the morphological changes of each organ and capturing the population-level statistics. This paper leverages the PSM approach and proposes a new approach for correspondence-point optimization of multiple organs that overcomes these limitations. The central idea of multilevel component analysis, is that the shape statistics consists of two mutually orthogonal subspaces: the within-organ subspace and the between-organ subspace. We formulate the correspondence optimization objective using this generative model. We evaluate the proposed method using synthetic shape data and clinical data for articulated joint structures of the spine, foot and ankle, and hip joint.

Prediction of femoral head coverage from articulated statistical shape models of patients with developmental dysplasia of the hip

Developmental dysplasia of the hip (DDH) is commonly described as reduced femoral head coverage due to anterolateral acetabular deficiency. Although reduced coverage is the defining trait of DDH, more subtle and localized anatomic features of the joint are also thought to contribute to symptom development and degeneration. These features are challenging to identify using conventional approaches. Herein, we assessed the morphology of the full femur and hemi-pelvis using an articulated statistical shape model (SSM). The model determined the morphological and pose-based variations associated with DDH in a population of Japanese females and established which of these variations predict coverage. Computed tomography (CT) images of 83 hips from 47 patients were segmented for input into a correspondence-based SSM. The dominant modes of variation in the model initially represented scale and pose. After removal of these factors through individual bone alignment, femoral version and neck-shaft angle, pelvic curvature, and acetabular version dominated the observed variation. Femoral head oblateness and prominence of the acetabular rim and various muscle attachment sites of the femur and hemi-pelvis were found to predict 3D CT-based coverage measurements (R2  = 0.5-0.7 for the full bones, R2  = 0.9 for the joint). Statement of Clinical Significance: Currently, clinical measurements of DDH only consider the morphology of the acetabulum. However, the results of this study demonstrated that variability in femoral head shape and several muscle attachment sites were predictive of femoral head coverage. These morphological differences may provide insight into improved clinical diagnosis and surgical planning based on functional adaptations of patients with DDH.

Benchmarking off-the-shelf statistical shape modeling tools in clinical applications

Statistical shape modeling (SSM) is widely used in biology and medicine as a new generation of morphometric approaches for the quantitative analysis of anatomical shapes. Technological advancements of in vivo imaging have led to the development of open-source computational tools that automate the modeling of anatomical shapes and their population-level variability. However, little work has been done on the evaluation and validation of such tools in clinical applications that rely on morphometric quantifications(e.g., implant design and lesion screening). Here, we systematically assess the outcome of widely used, state-of-the-art SSM tools, namely ShapeWorks, Deformetrica, and SPHARM-PDM. We use both quantitative and qualitative metrics to evaluate shape models from different tools. We propose validation frameworks for anatomical landmark/measurement inference and lesion screening. We also present a lesion screening method to objectively characterize subtle abnormal shape changes with respect to learned population-level statistics of controls. Results demonstrate that SSM tools display different levels of consistencies, where ShapeWorks and Deformetrica models are more consistent compared to models from SPHARM-PDM due to the groupwise approach of estimating surface correspondences. Furthermore, ShapeWorks and Deformetrica shape models are found to capture clinically relevant population-level variability compared to SPHARM-PDM models.

Multi-level multi-domain statistical shape model of the subtalar, talonavicular, and calcaneocuboid joints

Traditionally, two-dimensional conventional radiographs have been the primary tool to measure the complex morphology of the foot and ankle. However, the subtalar, talonavicular, and calcaneocuboid joints are challenging to assess due to their bone morphology and locations within the ankle. Weightbearing computed tomography is a novel high-resolution volumetric imaging mechanism that allows detailed generation of 3D bone reconstructions. This study aimed to develop a multi-domain statistical shape model to assess morphologic and alignment variation of the subtalar, talonavicular, and calcaneocuboid joints across an asymptomatic population and calculate 3D joint measurements in a consistent weightbearing position. Specific joint measurements included joint space distance, congruence, and coverage. Noteworthy anatomical variation predominantly included the talus and calcaneus, specifically an inverse relationship regarding talar dome heightening and calcaneal shortening. While there was minimal navicular and cuboid shape variation, there were alignment variations within these joints; the most notable is the rotational aspect about the anterior-posterior axis. This study also found that multi-domain modeling may be able to predict joint space distance measurements within a population. Additionally, variation across a population of these four bones may be driven far more by morphology than by alignment variation based on all three joint measurements. These data are beneficial in furthering our understanding of joint-level morphology and alignment variants to guide advancements in ankle joint pathological care and operative treatments.

Combined Estimation of Shape and Pose for Statistical Analysis of Articulating Joints

Quantifying shape variations in articulated joints is of utmost interest to understand the underlying joint biomechanics and associated clinical symptoms. For joint comparisons and analysis, the relative positions of the bones can confound subsequent analysis. Clinicians design specific image acquisition protocols to neutralize the individual pose variations. However, recent studies have shown that even specific acquisition protocols fail to achieve consistent pose. The individual pose variations are largely attributed to the day-to-day functioning of the patient, such as gait during walk, as well as interactions between specific morphologies and joint alignment. This paper presents a novel two-step method to neutralize such patient-specific variations while simultaneously preserving the inherent relationship of the articulated joint. The resulting shape models are then used to discover clinically relevant shape variations in a population of hip joints.

Thinking outside the glenohumeral box: Hierarchical shape variation of the periarticular anatomy of the scapula using statistical shape modeling

Variation in the shape of the glenoid and periarticular anatomy of the scapula has been associated with shoulder pathology. The goal of this study was to identify the modes of shape variation of periarticular scapular anatomy in relation to the glenoid in nonpathologic shoulders. Computed tomography scans of 31 cadaveric scapulae, verified to be free of pathology, were three-dimensionally reconstructed. Statistical shape modeling and principal component analysis identified the modes of shape variation across the population. Corresponding linear and angular measurements quantified the morphometric variance identified by the modes. Linear measures were normalized to the radius of the inferior glenoid to account for differences in the scaling of the bones. Five modes captured 89.7% of total shape variation of the glenoid and periarticular anatomy. Apart from size differences (mode 1: 33.0%), acromial anatomy accounted for the largest variation (mode 2: 32.0%). Further modes described variation in glenoid inclination (mode 3: 11.8%), coracoid orientation and size (mode 4: 9.0%), and variation in coracoacromial (CA) morphology (mode 5: 3.1%). The average scapula had a mean acromial tilt of 49 ± 7°, scapular spine angle of 61 ± 6°, the glenoid inclination of 84 ± 4°, coracoid deviation angle of 26 ± 4°, coracoid length of 3.7 ± 0.3 glenoid radii, and a CA base length of 5.6 ± 0.5 radii. In this study, the identified shape modes explain almost all of the variance in scapular anatomy. The acromion exhibited the highest variance of all periarticular anatomic structures of the scapula in relation to the glenoid, which may play a role in many shoulder pathologies.

An Optimal, Generative Model for Estimating Multi-Label Probabilistic Maps

Multi-label probabilistic maps, a.k.a. probabilistic segmentations, parameterize a population of intimately co-existing anatomical shapes and are useful for various medical imaging applications, such as segmentation, anatomical atlases, shape analysis, and consensus generation. Existing methods to estimate probabilistic segmentations rely on ad hoc intermediate representations (e.g., average of Gaussian-smoothed label maps and smoothed signed distance maps) that do not necessarily conform to the underlying generative process. Generative modeling of such maps could help discover as well as aide in the statistical analysis of sub-groups in a population via clustering and mixture modeling techniques. In this paper, we propose an estimation of multi-label probabilistic maps and showcase their favorable performance for modeling anatomical shapes such as the left atrium of the human heart and brain structures. The proposed formulation relies on a constrained optimization in the natural parameter space of the exponential family form of categorical distributions. A smoothness prior provides generalizability in the model and helps achieve greater performance in modeling tasks for unseen samples. We demonstrate and compare the effectiveness of the proposed method for Bayesian image segmentation, multi-atlas segmentation, and shape-based clustering.

Coracoacromial morphology: a contributor to recurrent traumatic anterior glenohumeral instability?

BACKGROUND

Although scapular morphology contributes to glenohumeral osteoarthritis and rotator cuff disease, its role in traumatic glenohumeral instability remains unknown. We hypothesized that coracoacromial and glenoid morphology would differ between healthy subjects and patients with recurrent traumatic anterior shoulder instability.

METHODS

Computed tomography scans of 31 cadaveric control scapulae and 54 scapulae of patients with recurrent traumatic anterior shoulder instability and Hill-Sachs lesions were 3-dimensionally reconstructed. Statistical shape modeling identified the modes of variation between the scapulae of both groups. Corresponding measurements quantified these modes in relation to the glenoid center (linear offset measures), defined by the best-fit circle of the inferior glenoid, or the glenoid center plane (angles), which bisects the glenoid longitudinally. Distances were normalized for glenoid size.

RESULTS

Compared with controls, the unstable coracoids were shorter (P = .004), with a more superior and medial offset of the tip (mean difference [MD], 7 and 3 mm, respectively; P < .001) and an origin closer to the 12-o'clock position (MD, 6°; P < .001). The unstable scapular spines originated closer to the 9-o'clock position (MD, 4°; P = .012), and the unstable acromions were more vertically oriented (MD, 6°; P < .001). The unstable glenoids had an increased height-width index (MD, 0.04; P = .021), had a flatter anterior-posterior radius of curvature (MD, 77 mm; P < .001), and were more anteriorly tilted (MD, 5°; P = .005).

CONCLUSIONS

Coracoacromial and glenoid anatomy differs between individuals with and without recurrent traumatic anterior shoulder instability. This pathologic anatomy is not addressed by current soft-tissue stabilization procedures and may contribute to instability recurrence.

DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images

Statistical shape modeling is an important tool to characterize variation in anatomical morphology. Typical shapes of interest are measured using 3D imaging and a subsequent pipeline of registration, segmentation, and some extraction of shape features or projections onto some lower-dimensional shape space, which facilitates subsequent statistical analysis. Many methods for constructing compact shape representations have been proposed, but are often impractical due to the sequence of image preprocessing operations, which involve significant parameter tuning, manual delineation, and/or quality control by the users. We propose DeepSSM: a deep learning approach to extract a low-dimensional shape representation directly from 3D images, requiring virtually no parameter tuning or user assistance. DeepSSM uses a convolutional neural network (CNN) that simultaneously localizes the biological structure of interest, establishes correspondences, and projects these points onto a low-dimensional shape representation in the form of PCA loadings within a point distribution model. To overcome the challenge of the limited availability of training images with dense correspondences, we present a novel data augmentation procedure that uses existing correspondences on a relatively small set of processed images with shape statistics to create plausible training samples with known shape parameters. In this way, we leverage the limited CT/MRI scans (40-50) into thousands of images needed to train a deep neural net. After the training, the CNN automatically produces accurate low-dimensional shape representations for unseen images. We validate DeepSSM for three different applications pertaining to modeling pediatric cranial CT for characterization of metopic craniosynostosis, femur CT scans identifying morphologic deformities of the hip due to femoroacetabular impingement, and left atrium MRI scans for atrial fibrillation recurrence prediction.

Quantitative comparison of cortical bone thickness using correspondence-based shape modeling in patients with cam femoroacetabular impingement

The proximal femur is abnormally shaped in patients with cam-type femoroacetabular impingement (FAI). Impingement may elicit bone remodeling at the proximal femur, causing increases in cortical bone thickness. We used correspondence-based shape modeling to quantify and compare cortical thickness between cam patients and controls for the location of the cam lesion and the proximal femur. Computed tomography images were segmented for 45 controls and 28 cam-type FAI patients. The segmentations were input to a correspondence-based shape model to identify the region of the cam lesion. Median cortical thickness data over the region of the cam lesion and the proximal femur were compared between mixed-gender and gender-specific groups. Median [interquartile range] thickness was significantly greater in FAI patients than controls in the cam lesion (1.47 [0.64] vs. 1.13 [0.22] mm, respectively; p < 0.001) and proximal femur (1.28 [0.30] vs. 0.97 [0.22] mm, respectively; p < 0.001). Maximum thickness in the region of the cam lesion was more anterior and less lateral (p < 0.001) in FAI patients. Male FAI patients had increased thickness compared to male controls in the cam lesion (1.47 [0.72] vs. 1.10 [0.19] mm, respectively; p < 0.001) and proximal femur (1.25 [0.29] vs. 0.94 [0.17] mm, respectively; p < 0.001). Thickness was not significantly different between male and female controls.

CLINICAL SIGNIFICANCE

Studies of non-pathologic cadavers have provided guidelines regarding safe surgical resection depth for FAI patients. However, our results suggest impingement induces cortical thickening in cam patients, which may strengthen the proximal femur. Thus, these previously established guidelines may be too conservative. © 2016 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 35:1743-1753, 2017.

OPTIMAL PARAMETER MAP ESTIMATION FOR SHAPE REPRESENTATION: A GENERATIVE APPROACH

Probabilistic label maps are a useful tool for important medical image analysis tasks such as segmentation, shape analysis, and atlas building. Existing methods typically rely on blurred signed distance maps or smoothed label maps to model uncertainties and shape variabilities, which do not conform to any generative model or estimation process, and are therefore suboptimal. In this paper, we propose to learn probabilistic label maps using a generative model on given set of binary label maps. The proposed approach generalizes well on unseen data while simultaneously capturing the variability in the training samples. Efficiency of the proposed approach is demonstrated for consensus generation and shape-based clustering using synthetic datasets as well as left atrial segmentations from late-gadolinium enhancement MRI.