Toward atlas-assisted automatic interpretation of MRI morphological brain scans in the presence of tumor

Fibre tract segmentation for intraoperative diffusion MRI in neurosurgical patients using tract-specific orientation atlas and tumour deformation modelling

Purpose:

Intraoperative diffusion MRI could provide a means of visualising brain fibre tracts near a neurosurgical target after preoperative images have been invalidated by brain shift. We propose an atlas-based intraoperative tract segmentation method, as the standard preoperative method, streamline tractography, is unsuitable for intraoperative implementation.

Methods:

A tract-specific voxel-wise fibre orientation atlas is constructed from healthy training data. After registration with a target image, a radial tumour deformation model is applied to the orientation atlas to account for displacement caused by lesions. The final tract map is obtained from the inner product of the atlas and target image fibre orientation data derived from intraoperative diffusion MRI.

Results:

The simple tumour model takes only seconds to effectively deform the atlas into alignment with the target image. With minimal processing time and operator effort, maps of surgically relevant tracts can be achieved that are visually and qualitatively comparable with results obtained from streamline tractography.

Conclusion:

Preliminary results demonstrate feasibility of intraoperative streamline-free tract segmentation in challenging neurosurgical cases. Demonstrated results in a small number of representative sample subjects are realistic despite the simplicity of the tumour deformation model employed. Following this proof of concept, future studies will focus on achieving robustness in a wide range of tumour types and clinical scenarios, as well as quantitative validation of segmentations.

Towards an Architecture of a Multi-purpose, User-Extendable Reference Human Brain Atlas

Human brain atlas development is predominantly research-oriented and the use of atlases in clinical practice is limited. Here I introduce a new definition of a reference human brain atlas that serves education, research and clinical applications, and is extendable by its user. Subsequently, an architecture of a multi-purpose, user-extendable reference human brain atlas is proposed and its implementation discussed. The human brain atlas is defined as a vehicle to gather, present, use, share, and discover knowledge about the human brain with highly organized content, tools enabling a wide range of its applications, massive and heterogeneous knowledge database, and means for content and knowledge growing by its users. The proposed architecture determines major components of the atlas, their mutual relationships, and functional roles. It contains four functional units, core cerebral models, knowledge database, research and clinical data input and conversion, and toolkit (supporting processing, content extension, atlas individualization, navigation, exploration, and display), all united by a user interface. Each unit is described in terms of its function, component modules and sub-modules, data handling, and implementation aspects. This novel architecture supports brain knowledge gathering, presentation, use, sharing, and discovery and is broadly applicable and useful in student- and educator-oriented neuroeducation for knowledge presentation and communication, research for knowledge acquisition, aggregation and discovery, and clinical applications in decision making support for prevention, diagnosis, treatment, monitoring, and prediction. It establishes a backbone for designing and developing new, multi-purpose and user-extendable brain atlas platforms, serving as a potential standard across labs, hospitals, and medical schools.

[A tissue recovery-based brain tumor image registration method]

We propose an algorithm for registration between brain tumor images and normal brain images based on tissue recovery. U-Net is first used in BraTS2018 dataset to segment the brain tumors, and PConv-Net is then used to simulate the generation of missing normal tissues in the tumor region to replace the tumor region. Finally, the normal brain image is registered to the tissue recovery brain image. We evaluated the effectiveness of this method by comparing the registration results of the repaired image and the tumor image corresponding to the surrounding tissues of the tumor area. The experimental results showed that the proposed method could reduce the effect of pathological variation, achieve a high registration accuracy, and effectively simulate and generate normal tissues to replace the tumor regions, thus improving the registration effect between brain tumor images and normal brain images.

Evolution of Human Brain Atlases in Terms of Content, Applications, Functionality, and Availability

Human brain atlases have been evolving tremendously, propelled recently by brain big projects, and driven by sophisticated imaging techniques, advanced brain mapping methods, vast data, analytical strategies, and powerful computing. We overview here this evolution in four categories: content, applications, functionality, and availability, in contrast to other works limited mostly to content. Four atlas generations are distinguished: early cortical maps, print stereotactic atlases, early digital atlases, and advanced brain atlas platforms, and 5 avenues in electronic atlases spanning the last two generations. Content-wise, new electronic atlases are categorized into eight groups considering their scope, parcellation, modality, plurality, scale, ethnicity, abnormality, and a mixture of them. Atlas content developments in these groups are heading in 23 various directions. Application-wise, we overview atlases in neuroeducation, research, and clinics, including stereotactic and functional neurosurgery, neuroradiology, neurology, and stroke. Functionality-wise, tools and functionalities are addressed for atlas creation, navigation, individualization, enabling operations, and application-specific. Availability is discussed in media and platforms, ranging from mobile solutions to leading-edge supercomputers, with three accessibility levels. The major application-wise shift has been from research to clinical practice, particularly in stereotactic and functional neurosurgery, although clinical applications are still lagging behind the atlas content progress. Atlas functionality also has been relatively neglected until recently, as the management of brain data explosion requires powerful tools. We suggest that the future human brain atlas-related research and development activities shall be founded on and benefit from a standard framework containing the core virtual brain model cum the brain atlas platform general architecture.

Human Brain Atlases in Stroke Management

Stroke is a leading cause of death and a major cause of permanent disability. Its management is demanding because of variety of protocols, imaging modalities, pulse sequences, hemodynamic maps, criteria for treatment, and time constraints to promptly evaluate and treat. To cope with some of these issues, we propose novel, patented solutions in stroke management by employing multiple brain atlases for diagnosis, treatment, and prediction. Numerous and diverse CT and MRI scans are used: ARIC cohort, ischemic and hemorrhagic stroke CT cases, MRI cases with multiple pulse sequences, and 128 stroke CT patients, each with 170 variables and one year follow-up. The method employs brain atlases of anatomy, blood supply territories, and probabilistic stroke atlas. It rapidly maps an atlas to scan and provides atlas-assisted scan processing. Atlas-to-scan mapping is application-dependent and handles three types of regions of interest (ROIs): atlas-defined ROIs, atlas-quantified ROIs, and ROIs creating an atlas. An ROI is defined by atlas-guided anatomy or scan-derived pathology. The atlas defines ROI or quantifies it. A brain atlas potential has been illustrated in four atlas-assisted applications for stroke occurrence prediction and screening, rapid and automatic stroke diagnosis in emergency room, quantitative decision support in thrombolysis in ischemic stroke, and stroke outcome prediction and treatment assessment. The use of brain atlases in stroke has many potential advantages, including rapid processing, automated and robust handling, wide range of applications, and quantitative assessment. Further work is needed to enhance the developed prototypes, clinically validate proposed solutions, and introduce them to clinical practice.

Human brain atlasing: past, present and future

We have recently witnessed an explosion of large-scale initiatives and projects addressing mapping, modeling, simulation and atlasing of the human brain, including the BRAIN Initiative, the Human Brain Project, the Human Connectome Project (HCP), the Big Brain, the Blue Brain Project, the Allen Brain Atlas, the Brainnetome, among others. Besides these large and international initiatives, there are numerous mid-size and small brain atlas-related projects. My contribution to these global efforts has been to create adult human brain atlases in health and disease, and to develop atlas-based applications. For over two decades with my R&D lab I developed 35 brain atlases, licensed to 67 companies and made available in about 100 countries. This paper has two objectives. First, it provides an overview of the state of the art in brain atlasing. Second, as it is already 20 years from the release of our first brain atlas, I summarise my past and present efforts, share my experience in atlas creation, validation and commercialisation, compare with the state of the art, and propose future directions.

Computational and mathematical methods in brain atlasing

Brain atlases have a wide range of use from education to research to clinical applications. Mathematical methods as well as computational methods and tools play a major role in the process of brain atlas building and developing atlas-based applications. Computational methods and tools cover three areas: dedicated editors for brain model creation, brain navigators supporting multiple platforms, and atlas-assisted specific applications. Mathematical methods in atlas building and developing atlas-aided applications deal with problems in image segmentation, geometric body modelling, physical modelling, atlas-to-scan registration, visualisation, interaction and virtual reality. Here I overview computational and mathematical methods in atlas building and developing atlas-assisted applications, and share my contribution to and experience in this field.

Usefulness of brain atlases in neuroradiology: Current status and future potential

Human brain atlases, although prevalent in medical education and stereotactic and functional neurosurgery, are not yet applied practically in neuroradiology. In a step towards introducing brain atlases to neuroradiology, we discuss nine different situations of potential atlas use: (1) to support interpretation of brain scans with clearly visible structures (to increase confidence of non-neuroradiologists); (2) to delineate and label scans of low anatomical content (with indiscernible or poorly visible anatomy); (3) to assist in generating the structured report; (4) to assist in interpreting small deep lesions, since an atlas's anatomical parcellation is higher than that of the interpreted scan; (5) to approximate distorted due to pathology (and unknown to the interpreter) anatomy and label it; (6) to cope with data explosion; (7) to assist in the interpretation of functional scans (to label the activation foci with the underlying anatomy and Brodmann's areas); (8) to support ischemic stroke image handling by means of atlases of anatomy and blood supply territories; and (9) to communicate image interpretation results (diagnosis) to others. The usefulness of the atlas for automatic structure identification, localisation, delineation, labelling and quantification, as well as for reporting and communication, potentially increases the interpreter's efficiency and confidence, as well as expedites image interpretation.

Toward the holistic, reference, and extendable atlas of the human brain, head, and neck

Despite numerous efforts, a fairly complete (holistic) anatomical model of the whole, normal, adult human brain, which is required as the reference in brain studies and clinical applications, has not yet been constructed. Our ultimate objective is to build this kind of atlas from advanced in vivo imaging. This work presents the taxonomy of our currently developed brain atlases and addresses the design, content, functionality, and current results in the holistic atlas development as well as atlas usefulness and future directions. We have developed to date 35 commercial brain atlases (along with numerous research prototypes), licensed to 63 companies and institutions, and made available to medical societies, organizations, medical schools, and individuals. These atlases have been applied in education, research, and clinical applications. Hundreds of thousands of patients have been treated by using our atlases. Based on this experience, the first version of the holistic and reference atlas of the brain, head, and neck has been developed and made available. The atlas has been created from multispectral 3 and 7 Tesla and high-resolution CT in vivo scans. It is fully 3D, scalable, interactive, and highly detailed with about 3,000 labeled components. This atlas forms a foundation for the development of a multi-level molecular, cellular, anatomical, physiological, and behavioral brain atlas platform.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Deformable registration of glioma images using EM algorithm and diffusion reaction modeling

This paper investigates the problem of atlas registration of brain images with gliomas. Multiparametric imaging modalities (T1, T1-CE, T2, and FLAIR) are first utilized for segmentations of different tissues, and to compute the posterior probability map (PBM) of membership to each tissue class, using supervised learning. Similar maps are generated in the initially normal atlas, by modeling the tumor growth, using reaction-diffusion equation. Deformable registration using a demons-like algorithm is used to register the patient images with the tumor bearing atlas. Joint estimation of the simulated tumor parameters (e.g., location, mass effect and degree of infiltration), and the spatial transformation is achieved by maximization of the log-likelihood of observation. An expectation-maximization algorithm is used in registration process to estimate the spatial transformation and other parameters related to tumor simulation are optimized through asynchronous parallel pattern search (APPSPACK). The proposed method has been evaluated on five simulated data sets created by statistically simulated deformations (SSD), and fifteen real multichannel glioma data sets. The performance has been evaluated both quantitatively and qualitatively, and the results have been compared to ORBIT, an alternative method solving a similar problem. The results show that our method outperforms ORBIT, and the warped templates have better similarity to patient images.

Non-diffeomorphic registration of brain tumor images by simulating tissue loss and tumor growth

Although a variety of diffeomorphic deformable registration methods exist in the literature, application of these methods in the presence of space-occupying lesions is not straightforward. The motivation of this work is spatial normalization of MR images from patients with brain tumors in a common stereotaxic space, aiming to pool data from different patients into a common space in order to perform group analyses. Additionally, transfer of structural and functional information from neuroanatomical brain atlases into the individual patient's space can be achieved via the inverse mapping, for the purpose of segmenting brains and facilitating surgical or radiotherapy treatment planning. A method that estimates the brain tissue loss and replacement by tumor is applied for achieving equivalent image content between an atlas and a patient's scan, based on a biomechanical model of tumor growth. Automated estimation of the parameters modeling brain tissue loss and displacement is performed via optimization of an objective function reflecting feature-based similarity and elastic stretching energy, which is optimized in parallel via APPSPACK (Asynchronous Parallel Pattern Search). The results of the method, applied to 21 brain tumor patients, indicate that the registration accuracy is relatively high in areas around the tumor, as well as in the healthy portion of the brain. Also, the calculated deformation in the vicinity of the tumor is shown to correlate highly with expert-defined visual scores indicating the tumor mass effect, thereby potentially leading to an objective approach to quantification of mass effect, which is commonly used in diagnosis.

New techniques for cartilage magnetic resonance imaging relaxation time analysis: texture analysis of flattened cartilage and localized intra- and inter-subject comparisons

MR relaxation time measurements of knee cartilage have shown potential to characterize knee osteoarthritis (OA). In this work, techniques that allow localized intra- and inter-subject comparisons of cartilage relaxation times, as well as cartilage flattening for texture analysis parallel and perpendicular to the natural cartilage layers, are presented. The localized comparisons are based on the registration of bone structures and the assignment of relaxation time feature vectors to each point in the bone-cartilage interface. Cartilage flattening was accomplished with Bezier splines and warping, and texture analysis was performed with second-order texture measures using gray-level co-occurrence matrices (GLCM). In a cohort of five normal subjects the performance and reproducibility of the techniques were evaluated using T1rho maps of femoral knee cartilage. The feasibility of creating a mean cartilage relaxation time map is also presented. Successful localized intra- and inter-subject T1rho comparisons were obtained with reproducibility similar to that reported in the literature for regional T2. Improvement of the reproducibility of GLCM features was obtained by flattening the T1rho maps. The results indicate that the presented techniques have potential in longitudinal and population studies of knee OA at different stages of the disease.

ORBIT: a multiresolution framework for deformable registration of brain tumor images

A deformable registration method is proposed for registering a normal brain atlas with images of brain tumor patients. The registration is facilitated by first simulating the tumor mass effect in the normal atlas in order to create an atlas image that is as similar as possible to the patient's image. An optimization framework is used to optimize the location of tumor seed as well as other parameters of the tumor growth model, based on the pattern of deformation around the tumor region. In particular, the optimization is implemented in a multiresolution and hierarchical scheme, and it is accelerated by using a principal component analysis (PCA)-based model of tumor growth and mass effect, trained on a computationally more expensive biomechanical model. Validation on simulated and real images shows that the proposed registration framework, referred to as ORBIT (optimization of tumor parameters and registration of brain images with tumors), outperforms other available registration methods particularly for the regions close to the tumor, and it has the potential to assist in constructing statistical atlases from tumor-diseased brain images.

Dense deformation field estimation for atlas-based segmentation of pathological MR brain images

Atlas registration is a recognized paradigm for the automatic segmentation of normal MR brain images. Unfortunately, atlas-based segmentation has been of limited use in presence of large space-occupying lesions. In fact, brain deformations induced by such lesions are added to normal anatomical variability and they may dramatically shift and deform anatomically or functionally important brain structures. In this work, we chose to focus on the problem of inter-subject registration of MR images with large tumors, inducing a significant shift of surrounding anatomical structures. First, a brief survey of the existing methods that have been proposed to deal with this problem is presented. This introduces the discussion about the requirements and desirable properties that we consider necessary to be fulfilled by a registration method in this context: To have a dense and smooth deformation field and a model of lesion growth, to model different deformability for some structures, to introduce more prior knowledge, and to use voxel-based features with a similarity measure robust to intensity differences. In a second part of this work, we propose a new approach that overcomes some of the main limitations of the existing techniques while complying with most of the desired requirements above. Our algorithm combines the mathematical framework for computing a variational flow proposed by Hermosillo et al. [G. Hermosillo, C. Chefd'Hotel, O. Faugeras, A variational approach to multi-modal image matching, Tech. Rep., INRIA (February 2001).] with the radial lesion growth pattern presented by Bach et al. [M. Bach Cuadra, C. Pollo, A. Bardera, O. Cuisenaire, J.-G. Villemure, J.-Ph. Thiran, Atlas-based segmentation of pathological MR brain images using a model of lesion growth, IEEE Trans. Med. Imag. 23 (10) (2004) 1301-1314.]. Results on patients with a meningioma are visually assessed and compared to those obtained with the most similar method from the state-of-the-art.

Fast Talairach Transformation for magnetic resonance neuroimages

We introduce and validate the Fast Talairach Transformation (FTT). FTT is a rapid version of the Talairach transformation (TT) with the modified Talairach landmarks. Landmark identification is fully automatic and done in 3 steps: calculation of midsagittal plane, computing of anterior commissure (AC) and posterior commissure (PC) landmarks, and calculation of cortical landmarks. To perform these steps, we use fast and anatomy-based algorithms employing simple operations. FTT was validated for 215 diversified T1-weighted and spoiled gradient recalled (SPGR) MRI data sets. It calculates the landmarks and warps the Talairach-Tournoux atlas fully automatically in about 5 sec on a standard computer. The average distance errors in landmark localization are (in mm): 1.16 (AC), 1.49 (PC), 0.08 (left), 0.13 (right), 0.48 (anterior), 0.16 (posterior), 0.35 (superior), and 0.52 (inferior). Extensions to FTT by introducing additional landmarks and applying nonlinear warping against the ventricular system are addressed. Application of FTT to other brain atlases of anatomy, function, tracts, cerebrovasculature, and blood supply territories is discussed. FTT may be useful in a clinical setting and research environment: (1) when the TT is used traditionally, (2) when a global brain structure positioning with quick searching and labeling is required, (3) in urgent cases for quick image interpretation (eg, acute stroke), (4) when the difference between nonlinear and piecewise linear warping is negligible, (5) when automatic processing of a large number of cases is required, (6) as an initial atlas-scan alignment before performing nonlinear warping, and (7) as an initial atlas-guided segmentation of brain structures before further local processing.

Dorsoventral Extension of the Talairach Transformation and Its Automatic Calculation for Magnetic Resonance Neuroimages

The Talairach transformation (TT), the most prevalent method for brain normalization and atlas-to-data warping, is conceptually simple, fast and can be automated. Two problems with the TT in the clinical setting that are addressed in this article are reduced accuracy at the orbitofrontal cortex and upper corpus callosum (CC) and unsuitability for functional neurosurgery because of incomplete scanning. To increase dorsoventral accuracy, we introduce 2 additional landmarks: the top of the CC (SM) and the most ventral point of the orbitofrontal cortex on the midsagittal slab (IM). A method for their automatic calculation is proposed and validated against 55 diversified magnetic resonance (MR) imaging cases. The SM and IM landmarks are identified accurately and robustly in an automatic way. The average error of SM localization is 0.69 mm, and 91% of all cases have an error not greater than 1 mm. The average error of IM localization is 0.98 mm, approximately three quarters of cases have an error not greater than 1 mm, and 95% of all cases have an error not larger than 2 mm. The SM is correlated (R(2) = 0.72) with the most superior cortical landmark, whereas the IM is only loosely correlated (R(2) = 0.22) with the most inferior cortical landmark. On average, the original TT overlays the atlas axial plate at -24 on the orbitofrontal cortex as opposed to the correct plate at -28. Therefore, 1-dimensional ventral scaling in the original TT is insufficient to cope with variability in the orbitofrontal cortex. The key advantages of our approach are the preserved conceptual simplicity of the TT, fully automatic identification of the new landmarks, improved accuracy of the atlas-to-data match without compromising performance, and enabled TT use in functional neurosurgery when a dorsal part of the brain is not available in the scan.