1
|
Mao Z, Zhao L, Huang S, Fan Y, Pui-Wai Lee A. Direct 3D ultrasound fusion for transesophageal echocardiography. Comput Biol Med 2021; 134:104502. [PMID: 34130220 DOI: 10.1016/j.compbiomed.2021.104502] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 05/10/2021] [Accepted: 05/11/2021] [Indexed: 10/21/2022]
Abstract
BACKGROUND Real-time three-dimensional transesophageal echocardiography (3D TEE) has been increasingly used in clinic for fast 3D analysis of cardiac anatomy and function. However, 3D TEE still suffers from the limited field of view (FoV). It is challenging to adopt conventional multi-view fusion methods to 3D TEE images because feature-based registration methods tend to fail in the ultrasound scenario, and conventional intensity-based methods have poor convergence properties and require an iterative coarse-to-fine strategy. METHODS A novel multi-view registration and fusion method is proposed to enlarge the FoV of 3D TEE images efficiently. A direct method is proposed to solve the registration problem in the Lie algebra space. Fast implementation is realized by searching voxels on three orthogonal planes between two volumes. Besides, a weighted-average 3D fusion method is proposed to fuse the aligned images seamlessly. For a sequence of 3D TEE images, they are fused incrementally. RESULTS Qualitative and quantitative results of in-vivo experiments indicate that the proposed registration algorithm outperforms a state-of-the-art PCA-based registration method in terms of accuracy and efficiency. Image registration and fusion performed on 76 in-vivo 3D TEE volumes from nine patients show apparent enlargement of FoV (enlarged around two times) in the obtained fused images. CONCLUSIONS The proposed methods can fuse 3D TEE images efficiently and accurately so that the whole Region of Interest (ROI) can be seen in a single frame. This research shows good potential to assist clinical diagnosis, preoperative planning, and future intraoperative guidance with 3D TEE.
Collapse
Affiliation(s)
- Zhehua Mao
- Centre for Autonomous Systems, Faculty of Engineering and Information Technology, University of Technology, Sydney, Australia.
| | - Liang Zhao
- Centre for Autonomous Systems, Faculty of Engineering and Information Technology, University of Technology, Sydney, Australia
| | - Shoudong Huang
- Centre for Autonomous Systems, Faculty of Engineering and Information Technology, University of Technology, Sydney, Australia
| | - Yiting Fan
- Department of Cardiology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Alex Pui-Wai Lee
- Division of Cardiology, Department of Medicine and Therapeutics, Prince of Wales Hospital and Laboratory of Cardiac Imaging and 3D Printing, Li Ka Shing Institute of Health Science, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, China
| |
Collapse
|
2
|
Mata G, Radojević M, Fernandez-Lozano C, Smal I, Werij N, Morales M, Meijering E, Rubio J. Automated Neuron Detection in High-Content Fluorescence Microscopy Images Using Machine Learning. Neuroinformatics 2019; 17:253-269. [PMID: 30215167 DOI: 10.1007/s12021-018-9399-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
The study of neuronal morphology in relation to function, and the development of effective medicines to positively impact this relationship in patients suffering from neurodegenerative diseases, increasingly involves image-based high-content screening and analysis. The first critical step toward fully automated high-content image analyses in such studies is to detect all neuronal cells and distinguish them from possible non-neuronal cells or artifacts in the images. Here we investigate the performance of well-established machine learning techniques for this purpose. These include support vector machines, random forests, k-nearest neighbors, and generalized linear model classifiers, operating on an extensive set of image features extracted using the compound hierarchy of algorithms representing morphology, and the scale-invariant feature transform. We present experiments on a dataset of rat hippocampal neurons from our own studies to find the most suitable classifier(s) and subset(s) of features in the common practical setting where there is very limited annotated data for training. The results indicate that a random forests classifier using the right feature subset ranks best for the considered task, although its performance is not statistically significantly better than some support vector machine based classification models.
Collapse
Affiliation(s)
- Gadea Mata
- Department of Mathematics and Computer Science, University of La Rioja, Logroño, Spain.
| | - Miroslav Radojević
- Biomedical Imaging Group Rotterdam, Departments of Medical Informatics and Radiology, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Carlos Fernandez-Lozano
- Department of Computer Science, University of A Coruña, A Coruña, Spain.,Instituto de Investigación Biomédica de A Coruña, Complexo Hospitalario Universitario de A Coruña, A Coruña, Spain
| | - Ihor Smal
- Biomedical Imaging Group Rotterdam, Departments of Medical Informatics and Radiology, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Niels Werij
- Biomedical Imaging Group Rotterdam, Departments of Medical Informatics and Radiology, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Miguel Morales
- Molecular Cognition Laboratory, Biophysics Institute, CSIC-UPV/EHU, Campus Universidad del País Vasco, Leioa, Spain
| | - Erik Meijering
- Biomedical Imaging Group Rotterdam, Departments of Medical Informatics and Radiology, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Julio Rubio
- Department of Mathematics and Computer Science, University of La Rioja, Logroño, Spain
| |
Collapse
|
3
|
Huang KT. Integrating computer vision and non-linear optimization for automated deformable registration of 3D medical images. Phys Med Biol 2019; 64:135014. [DOI: 10.1088/1361-6560/ab2202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
4
|
A novel image-based retrieval system for characterization of maxillofacial lesions in cone beam CT images. Int J Comput Assist Radiol Surg 2019; 14:785-796. [DOI: 10.1007/s11548-019-01946-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2018] [Accepted: 03/08/2019] [Indexed: 10/27/2022]
|
5
|
Huang Q, Zeng Z, Li X. 2.5-D Extended Field-of-View Ultrasound. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:851-859. [PMID: 29610066 DOI: 10.1109/tmi.2017.2776971] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Recently, the growing emphasis on medical ultrasound (US) has led to a rapid development of US extended field-of-view (EFOV) techniques. US EFOV techniques can be classified into three categories: 2-D US EFOV, 3-D US, and 3-D US EFOV. In this paper, we propose a novel EFOV method called 2.5-D US EFOV that combines both the advantages of the 2-D US EFOV and the 3-D US by generating a panorama on a curved image plane guided by a curved scanning trajectory of the US probe. In 2.5-D US EFOV, the real-time position and orientation of the US image plane can be recorded via an electromagnetic spatial sensor attached to the probe. The scanning direction is not necessarily straight and can be curved according to the regions of interest (ROI). To form the curved panorama, an image cutting method is proposed. Finally, the curved panorama is rendered in a 3-D space using a surface rendering based on a texture mapping technique. This allows 3-D measurements of lines and angles. Phantom experiments demonstrated that 2.5-D US EFOV images could show anatomical structures of ROI accurately and rapidly. The overall average errors for the distance and angle measurements are -0.097 ± 0.128 cm (-1% ± 1.2%) and 1.50° ± 1.60° (1.9% ± 2%), respectively. A typical extended US image can be reconstructed from 321 B-scans images within 3 s. The satisfying quantitative result on the spinal tissues of a scoliosis subject demonstrates that our system has potential applications in the assessment of musculoskeletal issues.
Collapse
|
6
|
Al-Khafaji SL, Zia A, Liew AWC. Spectral-Spatial Scale Invariant Feature Transform for Hyperspectral Images. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2018; 27:837-850. [PMID: 28880176 DOI: 10.1109/tip.2017.2749145] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Spectral-spatial feature extraction is an important task in hyperspectral image processing. In this paper we propose a novel method to extract distinctive invariant features from hyperspectral images for registration of hyperspectral images with different spectral conditions. Spectral condition means images are captured with different incident lights, viewing angles, or using different hyperspectral cameras. In addition, spectral condition includes images of objects with the same shape but different materials. This method, which is named spectral-spatial scale invariant feature transform (SS-SIFT), explores both spectral and spatial dimensions simultaneously to extract spectral and geometric transformation invariant features. Similar to the classic SIFT algorithm, SS-SIFT consists of keypoint detection and descriptor construction steps. Keypoints are extracted from spectral-spatial scale space and are detected from extrema after 3D difference of Gaussian is applied to the data cube. Two descriptors are proposed for each keypoint by exploring the distribution of spectral-spatial gradient magnitude in its local 3D neighborhood. The effectiveness of the SS-SIFT approach is validated on images collected in different light conditions, different geometric projections, and using two hyperspectral cameras with different spectral wavelength ranges and resolutions. The experimental results show that our method generates robust invariant features for spectral-spatial image matching.
Collapse
|
7
|
Mozaffari MH, Lee WS. Freehand 3-D Ultrasound Imaging: A Systematic Review. ULTRASOUND IN MEDICINE & BIOLOGY 2017; 43:2099-2124. [PMID: 28716431 DOI: 10.1016/j.ultrasmedbio.2017.06.009] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2017] [Revised: 06/01/2017] [Accepted: 06/05/2017] [Indexed: 05/20/2023]
Abstract
Two-dimensional ultrasound (US) imaging has been successfully used in clinical applications as a low-cost, portable and non-invasive image modality for more than three decades. Recent advances in computer science and technology illustrate the promise of the 3-D US modality as a medical imaging technique that is comparable to other prevalent modalities and that overcomes certain drawbacks of 2-D US. This systematic review covers freehand 3-D US imaging between 1970 and 2017, highlighting the current trends in research fields, the research methods, the main limitations, the leading researchers, standard assessment criteria and clinical applications. Freehand 3-D US systems are more prevalent in the academic environment, whereas in clinical applications and industrial research, most studies have focused on 3-D US transducers and improvement of hardware performance. This topic is still an interesting active area for researchers, and there remain many unsolved problems to be addressed.
Collapse
Affiliation(s)
- Mohammad Hamed Mozaffari
- School of Electrical Engineering and Computer Science (EECS), University of Ottawa, Ottawa, Ontario, Canada.
| | - Won-Sook Lee
- School of Electrical Engineering and Computer Science (EECS), University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
8
|
Che C, Mathai TS, Galeotti J. Ultrasound registration: A review. Methods 2017; 115:128-143. [DOI: 10.1016/j.ymeth.2016.12.006] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2016] [Revised: 12/07/2016] [Accepted: 12/08/2016] [Indexed: 11/29/2022] Open
|
9
|
A Markov random field approach to group-wise registration/mosaicing with application to ultrasound. Med Image Anal 2015; 24:106-124. [PMID: 26142928 DOI: 10.1016/j.media.2015.05.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2014] [Revised: 05/25/2015] [Accepted: 05/26/2015] [Indexed: 11/24/2022]
|
10
|
Gwo CY, Wei CH, Li Y, Chiu NH. Reconstruction of Banknote Fragments Based on Keypoint Matching Method. J Forensic Sci 2015; 60:906-13. [PMID: 25903323 DOI: 10.1111/1556-4029.12777] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2013] [Revised: 07/23/2014] [Accepted: 08/06/2014] [Indexed: 11/29/2022]
Abstract
Banknotes may be shredded by a scrap machine, ripped up by hand, or damaged in accidents. This study proposes an image registration method for reconstruction of multiple sheets of banknotes. The proposed method first constructs different scale spaces to identify keypoints in the underlying banknote fragments. Next, the features of those keypoints are extracted to represent their local patterns around keypoints. Then, similarity is computed to find the keypoint pairs between the fragment and the reference banknote. The banknote fragments can determine the coordinate and amend the orientation. Finally, an assembly strategy is proposed to piece multiple sheets of banknote fragments together. Experimental results show that the proposed method causes, on average, a deviation of 0.12457 ± 0.12810° for each fragment while the SIFT method deviates 1.16893 ± 2.35254° on average. The proposed method not only reconstructs the banknotes but also decreases the computing cost. Furthermore, the proposed method can estimate relatively precisely the orientation of the banknote fragments to assemble.
Collapse
Affiliation(s)
- Chih-Ying Gwo
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan
| | - Chia-Hung Wei
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan.,Graduate Institute of Biomedical Informatics, Taipei Medical University, 250 Wuxing Street, Taipei City, 110, Taiwan
| | - Yue Li
- College of Software, Nankai University, 94 Weijin Road, Nankai District, 300071, China
| | - Nan-Hsing Chiu
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan
| |
Collapse
|
11
|
Banerjee J, Klink C, Peters ED, Niessen WJ, Moelker A, van Walsum T. Fast and robust 3D ultrasound registration – Block and game theoretic matching. Med Image Anal 2015; 20:173-83. [DOI: 10.1016/j.media.2014.11.004] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2014] [Revised: 11/03/2014] [Accepted: 11/08/2014] [Indexed: 11/30/2022]
|
12
|
Dyer E, Zeeshan Ijaz U, Housden R, Prager R, Gee A, Treece G. A clinical system for three-dimensional extended-field-of-view ultrasound. Br J Radiol 2012; 85:e919-24. [PMID: 22972979 DOI: 10.1259/bjr/46007369] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVE This work is concerned with the creation of three-dimensional (3D) extended-field-of-view ultrasound from a set of volumes acquired using a mechanically swept 3D probe. 3D volumes of ultrasound data can be registered by attaching a position sensor to the probe; this can be an inconvenience in a clinical setting. A position sensor can also cause some misalignment due to patient movement and respiratory motion. We propose a combination of three-degrees-of-freedom image registration and an unobtrusively integrated inertial sensor for measuring orientation. The aim of this research is to produce a reliable and portable ultrasound system that is able to register 3D volumes quickly, making it suitable for clinical use. METHOD As part of a feasibility study we recruited 28 pregnant females attending for routine obstetric scans to undergo 3D extended-field-of-view ultrasound. A total of 49 data sets were recorded. Each registered data set was assessed for correct alignment of each volume by two independent observers. RESULTS In 77-83% of the data sets more than four consecutive volumes registered. The successful registration relies on good overlap between volumes and is adversely affected by advancing gestational age and foetal movement. CONCLUSION The development of reliable 3D extended-field-of-view ultrasound may help ultrasound practitioners to demonstrate the anatomical relation of pathology and provide a convenient way to store data.
Collapse
Affiliation(s)
- E Dyer
- Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK.
| | | | | | | | | | | |
Collapse
|
13
|
|
14
|
Koolwal AB, Barbagli F, Carlson CR, Liang DH. A fast slam approach to freehand 3-d ultrasound reconstruction for catheter ablation guidance in the left atrium. ULTRASOUND IN MEDICINE & BIOLOGY 2011; 37:2037-2054. [PMID: 22014856 DOI: 10.1016/j.ultrasmedbio.2011.08.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2010] [Revised: 07/31/2011] [Accepted: 08/09/2011] [Indexed: 05/31/2023]
Abstract
We present a method for real-time, freehand 3D ultrasound (3D-US) reconstruction of moving anatomy, with specific application towards guiding the catheter ablation procedure in the left atrium. Using an intracardiac echo (ICE) catheter with a pose (position/orientation) sensor mounted to its tip, we continually mosaic 2D-ICE images of a left atrium phantom model to form a 3D-US volume. Our mosaicing strategy employs a probabilistic framework based on simultaneous localization and mapping (SLAM), a technique commonly used in mobile robotics for creating maps of unexplored environments. The measured ICE catheter tip pose provides an initial estimate for compounding 2D-ICE image data into the 3D-US volume. However, we simultaneously consider the overlap-consistency shared between 2D-ICE images and the 3D-US volume, computing a "corrected" tip pose if need be to ensure spatially-consistent reconstruction. This allows us to compensate for anatomic movement and sensor drift that would otherwise cause motion artifacts in the 3D-US volume. Our approach incorporates 2D-ICE data immediately after acquisition, allowing us to continuously update the registration parameters linking sensor coordinates to 3D-US coordinates. This, in turn, enables real-time localization and display of sensorized therapeutic catheters within the 3D-US volume for facilitating procedural guidance.
Collapse
Affiliation(s)
- Aditya B Koolwal
- Department of Mechanical Engineering, Stanford University, Stanford, CA, USA.
| | | | | | | |
Collapse
|
15
|
|
16
|
Nonrigid Registration of Brain Tumor Resection MR Images Based on Joint Saliency Map and Keypoint Clustering. SENSORS 2009; 9:10270-90. [PMID: 22303173 PMCID: PMC3267221 DOI: 10.3390/s91210270] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/23/2009] [Revised: 12/01/2009] [Accepted: 12/09/2009] [Indexed: 11/25/2022]
Abstract
This paper proposes a novel global-to-local nonrigid brain MR image registration to compensate for the brain shift and the unmatchable outliers caused by the tumor resection. The mutual information between the corresponding salient structures, which are enhanced by the joint saliency map (JSM), is maximized to achieve a global rigid registration of the two images. Being detected and clustered at the paired contiguous matching areas in the globally registered images, the paired pools of DoG keypoints in combination with the JSM provide a useful cluster-to-cluster correspondence to guide the local control-point correspondence detection and the outlier keypoint rejection. Lastly, a quasi-inverse consistent deformation is smoothly approximated to locally register brain images through the mapping the clustered control points by compact support radial basis functions. The 2D implementation of the method can model the brain shift in brain tumor resection MR images, though the theory holds for the 3D case.
Collapse
|