2
|
Peng W, Wu W, Zhang J, Xie H, Zhang S, Gu L. An automatic framework for estimating the pose of the catheter distal section using a coarse-to-fine network. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 225:107036. [PMID: 35905696 DOI: 10.1016/j.cmpb.2022.107036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 06/22/2022] [Accepted: 07/20/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE During percutaneous coronary intervention procedures, generally only 2D X-ray images are provided. The consequent lack of depth perception makes it difficult for interventionists to visually estimate the pose of medical tools inside the vasculature, especially for novices. Although some automatic methods have been developed to aid interventionists, it is still a challenging task to obtain stable and accurate pose estimation. In this paper, we describe a learning-based framework for estimating the pose of the catheter distal section (CDS). The main innovation of this framework is the proposal of a coarse-to-fine fusion network (CFF-Net) which can achieve the shape and orientation estimation for the CDS. METHODS By adopting a two-step fusion, CFF-Net progressively solves the shape and orientation ambiguities. The first step is the early fusion where the 2D projection image fuses with the shape prior before input, which makes the estimated result own a specific catheter distal shape. The second step is the late fusion where CFF-Net fuse feature maps and the orientation data from Electromagnetic (EM) sensors to confirm the overall orientation of the CDS. Finally, the estimated pose in the EM space will be obtained after we combine the estimated shape and orientation from CFF-Net with the position information from the EM sensor. RESULTS The effectiveness of CFF-Net has been verified in a simulated environment where RMSE of CFF-Net is 0.706 ± 0.121 mm. This approach was further transferred from simulation to reality using the real-world data, where RMSE of CFF-Net is 1.121 ± 0.124 mm and RMSE of the whole proposed framework is 1.577 ± 0.144 mm. CONCLUSION In simulated and real-world experiments, our proposed approach has been proven to achieve high accuracy while ensuring real-time processing for estimating the pose of the CDS.
Collapse
Affiliation(s)
- Wenjia Peng
- School of Biomedical Engineering and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Wei Wu
- School of Biomedical Engineering and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Jingyang Zhang
- School of Biomedical Engineering and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Hongzhi Xie
- Department of Cardiology, Peking Union Medical College Hospital, Peking, China.
| | - Shuyang Zhang
- Department of Cardiology, Peking Union Medical College Hospital, Peking, China
| | - Lixu Gu
- School of Biomedical Engineering and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
3
|
Bertsche D, Rasche V, Rottbauer W, Vernikouskaya I. 3D localization from 2D X-ray projection. Int J Comput Assist Radiol Surg 2022; 17:1553-1558. [PMID: 35819654 PMCID: PMC9463320 DOI: 10.1007/s11548-022-02709-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 06/21/2022] [Indexed: 11/28/2022]
Abstract
PURPOSE Most cardiology procedures are guided using X-ray (XR) fluoroscopy. However, the projective nature of the XR fluoroscopy does not allow for true depth perception as required for safe and efficient intervention guidance in structural heart diseases. For improving guidance, different methods have been proposed often being radiation-intensive, time-consuming, or expensive. We propose a simple 3D localization method based on a single monoplane XR projection using a co-registered centerline model. METHODS The method is based on 3D anatomic surface models and corresponding centerlines generated from preprocedural imaging. After initial co-registration, 2D working points identified in monoplane XR projections are localized in 3D by minimizing the angle between the projection lines of the centerline points and the working points. The accuracy and reliability of the located 3D positions were assessed in 3D using phantom data and in patient data projected to 2D obtained during placement of embolic protection system in interventional procedures. RESULTS With the proposed methods, 2D working points identified in monoplane XR could be successfully located in the 3D phantom and in the patient-specific 3D anatomy. Accuracy in the phantom (3D) resulted in 1.6 mm (± 0.8 mm) on average, and 2.7 mm (± 1.3 mm) on average in the patient data (2D). CONCLUSION The use of co-registered centerline models allows reliable and accurate 3D localization of devices from a single monoplane XR projection during placement of the embolic protection system in TAVR. The extension to different vascular interventions and combination with automatic methods for device detection and registration might be promising.
Collapse
Affiliation(s)
- Dagmar Bertsche
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany
| | - Volker Rasche
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany
| | - Wolfgang Rottbauer
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany
| | - Ina Vernikouskaya
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| |
Collapse
|
4
|
Cheng R, Crouzier M, Hug F, Tucker K, Juneau P, McCreedy E, Gandler W, McAuliffe MJ, Sheehan FT. Automatic quadriceps and patellae segmentation of MRI with cascaded U 2 -Net and SASSNet deep learning model. Med Phys 2022; 49:443-460. [PMID: 34755359 PMCID: PMC8758556 DOI: 10.1002/mp.15335] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Revised: 10/18/2021] [Accepted: 10/20/2021] [Indexed: 01/03/2023] Open
Abstract
PURPOSE Automatic muscle segmentation is critical for advancing our understanding of human physiology, biomechanics, and musculoskeletal pathologies, as it allows for timely exploration of large multi-dimensional image sets. Segmentation models are rarely developed/validated for the pediatric model. As such, autosegmentation is not available to explore how muscle architectural changes during development and how disease/pathology affects the developing musculoskeletal system. Thus, we aimed to develop and validate an end-to-end, fully automated, deep learning model for accurate segmentation of the rectus femoris and vastus lateral, medialis, and intermedialis using a pediatric database. METHODS We developed a two-stage cascaded deep learning model in a coarse-to-fine manner. In the first stage, the U2 -Net roughly detects the muscle subcompartment region. Then, in the second stage, the shape-aware 3D semantic segmentation method SASSNet refines the cropped target regions to generate the more finer and accurate segmentation masks. We utilized multifeature image maps in both stages to stabilize performance and validated their use with an ablation study. The second-stage SASSNet was independently run and evaluated with three different cropped region resolutions: the original image resolution, and images downsampled 2× and 4× (high, mid, and low). The relationship between image resolution and segmentation accuracy was explored. In addition, the patella was included as a comparator to past work. We evaluated segmentation accuracy using leave-one-out testing on a database of 3D MR images (0.43 × 0.43 × 2 mm) from 40 pediatric participants (age 15.3 ± 1.9 years, 55.8 ± 11.8 kg, 164.2 ± 7.9 cm, 38F/2 M). RESULTS The mid-resolution second stage produced the best results for the vastus medialis, rectus femoris, and patella (Dice similarity coefficient = 95.0%, 95.1%, 93.7%), whereas the low-resolution second stage produced the best results for the vastus lateralis and vastus intermedialis (DSC = 94.5% and 93.7%). In comparing the low- to mid-resolution cases, the vasti intermedialis, vastus medialis, rectus femoris, and patella produced significant differences (p = 0.0015, p = 0.0101, p < 0.0001, p = 0.0003) and the vasti lateralis did not (p = 0.2177). The high-resolution stage 2 had significantly lower accuracy (1.0 to 4.4 dice percentage points) compared to both the mid- and low-resolution routines (p value ranged from < 0.001 to 0.04). The one exception was the rectus femoris, where there was no difference between the low- and high-resolution cases. The ablation study demonstrated that the multifeature is more reliable than the single feature. CONCLUSIONS Our successful implementation of this two-stage segmentation pipeline provides a critical tool for expanding pediatric muscle physiology and clinical research. With a relatively small and variable dataset, our fully automatic segmentation technique produces accuracies that matched or exceeded the current state of the art. The two-stage segmentation avoids memory issues and excessive run times by using a first stage focused on cropping out unnecessary data. The excellent Dice similarity coefficients improve upon previous template-based automatic and semiautomatic methodologies targeting the leg musculature. More importantly, with a naturally variable dataset (size, shape, etc.), the proposed model demonstrates slightly improved accuracies, compared to previous neural networks methods.
Collapse
Affiliation(s)
- Ruida Cheng
- Scientific Application Services (SAS), Office of Scientific Computing Services (OSCS), Office of Intramural Research, Center of Information Technology, NIH, Bethesda, MD, USA
| | - Marion Crouzier
- University of Nantes, Movement, Interactions, Performance, MIP, EA 4334, F-44000 Nantes, France,The University of Queensland, School of Biomedical Sciences, Brisbane
| | - François Hug
- Institut Universitaire de France (IUF), Paris, France,Université Côte d’Azur, LAMHESS, Nice, France
| | - Kylie Tucker
- The University of Queensland, School of Biomedical Sciences, Brisbane
| | - Paul Juneau
- NIH Library, Office of Research Services, National Institutes of Health, Bethesda, MD, USA
| | - Evan McCreedy
- Scientific Application Services (SAS), Office of Scientific Computing Services (OSCS), Office of Intramural Research, Center of Information Technology, NIH, Bethesda, MD, USA
| | - William Gandler
- Scientific Application Services (SAS), Office of Scientific Computing Services (OSCS), Office of Intramural Research, Center of Information Technology, NIH, Bethesda, MD, USA
| | - Matthew J. McAuliffe
- Scientific Application Services (SAS), Office of Scientific Computing Services (OSCS), Office of Intramural Research, Center of Information Technology, NIH, Bethesda, MD, USA
| | - Frances T. Sheehan
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, MD, USA
| |
Collapse
|