Supervised recursive segmentation of volumetric CT images for 3D reconstruction of lung and vessel tree

Survey of methods and principles in three-dimensional reconstruction from two-dimensional medical images

Three-dimensional (3D) reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units. In the coming years, most patient care will shift toward this new paradigm. However, development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved, most of which are dependent on human expertise. In this review, a survey of pre-processing steps was conducted, and reconstruction techniques for several organs in medical diagnosis were studied. Various methods and principles related to 3D reconstruction were highlighted. The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.

An atlas of anatomical variants of subsegmental pulmonary arteries and recognition error analysis

Background

Surgery, including lobectomy and segmentectomy, is the major curative intervention for lung cancer. Surgical planning for pulmonary surgery is difficult due to the high variation rate of pulmonary arteries and needs a fine-grained atlas as a reference. We conducted a study to create a surgically oriented atlas and analyzed the error encountered during the production.

Method

A total of 100 Chest CTs performed at Peking University People’s Hospital from 2013.09 to 2020.10 were randomly selected for segmental artery labeling. Dicom files were collected for 3D reconstruction. Manual segmentation of each segmental artery was performed by 4 thoracic surgeons. Cross-validation by surgeons was performed to establish the golden standard based on their consensus. Initial recognition errors were recorded accordingly.

Result

The most frequently seen variants for the right upper lobe is 2-branch RA1+2rec+3 and RA2asc; right middle lobe 2-branch RA4a and RA4b+5; right lower lobe 3-branch RA7, RA8 and RA9+10; left upper lobe 3-branch LA1+2a+3, LA1+2b, LA1+2c and 1-branch LA4+5; left lower lobe 2-branch LA8 and LA9+10. Top 5 segmental error occurs in RA4 (23%), LA8 (17%), RA9 (17%), RA8 (14%) and LA9 (11%). A rapid surgical planning tool form was created based on high frequency anatomic variants.

Conclusion

Our research provided an atlas for lobectomy and segmentectomy at the subsegmental or more distal level. We demonstrated that the recognition accuracy of pulmonary arteries in a non-time-sensitive experimental scenario was still unfavorable. We also suggest that extra attention should be paid to certain surgeries during the surgical planning process.

Lung nodule pre-diagnosis and insertion path planning for chest CT images

Medical image processing has proven to be effective and feasible for assisting oncologists in diagnosing lung, thyroid, and other cancers, especially at early stage. However, there is no reliable method for the recognition, screening, classification, and detection of nodules, and even deep learning-based methods have limitations. In this study, we mainly explored the automatic pre-diagnosis of lung nodules with the aim of accurately identifying nodules in chest CT images, regardless of the benign and malignant nodules, and the insertion path planning of suspected malignant nodules, used for further diagnosis by robotic-based biopsy puncture. The overall process included lung parenchyma segmentation, classification and pre-diagnosis, 3-D reconstruction and path planning, and experimental verification. First, accurate lung parenchyma segmentation in chest CT images was achieved using digital image processing technologies, such as adaptive gray threshold, connected area labeling, and mathematical morphological boundary repair. Multi-feature weight assignment was then adopted to establish a multi-level classification criterion to complete the classification and pre-diagnosis of pulmonary nodules. Next, 3-D reconstruction of lung regions was performed using voxelization, and on its basis, a feasible local optimal insertion path with an insertion point could be found by avoiding sternums and/or key tissues in terms of the needle-inserting path. Finally, CT images of 900 patients from Lung Image Database Consortium and Image Database Resource Initiative were chosen to verify the validity of pulmonary nodule diagnosis. Our previously designed surgical robotic system and a custom thoracic model were used to validate the effectiveness of the insertion path. This work can not only assist doctors in completing the pre-diagnosis of pulmonary nodules but also provide a reference for clinical biopsy puncture of suspected malignant nodules considered by doctors.

Towards Machine Learning-Aided Lung Cancer Clinical Routines: Approaches and Open Challenges

Advancements in the development of computer-aided decision (CAD) systems for clinical routines provide unquestionable benefits in connecting human medical expertise with machine intelligence, to achieve better quality healthcare. Considering the large number of incidences and mortality numbers associated with lung cancer, there is a need for the most accurate clinical procedures; thus, the possibility of using artificial intelligence (AI) tools for decision support is becoming a closer reality. At any stage of the lung cancer clinical pathway, specific obstacles are identified and “motivate” the application of innovative AI solutions. This work provides a comprehensive review of the most recent research dedicated toward the development of CAD tools using computed tomography images for lung cancer-related tasks. We discuss the major challenges and provide critical perspectives on future directions. Although we focus on lung cancer in this review, we also provide a more clear definition of the path used to integrate AI in healthcare, emphasizing fundamental research points that are crucial for overcoming current barriers.

Thoracoscopic segmentectomy and lobectomy assisted by three-dimensional computed-tomography bronchography and angiography for the treatment of primary lung cancer

BACKGROUND

Anatomical segmentectomy has been proposed as a substitution for lobectomy for early-stage lung cancer. However, it requires technical meticulousness due to the complex anatomical variations of segmental vessels and bronchi.

AIM

To assess the safety and feasibility of three-dimensional computed-tomography bronchography and angiography (3D-CTBA) in performing video-assisted thoracoscopic surgery (VATS) for lung cancers.

METHODS

In this study, we enrolled 123 patients who consented to undergo thoracoscopic segmentectomy and lobectomy assisted by 3D-CTBA between May 2017 and June 2019. The image data of enhanced computed tomography (CT) scans was reconstructed three-dimensionally by the Mimics software. The results of preoperative 3D-CTBA, in combination with intraoperative navigation, guided the surgery.

RESULTS

A total of 59 women and 64 men were enrolled, of whom 57 (46.3%) underwent segmentectomy and 66 (53.7%) underwent lobectomy. The majority of tumor appearance on CT was part-solid ground-glass nodule (pGGN; 55.3%). The mean duration of chest tube placement was 3.5 ± 1.6 d, and the average length of postoperative hospital stay was 6.8 ± 1.8 d. Surgical complications included one case of pneumonia and four cases of prolonged air leak lasting > 5 d. Notably, there was no intraoperative massive hemorrhage, postoperative intensive-care unit stay, or 30-d mortality. Preoperative 3D-CTBA images can display clearly and vividly the targeted structure and the variations of vessels and bronchi. To reduce the risk of locoregional recurrence, the application of 3D-CTBA with a virtual 3D surgical margin help the VATS surgeon determine accurate distances and positional relations among the tumor, bronchial trees, and the intersegmental vessels. Three-dimensional navigation was performed to confirm the segmental structure, precisely cut off the targeted segment, and avoid intersegmental veins injury.

CONCLUSION

VATS and 3D-CTBA worked in harmony in our study. This combination also provided a new pattern of transition from lesion-directed location of tumors to computer-aided surgery for the management of early lung cancer.

A morphing-Based 3D point cloud reconstruction framework for medical image processing

BACKGROUND AND OBJECTIVE

In the virtual surgery simulation system, the reconstruction of a highly precise soft tissue 3D model is an effective method to improve the user's visual telepresence. However, the traditional point cloud generation method based on subdivision and filling is unsatisfactory due to its low accuracy and slow speed.

METHODS

To address this problem, we present a novel 3D point cloud reconstructing model based on Morphing. The 3D surface model of soft tissue (live) is obtained from a series of 2D CT images using Mimics. The 3D voxel model of soft tissue is reconstructed through a sequential change of the 3D surface model by utilizing Morphing. A nonlinear interpolation method is used to fit the irregular shape of the model and improve simulation accuracy.

RESULTS

The point cloud model builds from discrete points, avoiding the problems of instability and computational complexity, which are inherent in both the surface and volume models for soft tissue. Compared with the volumetric subdividing and voxel filling method, the simulation results show that the 3D cloud model reconstructed based on Morphing is more fast, accurate and consistent with the real soft tissue.

CONCLUSIONS

The simulating experiment of soft tissue deformation using 3D point cloud model which reconstructed using moprhing proved our method is effective and correct.

An Approach for Pulmonary Vascular Extraction from Chest CT Images

Pulmonary vascular extraction from chest CT images plays an important role in the diagnosis of lung disease. To improve the accuracy rate of pulmonary vascular segmentation, a new pulmonary vascular extraction approach is proposed in this study. First, the lung tissue is extracted from chest CT images by region-growing and maximum between-class variance methods. Then the holes of the extracted region are filled by morphological operations to obtain complete lung region. Second, the points of the pulmonary vascular of the middle slice of the chest CT images are extracted as the original seed points. Finally, the seed points are spread throughout the lung region based on the fast marching method to extract the pulmonary vascular in the gradient image. Results of pulmonary vascular extraction from chest CT image datasets provided by the introduced approach are presented and discussed. Based on the ground truth pixels and the resulting quality measures, it can be concluded that the average accuracy of this approach is about 90%. Extensive experiments demonstrate that the proposed method has achieved the best performance in pulmonary vascular extraction compared with other two widely used methods.

Semi-automatic segmentation of femur based on harmonic barrier

BACKGROUND AND OBJECTIVE

Segmentation of the femur from the hip joint in computed tomography (CT) is an important preliminary step in hip surgery planning and simulation. However, this is a time-consuming and challenging task due to the weak boundary, the varying topology of the hip joint, and the extremely narrow or blurred space between the femoral head and the acetabulum. To address these problems, this study proposed a semi-automatic segmentation framework based on harmonic fields for accurate segmentation.

METHODS

The proposed method comprises three steps. First, with high-level information provided by the user, shape information provided by neighboring slices as well as the statistical information in the mask, a region selection method is proposed to effectively locate joint space for the harmonic field. Second, incorporated with an improved gradient, the harmonic field is used to adaptively extract a curve as the barrier that separates the femoral head from the acetabulum accurately. Third, a divide and conquer segmentation strategy based on the harmonic barrier is used to combine the femoral head part and body part as the final segmentation result.

RESULTS

We have tested 40 hips with considerately narrow or disappeared joint spaces. The experimental results are evaluated based on Jaccard, Dice, directional cut discrepancy (DCD) and receiver operating characteristic (ROC), and we achieve the higher Jaccard of 84.02%, Dice of 85.96%, area under curve (AUC) of 89.3%, and the lower error with DCD of 0.52mm. The effective ratio of our method is 79.1% even for cases with severe malformation. The results show that our method performs best in terms of effectiveness and accuracy on the whole data set.

CONCLUSIONS

The proposed method is efficient to segment femurs with narrow joint space. The accurate segmentation results can assist the physicians for osteoarthritis diagnosis in future.

Automatic Liver Segmentation from CT Images Using Single-Block Linear Detection

Automatic liver segmentation not only plays an important role in the analysis of liver disease, but also reduces the cost and humanity's impact in segmentation. In addition, liver segmentation is a very challenging task due to countless anatomical variations and technical difficulties. Many methods have been designed to overcome these challenges, but these methods still need to be improved to obtain the desired segmentation precision. In this paper, a fast algorithm is proposed for liver extraction from CT images with single-block linear detection. The proposed method does not require iteration; thus, the computational time and complexity are decreased enormously. In addition, the initialization is not crucial in the algorithm, so the algorithm's robustness and specificity are improved. The experimental evaluation of the proposed method revealed effective segmentation in normal and abnormal (liver hemangioma and liver cancer) abdominal CT images. The average sensitivity, accuracy, and specificity for liver cancer are 96.59%, 98.65%, and 99.03%, respectively. The results of image segmentation approximate the manual segmentation results by the technical doctor. Moreover, our method shows superior flexibility to newly published method with comparable performance. The advantage of our method is verified with experimental results, which is described in detail.

Three-dimensional lung nodule segmentation and shape variance analysis to detect lung cancer with reduced false positives

The three-dimensional analysis on lung computed tomography scan was carried out in this study to detect the malignant lung nodules. An automatic three-dimensional segmentation algorithm proposed here efficiently segmented the tissue clusters (nodules) inside the lung. However, an automatic morphological region-grow segmentation algorithm that was implemented to segment the well-circumscribed nodules present inside the lung did not segment the juxta-pleural nodule present on the inner surface of wall of the lung. A novel edge bridge and fill technique is proposed in this article to segment the juxta-pleural and pleural-tail nodules accurately. The centroid shift of each candidate nodule was computed. The nodules with more centroid shift in the consecutive slices were eliminated since malignant nodule’s resultant position did not usually deviate. The three-dimensional shape variation and edge sharp analyses were performed to reduce the false positives and to classify the malignant nodules. The change in area and equivalent diameter was more for malignant nodules in the consecutive slices and the malignant nodules showed a sharp edge. Segmentation was followed by three-dimensional centroid, shape and edge analysis which was carried out on a lung computed tomography database of 20 patient with 25 malignant nodules. The algorithms proposed in this article precisely detected 22 malignant nodules and failed to detect 3 with a sensitivity of 88%. Furthermore, this algorithm correctly eliminated 216 tissue clusters that were initially segmented as nodules; however, 41 non-malignant tissue clusters were detected as malignant nodules. Therefore, the false positive of this algorithm was 2.05 per patient.