Assessment of real-time 3D visualization for cardiothoracic diagnostic evaluation and surgery planning

Virtual Reality in the Preoperative Planning of Adult Aortic Surgery: A Feasibility Study

Background: Complex aortic anatomy needs careful preoperative planning in which a patient-tailored approach with novel immersive techniques could serve as a valuable addition to current preoperative imaging. This pilot study aimed to investigate the technical feasibility of virtual reality (VR) as an additional imaging tool for preoperative planning in ascending aortic surgery. Methods: Ten cardiothoracic surgeons were presented with six patients who had each undergone a recent repair of the ascending aorta. Two-dimensional computed tomography images of each patient were assessed prior to the VR session. After three-dimensional (3D) VR rendering and 3D segmentation of the ascending aorta and aortic arch, the reconstructions were analyzed by each surgeon in VR via a head-mounted display. Each cardiothoracic surgeon completed a questionnaire after each planning procedure. The results of their assessments were compared to the performed operations. The primary endpoint of the present study was a change of surgical approach from open to clamped distal anastomosis, and vice versa. Results: Compared with conventional imaging, 80% of surgeons found that VR prepared them better for surgery. In 33% of cases (two out of six), the preoperative decision was adjusted due to the 3D VR-based evaluation of the anatomy. Surgeons rated CardioVR usefulness, user-friendliness, and satisfaction with median scores of 3.8 (IQR: 3.5-4.1), 4.2 (IQR: 3.8-4.6,) and 4.1 (IQR: 3.8-4.7) on a five-point Likert scale, respectively. Conclusions: Three-dimensional VR imaging was associated with improved anatomical understanding among surgeons and could be helpful in the future preoperative planning of ascending aortic surgery.

COVID-view: Diagnosis of COVID-19 using Chest CT

Significant work has been done towards deep learning (DL) models for automatic lung and lesion segmentation and classification of COVID-19 on chest CT data. However, comprehensive visualization systems focused on supporting the dual visual+DL diagnosis of COVID-19 are non-existent. We present COVID-view, a visualization application specially tailored for radiologists to diagnose COVID-19 from chest CT data. The system incorporates a complete pipeline of automatic lungs segmentation, localization/isolation of lung abnormalities, followed by visualization, visual and DL analysis, and measurement/quantification tools. Our system combines the traditional 2D workflow of radiologists with newer 2D and 3D visualization techniques with DL support for a more comprehensive diagnosis. COVID-view incorporates a novel DL model for classifying the patients into positive/negative COVID-19 cases, which acts as a reading aid for the radiologist using COVID-view and provides the attention heatmap as an explainable DL for the model output. We designed and evaluated COVID-view through suggestions, close feedback and conducting case studies of real-world patient data by expert radiologists who have substantial experience diagnosing chest CT scans for COVID-19, pulmonary embolism, and other forms of lung infections. We present requirements and task analysis for the diagnosis of COVID-19 that motivate our design choices and results in a practical system which is capable of handling real-world patient cases.

Current trends in thoracic surgery

Thoracic surgery has evolved drastically in recent years. Although thoracic surgeons mainly deal with tumorous lesion in the lungs, mediastinum, and pleura, they also perform lung transplantation surgery in patients with end-stage lung disease. Herein, we introduce various major current topics in thoracic surgery. Minimally invasive surgical procedures include robot-assisted thoracic surgery and uniportal video-assisted thoracic surgery. Novel techniques for sublobar resection include virtual-assisted lung mapping, image-guided video-assisted thoracic surgery, and segmentectomy using indocyanine green. Three-dimensional (3D) computed tomography (CT) simulation consists of surgeon-friendly 3D-CT image analysis systems and new-generation, dynamic 3D-CT imaging systems. Updates in cadaveric lung transplantation include use of marginal donors, including donation after circulatory death, and ex vivo lung perfusion for such donors. Topics in living donor lobar lung transplantation include size matching, donor issues, and new surgical techniques. During routine clinical practice, thoracic surgeons encounter various pivotal topics related to thoracic surgery, which are described in this report.

Three-dimensional image in lung transplantation

Three-dimensional computed tomography (3D-CT) technologies have been developed and, recently, high-speed and high-quality 3D-CT technologies have been introduced to the field of thoracic surgery. The purpose of this manuscript is to demonstrate the clinical application of 3D-CT technologies in lung transplantation. In Japan, because of the severe donor shortage, living-donor lobar lung transplantation (LDLLT) is essential, in addition to cadaveric lung transplantation. In LDLLT, size matching is a grave issue, since ideal size matching between donor and recipient is usually difficult because of the limited population of potential donor. Size matching using pulmonary function test results has been widely used as a gold standard, but anatomical size matching using 3D-CT volumetry data has also been utilized in LDLLT. In donor lobectomy, 3D-CT images provided a variety of information regarding anatomical variation of pulmonary vessels and bronchial trees preoperatively. These images ensure surgical quality and safety, and they also affect surgical procedures for the recipient. 3D-CT images are also utilized in various aspects of postoperative care, such as detection of chronic lung allograft dysfunction and clarification of its subtypes. Furthermore, preoperative 3D-CT simulation is useful in developing and performing a special surgical procedure, such as right-to-left inverted LDLLT. In conclusion, following the introduction of 3D-CT to the field of thoracic surgery, various 3D-CT images and their application to preoperative simulations have been introduced in lung transplantation. In the near future, this technique will become more prevalent, and frequent use by thoracic surgeons will be seen worldwide in daily practice.

A survey among Brazilian thoracic surgeons about the use of preoperative 2D and 3D images

BACKGROUND

Describe the characteristics of how the thoracic surgeon uses the 2D/3D medical imaging to perform surgical planning, clinical practice and teaching in thoracic surgery and check the initial choice and the final choice of the Brazilian Thoracic surgeon as the 2D and 3D models pictures before and after acquiring theoretical knowledge on the generation, manipulation and interactive 3D views.

METHODS

A descriptive research type Survey cross to data provided by the Brazilian Thoracic Surgeons (members of the Brazilian Society of Thoracic Surgery) who responded to the online questionnaire via the internet on their computers or personal devices.

RESULTS

Of the 395 invitations visualized distributed by email, 107 surgeons completed the survey. There was no statically difference when comparing the 2D vs. 3D models pictures for the following purposes: diagnosis, assessment of the extent of disease, preoperative surgical planning, and communication among physicians, resident training, and undergraduate medical education. Regarding the type of tomographic image display routinely used in clinical practice (2D or 3D or 2D-3D model image) and the one preferred by the surgeon at the end of the questionnaire. Answers surgeons for exclusive use of 2D images: initial choice =50.47% and preferably end =14.02%. Responses surgeons to use 3D models in combination with 2D images: initial choice =48.60% and preferably end =85.05%. There was a significant change in the final selection of 3D models used together with the 2D images (P<0.0001).

CONCLUSIONS

There is a lack of knowledge of the 3D imaging, as well as the use and interactive manipulation in dedicated 3D applications, with consequent lack of uniformity in the surgical planning based on CT images. These findings certainly confirm in changing the preference of thoracic surgeons of 2D views of technologies for 3D images.

Supervised recursive segmentation of volumetric CT images for 3D reconstruction of lung and vessel tree

Three dimensional reconstruction of lung and vessel tree has great significance to 3D observation and quantitative analysis for lung diseases. This paper presents non-sheltered 3D models of lung and vessel tree based on a supervised semi-3D lung tissues segmentation method. A recursive strategy based on geometric active contour is proposed instead of the "coarse-to-fine" framework in existing literature to extract lung tissues from the volumetric CT slices. In this model, the segmentation of the current slice is supervised by the result of the previous one slice due to the slight changes between adjacent slice of lung tissues. Through this mechanism, lung tissues in all the slices are segmented fast and accurately. The serious problems of left and right lungs fusion, caused by partial volume effects, and segmentation of pleural nodules can be settled meanwhile during the semi-3D process. The proposed scheme is evaluated by fifteen scans, from eight healthy participants and seven participants suffering from early-stage lung tumors. The results validate the good performance of the proposed method compared with the "coarse-to-fine" framework. The segmented datasets are utilized to reconstruct the non-sheltered 3D models of lung and vessel tree.

A hybrid approach to segmentation of diseased lung lobes

Complete segmentation of diseased lung lobes by automatically identifying fissure surfaces is a nontrivial task, due to incomplete, disrupted, and deformed fissures. In this paper, we present a novel algorithm employing a hybrid two-dimensional/three-dimensional approach for segmenting diseased lung lobes. Our approach models complete fissure surfaces from partial fissures found in individual computed tomography (CT) images. Evaluated using 24 patients' lungs with a variety of different diseases, our algorithm produced root-mean square errors of 2.21 ± 1.21, 2.51 ± 1.36, and 2.38 ± 1.27 mm for segmenting the left oblique fissure (LOF), right oblique fissure (ROF) and right horizontal fissure (RHF), respectively. The average accuracies for segmenting the LOF, ROF, and RHF are 86.59%, 84.80%, and 82.62%, using our ±3-mm percentile measure. These results indicate the feasibility of developing an automatic algorithm for complete segmentation of diseased lung lobes.

A guide to stereoscopic 3D displays in medicine

Stereoscopic displays can potentially improve many aspects of medicine. However, weighing the advantages and disadvantages of such displays remains difficult, and more insight is needed to evaluate whether stereoscopic displays are worth adopting. In this article, we begin with a review of monocular and binocular depth cues. We then apply this knowledge to examine how stereoscopic displays can potentially benefit diagnostic imaging, medical training, and surgery. It is apparent that the binocular depth information afforded by stereo displays 1) aid the detection of diagnostically relevant shapes, orientations, and positions of anatomical features, especially when monocular cues are absent or unreliable; 2) help novice surgeons orient themselves in the surgical landscape and perform complicated tasks; and 3) improve the three-dimensional anatomical understanding of students with low visual-spatial skills. The drawbacks of stereo displays are also discussed, including extra eyewear, potential three-dimensional misperceptions, and the hurdle of overcoming familiarity with existing techniques. Finally, we list suggested guidelines for the optimal use of stereo displays. We provide a concise guide for medical practitioners who want to assess the potential benefits of stereo displays before adopting them.

Identification of lobar fissures in pathological lungs

Identification of lobar fissures in human lungs is a non-trivial task due to their variable shape and appearance, along with the low contrast and high noise in computed tomographic (CT) images. Pathologies in the lungs can further complicate this identification by deforming and/or disrupting the lobar fissures. Current algorithms rely on the general anatomy of the lungs to find fissures affected by pathologies. This can be unreliable as deformations and/or disruptions of these fissures will alter the general lung anatomy. To overcome this, we developed an algorithm with the following novelties: (1) a new application of neural network based texture analysis to generalize fissure regions; and (2) a new method of fissure surface identification. We tested our algorithm on CT image stacks from 8 anonymous patients with pathological lungs. Compared to manually segmented fissures, our algorithm produced an average mean difference of 0.71 mm and 0.68 mm for identifying the left and right oblique fissures, respectively. Using a 3-mm percentile measure, the algorithm yielded an average accuracy of 86.8% for the left oblique fissure with a mean worst-case error of 3.18 mm. For the right oblique fissure, the algorithm produced an accuracy of 88.8% with a mean worst-case error of 3.13 mm. The above results show feasibility of using our algorithm for identifying fissures in pathological lungs.

A study on using texture analysis methods for identifying lobar fissure regions in isotropic CT images

The major hurdle for segmenting lung lobes in computed tomographic (CT) images is to identify fissure regions, which encase lobar fissures. Accurate identification of these regions is difficult due to the variable shape and appearance of the fissures, along with the low contrast and high noise associated with CT images. This paper studies the effectiveness of two texture analysis methods - the gray level co-occurrence matrix (GLCM) and the gray level run length matrix (GLRLM) - in identifying fissure regions from isotropic CT image stacks. To classify GLCM and GLRLM texture features, we applied a feed-forward back-propagation neural network and achieved the best classification accuracy utilizing 16 quantized levels for computing the GLCM and GLRLM texture features and 64 neurons in the input/hidden layers of the neural network. Tested on isotropic CT image stacks of 24 patients with the pathologic lungs, we obtained accuracies of 86% and 87% for identifying fissure regions using the GLCM and GLRLM methods, respectively. These accuracies compare favorably with surgeons/radiologists' accuracy of 80% for identifying fissure regions in clinical settings. This shows promising potential for segmenting lung lobes using the GLCM and GLRLM methods.

Segmentation of lung lobes in high-resolution isotropic CT images

Modern multislice computed tomography (CT) scanners produce isotropic CT images with a thickness of 0.6 mm. These CT images offer detailed information of lung cavities, which could be used for better surgical planning of treating lung cancer. The major challenge for developing a surgical planning system is the automatic segmentation of lung lobes by identifying the lobar fissures. This paper presents a lobe segmentation algorithm that uses a two-stage approach: 1) adaptive fissure sweeping to find fissure regions and 2) wavelet transform to identify the fissure locations and curvatures within these regions. Tested on isotropic CT image stacks from nine anonymous patients with pathological lungs, the algorithm yielded an accuracy of 76.7%-94.8% with strict evaluation criteria. In comparison, surgeons obtain an accuracy of 80% for localizing the fissure regions in clinical CT images with a thickness of 2.5-7.0 mm. As well, this paper describes a procedure for visualizing lung lobes in three dimensions using software--amira--and the segmentation algorithm. The procedure, including the segmentation, needed about 5 min for each patient. These results provide promising potential for developing an automatic algorithm to segment lung lobes for surgical planning of treating lung cancer.

Surgical simulation – a new tool to evaluate surgical incisions in congenital heart disease?

We introduce a new concept for preoperative planning and surgical education in congenital heart disease: surgical simulation. Recent advances in three-dimensional image acquisition have provided a new means to virtually reconstruct accurate morphological models while computer visualisation hardware now allows simulation of elastic tissue deformations interactively. Incision simulation is performed in two patients with complex congenital heart disease to preoperatively evaluate potential corrective surgical strategies. The relevant cardiac morphology was correctly depicted by the virtual models on which arbitrary incisions could be performed. By visualising the morphology in respect to each incision, different surgical strategies could be evaluated pre-operatively. We have taken the first step towards a clinically useful incision simulator for procedures in congenital heart disease and made an initial evaluation. With further developments it is likely that new tools for patient-specific preoperative planning and surgical training will emerge based on the presented ideas.