Stereotactic Pelvic Navigation With Augmented Reality for Transanal Total Mesorectal Excision

Stereotactic Navigation-Assisted Laparoscopic Resection of Challenging Low Pelvic Tumors: A Case Series

(1) Introduction: The laparoscopic approach to low pelvic tumors is challenging and hindered by suboptimal tumor visualization and dissection, with possible oncological failure. Stereotactic navigation provides real-time image guidance that may optimize safety, accuracy, and precision when dissecting challenging low pelvic tumors. (2) Methods: Preoperative CT images were acquired with eight skin-fixed fiducials and loaded into a navigation system. A patient tracker was mounted on the bed side. Patient-to-image paired point registration was performed, and an instrument tracker was mounted on a laparoscopic instrument and calibrated for instrument tracking. Surgical operations were performed with real-time stereotactic navigation assistance. (3) Results: Three patients underwent stereotactic navigation surgery. Fiducial registration errors were good to optimal (±1.9, ±3.4, and ±3.4 mm). Lesions were easily identified and targeted with real-time navigation. Surgeries were uneventful. Histopathology examinations identified one retro-rectal schwannoma, one lateral pelvic recurrence from rectal adenocarcinoma, and one advanced anal canal carcinoma. No navigation-related complications, readmissions, or postoperative mortalities were observed. (4) Conclusions: The application of laparoscopic stereotactic navigation surgery to complex low pelvic tumors is feasible and could impact oncological surgical quality by enabling tumor targeting and ensuring resection margins. Further wider series are needed to confirm stereotactic navigation's impact on challenging low pelvic tumors.

Navigation-Assisted Surgery for Locally Advanced Primary and Recurrent Rectal Cancer

BACKGROUND

In some surgical disciplines, navigation-assisted surgery has become standard of care, but in rectal cancer, indications for navigation and the utility of different technologies remain undetermined.

METHODS

The NAVI-LARRC prospective study (NCT04512937; IDEAL Stage 2a) evaluated feasibility of navigation in patients with locally advanced primary (LARC) and recurrent rectal cancer (LRRC). Included patients had advanced tumours with high risk of incomplete (R1/R2) resection, and navigation was considered likely to improve the probability of complete resection (R0). Tumours were classified according to pelvic compartmental involvement, as suggested by the Royal Marsden group. The BrainlabTM navigation platform was used for preoperative segmentation of tumour and pelvic anatomy, and for intraoperative navigation with optical tracking. R0 resection rates, surgeons' experiences, and adherence to the preoperative resection plan were assessed.

RESULTS

Seventeen patients with tumours involving the posterior/lateral compartments underwent navigation-assisted procedures. Fifteen patients required abdominosacral resection, and 3 had resection of the sciatic nerve. R0 resection was obtained in 6/8 (75%) LARC and 6/9 (69%) LRRC cases. Preoperative segmentation was time-consuming (median 3.5 h), but intraoperative navigation was accurate. Surgeons reported navigation to be feasible, and adherence to the resection plan was satisfactory.

CONCLUSIONS

Navigation-assisted surgery using optical tracking was feasible. The preoperative planning was time-consuming, but intraoperative navigation was accurate and resulted in acceptable R0 resection rates. Selected patients are likely to benefit from navigation-assisted surgery.

User's image perception improved strategy and application of augmented reality systems in smart medical care: A review

BACKGROUND

Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures.

METHODS

This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field.

RESULTS

A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment.

CONCLUSION

This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.

Practical Application of Augmented/Mixed Reality Technologies in Surgery of Abdominal Cancer Patients

The technology of augmented and mixed reality (AR/MR) is useful in various areas of modern surgery. We considered the use of augmented and mixed reality technologies as a method of preoperative planning and intraoperative navigation in abdominal cancer patients. Practical use of AM/MR raises a range questions, which demand suitable solutions. The difficulties and obstacles we encountered in the practical use of AR/MR are presented, along with the ways we chose to overcome them. The most demonstrative case is covered in detail. The three-dimensional anatomical model obtained from the CT scan needed to be rigidly attached to the patient’s body, and therefore an invasive approach was developed, using an orthopedic pin fixed to the pelvic bones. The pin is used both similarly to an X-ray contrast marker and as a marker for augmented reality. This solution made it possible, not only to visualize the anatomical structures of the patient and the border zone of the tumor, but also to change the position of the patient during the operation. In addition, a noninvasive (skin-based) marking method was developed that allows the application of mixed and augmented reality during operation. Both techniques were used (8 clinical cases) for preoperative planning and intraoperative navigation, which allowed surgeons to verify the radicality of the operation, to have visual control of all anatomical structures near the zone of interest, and to reduce the time of surgical intervention, thereby reducing the complication rate and improving the rehabilitation period.

Real-time vascular anatomical image navigation for laparoscopic surgery: experimental study

BACKGROUND

Recognition of the inferior mesenteric artery (IMA) during colorectal cancer surgery is crucial to avoid intraoperative hemorrhage and define the appropriate lymph node dissection line. This retrospective feasibility study aimed to develop an IMA anatomical recognition model for laparoscopic colorectal resection using deep learning, and to evaluate its recognition accuracy and real-time performance.

METHODS

A complete multi-institutional surgical video database, LapSig300 was used for this study. Intraoperative videos of 60 patients who underwent laparoscopic sigmoid colon resection or high anterior resection were randomly extracted from the database and included. Deep learning-based semantic segmentation accuracy and real-time performance of the developed IMA recognition model were evaluated using Dice similarity coefficient (DSC) and frames per second (FPS), respectively.

RESULTS

In a fivefold cross-validation conducted using 1200 annotated images for the IMA semantic segmentation task, the mean DSC value was 0.798 (± 0.0161 SD) and the maximum DSC was 0.816. The proposed deep learning model operated at a speed of over 12 FPS.

CONCLUSION

To the best of our knowledge, this is the first study to evaluate the feasibility of real-time vascular anatomical navigation during laparoscopic colorectal surgery using a deep learning-based semantic segmentation approach. This experimental study was conducted to confirm the feasibility of our model; therefore, its safety and usefulness were not verified in clinical practice. However, the proposed deep learning model demonstrated a relatively high accuracy in recognizing IMA in intraoperative images. The proposed approach has potential application in image navigation systems for unfixed soft tissues and organs during various laparoscopic surgeries.

Stereotactic navigation using registration based on intra-abdominal landmarks in robotic-assisted lateral pelvic lymph node dissection

BACKGROUND

We carried out robot-assisted lateral pelvic lymph node dissection (LPLND) for rectal cancer with a stereotactic navigation system. The purpose of this study was to evaluate the accuracy and feasibility of the system.

METHODS

We constructed a navigation system based on the Polaris Spectra optical tracking device (Northern Digital Inc., Canada) and the open-source software 3D Slicer (version 3.8.1; http://www.slicer.org ). We used the landmark-based registration method for patient-to-image registration. Body surface landmarks and intra-abdominal landmarks were used. We evaluated the time required for registration and target registration error (TRE; the distance between corresponding points after registration) for the root of the superior gluteal artery the root of the obturator or superior vesical artery, and the obturator foramen during minimally invasive LPLND for rectal cancer. Five patients who had LPLND for rectal cancer at the University of Tokyo Hospital between September 2020 and May 2021 were enrolled.

RESULTS

The mean time required for registration was 49 s with the body surface landmarks and 88 s with the intra-abdominal landmarks. The mean TRE improved markedly when the registration was performed using intra-abdominal landmarks. The mean TRE of the root of the superior gluteal artery, the root of the obturator or superior vesical artery, and the obturator foramen were 55.8 mm, 53.4 mm, and 55.2 mm with the body surface landmarks and 11.8 mm, 10.0 mm, and 12.6 mm with the intra-abdominal landmarks, respectively. There were no adverse events related to the registration process.

CONCLUSIONS

When stereotactic navigation systems are used for minimally invasive LPLND, the use of intra-abdominal landmarks for registration is feasible and may allow simpler and more accurate navigation than the use of body surface landmarks.

[Perioperative augmented reality technology in surgical treatment of locally advanced recurrent rectal cancer]

Rectal cancer occupies the leading position among cancers, and incidence of locally advanced recurrences is still high despite comprehensive treatment. Combined resections are usually associated with high perioperative risks. These procedures are technically complex interventions requiring further improvement. Virtual reality technology in surgical treatment of locally advanced rectal cancer recurrence has not been widely discussed. The authors present multidisciplinary construction of the matched topographic-anatomical virtual model and virtual planning of the combined surgical intervention. Intraoperative use of augmented reality allowed specifying topographic and anatomical features of surgical area, level of vascular ligation, localization of tumor fixation points and resection borders. These data ensured safety and quality of resection. Further research of augmented reality technology and improvement of its technical aspects will improve the results of surgical treatment of patients with locally advanced pelvic tumors and recurrences.

Supporting laparoscopic general surgery training with digital technology: The United Kingdom and Ireland paradigm

Surgical training in the UK and Ireland has faced challenges following the implementation of the European Working Time Directive and postgraduate training reform. The health services are undergoing a digital transformation; digital technology is remodelling the delivery of surgical care and surgical training. This review aims to critically evaluate key issues in laparoscopic general surgical training and the digital technology such as virtual and augmented reality, telementoring and automated workflow analysis and surgical skills assessment. We include pre-clinical, proof of concept research and commercial systems that are being developed to provide solutions. Digital surgical technology is evolving through interdisciplinary collaboration to provide widespread access to high-quality laparoscopic general surgery training and assessment. In the future this could lead to integrated, context-aware systems that support surgical teams in providing safer surgical care.

Association of Image-Guided Navigation With Complete Resection Rate in Patients With Locally Advanced Primary and Recurrent Rectal Cancer: A Nonrandomized Controlled Trial

IMPORTANCE

The percentage of tumor-positive surgical resection margin rates in patients treated for locally advanced primary or recurrent rectal cancer is high. Image-guided navigation may improve complete resection rates.

OBJECTIVE

To ascertain whether image-guided navigation during rectal cancer resection improves complete resection rates compared with surgical procedures without navigation.

DESIGN, SETTING, AND PARTICIPANTS

This prospective single-center nonrandomized controlled trial was conducted at the Netherlands Cancer Institute-Antoni van Leeuwenhoek in Amsterdam, the Netherlands. The prospective or navigation cohort included adult patients with locally advanced primary or recurrent rectal cancer who underwent resection with image-guided navigation between February 1, 2016, and September 30, 2019, at the tertiary referral hospital. Clinical results of this cohort were compared with results of the historical cohort, which was composed of adult patients who received rectal cancer resection without image-guided navigation between January 1, 2009, and December 31, 2015.

INTERVENTION

Rectal cancer resection with image-guided navigation.

MAIN OUTCOMES AND MEASURES

The primary end point was the complete resection rate, measured by the amount of tumor-negative resection margin rates. Secondary outcomes were safety and usability of the system. Safety was evaluated by the number of navigation system-associated surgical adverse events. Usability was assessed from responses to a questionnaire completed by the participating surgeons after each procedure.

RESULTS

In total, 33 patients with locally advanced or recurrent rectal cancer were included (23 men [69.7%]; median [interquartile range] age at start of treatment, 61 [55.0-69.0] years). With image-guided navigation, a radical resection (R0) was achieved in 13 of 14 patients (92.9%; 95% CI, 66.1%-99.8%) after primary resection of locally advanced tumors and in 15 of 19 patients (78.9%; 95% CI, 54.4%-94.0%) after resection of recurrent rectal cancer. No navigation system-associated complications occurred before or during surgical procedures. In the historical cohort, 142 patients who underwent resection without image-guided navigation were included (95 men [66.9%]; median [interquartile range] age at start of treatment, 64 [55.0-70.0] years). In these patients, an R0 resection was accomplished in 85 of 101 patients (84.2%) with locally advanced rectal cancer and in 20 of 41 patients (48.8%) with recurrent rectal cancer. A significant difference was found between the navigation and historical cohorts after recurrent rectal cancer resection (21.1% vs 51.2%; P = .047). For locally advanced primary tumor resection, the difference was not significant (7.1% vs 15.8%; P = .69). Surgeons stated in completed questionnaires that the navigation system improved decisiveness and helped with tumor localization.

CONCLUSIONS AND RELEVANCE

Findings of this study suggest that image-guided navigation used during rectal cancer resection is safe and intuitive and may improve tumor-free resection margin rates in recurrent rectal cancer.

TRIAL REGISTRATION

Netherlands Trial Register Identifier: NTR7184.

Vision-based tracking system for augmented reality to localize recurrent laryngeal nerve during robotic thyroid surgery

We adopted a vision-based tracking system for augmented reality (AR), and evaluated whether it helped surgeons to localize the recurrent laryngeal nerve (RLN) during robotic thyroid surgery. We constructed an AR image of the trachea, common carotid artery, and RLN using CT images. During surgery, an AR image of the trachea and common carotid artery were overlaid on the physical structures after they were exposed. The vision-based tracking system was activated so that the AR image of the RLN followed the camera movement. After identifying the RLN, the distance between the AR image of the RLN and the actual RLN was measured. Eleven RLNs (9 right, 4 left) were tested. The mean distance between the RLN AR image and the actual RLN was 1.9 ± 1.5 mm (range 0.5 to 3.7). RLN localization using AR and vision-based tracking system was successfully applied during robotic thyroidectomy. There were no cases of RLN palsy. This technique may allow surgeons to identify hidden anatomical structures during robotic surgery.

Computer-assisted real-time automatic prostate segmentation during TaTME: a single-center feasibility study

BACKGROUND

Urethral injuries (UIs) are significant complications pertaining to transanal total mesorectal excision (TaTME). It is important for surgeons to identify the prostate during TaTME to prevent UI occurrence; intraoperative image navigation could be considered useful in this regard. This study aims at developing a deep learning model for real-time automatic prostate segmentation based on intraoperative video during TaTME. The proposed model's performance has been evaluated.

METHODS

This was a single-institution retrospective feasibility study. Semantic segmentation of the prostate area was performed using a convolutional neural network (CNN)-based approach. DeepLab v3 plus was utilized as the CNN model for the semantic segmentation task. The Dice coefficient (DC), which is calculated based on the overlapping area between the ground truth and predicted area, was utilized as an evaluation metric for the proposed model.

RESULTS

Five hundred prostate images were randomly extracted from 17 TaTME videos, and the prostate area was manually annotated on each image. Fivefold cross-validation tests were performed, and as observed, the average DC value equaled 0.71 ± 0.04, the maximum value being 0.77. Additionally, the model operated at 11 fps, which provides acceptable real-time performance.

CONCLUSIONS

To the best of the authors' knowledge, this is the first effort toward realization of computer-assisted TaTME, and results obtained in this study suggest that the proposed deep learning model can be utilized for real-time automatic prostate segmentation. In future endeavors, the accuracy and performance of the proposed model will be improved to enable its use in practical applications, and its capability to reduce UI risks during TaTME will be verified.

Stereotactic Navigation for Rectal Surgery: Comparison of 3-Dimensional C-Arm-Based Registration to Paired-Point Registration

BACKGROUND

Stereotactic navigation enables surgeons to use the preoperative CT or MRI images as a real-time "navigation map." Although stereotactic navigation has been established in neurosurgery and orthopedic surgery, whether this technology is applicable to GI tract surgery remains challenging because of tissue deformation and organ motion. A critical component of this technology is the registration that links the patient's actual body to the preoperative imaging data.

OBJECTIVE

The objective was to assess the applicability of stereotactic navigation in rectal surgery, focusing on the registration method.

DESIGN

This study was based on a prospective case series.

SETTING

The study was conducted in a single university hospital.

PATIENTS

Four patients who underwent laparoscopic rectal surgery were included.

INTERVENTIONS

Paired-point registration was performed for 2 cases, whereas 3-dimensional C-arm-based registration was performed for the other 2 cases. In addition, 3-dimensional C-arm-based registration was performed twice during the operation.

MAIN OUTCOME MEASURE

Navigation accuracy was evaluated by measuring target registration error at 8 anatomical landmarks.

RESULTS

Target registration error of the 3-dimensional C-arm-based registration group was significantly smaller than that of the paired-point registration group (median, 19.5 mm vs 54.1 mm; p < 0.001). In particular, the error of Z-axis (cranial-to-caudal direction) was significantly smaller in 3-dimensional C-arm-based registration (median, 12.4 mm vs 48.8 mm; p < 0.001). In one case in the 3-dimensional C-arm-based registration group, target registration error of the second registration became significantly smaller than that of the first registration (p = 0.008).

LIMITATIONS

This was an observational study with small sample size.

CONCLUSION

Three-dimensional C-arm-based registration could be performed with the patient in a lithotomy position with head down and lateral tilt without being affected by positional changes. Three-dimensional C-arm-based registration resulted in significantly higher navigation accuracy than paired-point registration, and its accuracy could be further improved by intraoperative re-registration.

Accurate surgical navigation with real-time tumor tracking in cancer surgery

In the past decades, image-guided surgery has evolved rapidly. In procedures with a relatively fixed target area, like neurosurgery and orthopedics, this has led to improved patient outcomes. In cancer surgery, intraoperative guidance could be of great benefit to secure radical resection margins since residual disease is associated with local recurrence and poor survival. However, most tumor lesions are mobile with a constantly changing position. Here, we present an innovative technique for real-time tumor tracking in cancer surgery. In this study, we evaluated the feasibility of real-time tumor tracking during rectal cancer surgery. The application of real-time tumor tracking using an intraoperative navigation system is feasible and safe with a high median target registration accuracy of 3 mm. This technique allows oncological surgeons to obtain real-time accurate information on tumor location, as well as critical anatomical information. This study demonstrates that real-time tumor tracking is feasible and could potentially decrease positive resection margins and improve patient outcome.