New three-dimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: procedures for gasless single-port radical nephrectomy

Augmented reality-based autostereoscopic surgical visualization system for telesurgery

PURPOSE

The visualization of remote surgical scenes is the key to realizing the remote operation of surgical robots. However, current non-endoscopic surgical robot systems lack an effective visualization tool to offer sufficient surgical scene information and depth perception.

METHODS

We propose a novel autostereoscopic surgical visualization system integrating 3D intraoperative scene reconstruction, autostereoscopic 3D display, and augmented reality-based image fusion. The preoperative organ structure and the intraoperative surface point cloud are obtained from medical imaging and the RGB-D camera, respectively, and aligned by an automatic marker-free intraoperative registration algorithm. After registration, preoperative meshes with precalculated illumination and intraoperative textured point cloud are blended in real time. Finally, the fused image is shown on a 3D autostereoscopic display device to achieve depth perception.

RESULTS

A prototype of the autostereoscopic surgical visualization system was built. The system had a horizontal image resolution of 1.31 mm, a vertical image resolution of 0.82 mm, an average rendering rate of 33.1 FPS, an average registration rate of 20.5 FPS, and average registration errors of approximately 3 mm. A telesurgical robot prototype based on 3D autostereoscopic display was built. The quantitative evaluation experiments showed that our system achieved similar operational accuracy (1.79 ± 0.87 mm) as the conventional system (1.95 ± 0.71 mm), while having advantages in terms of completion time (with 34.11% reduction) and path length (with 35.87% reduction). Post-experimental questionnaires indicated that the system was user-friendly for novices and experts.

CONCLUSION

We propose a 3D surgical visualization system with augmented instruction and depth perception for telesurgery. The qualitative and quantitative evaluation results illustrate the accuracy and efficiency of the proposed system. Therefore, it shows great prospects in robotic surgery and telesurgery.

Novel anatomical apical dissection utilizing puboprostatic "open-collar" technique: Impact on apical surgical margin and early continence recovery

Purpose

To evaluate the impact of modifications to anatomical apical dissection including a puboprostatic open-collar technique, which visualizes the lateral aspect of the apex and dorsal vein complex (DVC) covering the rhabdosphincter while preserving the puboprostatic collar, on positive surgical margin (PSM) and continence recovery.

Methods

One-hundred-and-sixty-seven patients underwent gasless single-port retroperitoneoscopic radical prostatectomy using a three-dimensional head-mounted display system. Sequentially modified surgical techniques comprised puboprostatic open-collar technique, sutureless transection of the DVC, retrograde urethral dissection, and anterior reconstruction. The associations of these modifications with PSM and continence recovery were assessed.

Results

The puboprostatic open-collar technique, sutureless DVC transection, and retrograde urethral dissection were significantly associated with lower apical PSM (P = 0.003, 0.003, and 0.010, respectively). The former two also showed similar associations in 84 patients with anterior apical tumor (P = 0.021 and 0.030, respectively). Among 92 patients undergoing all of these three procedures, overall and apical PSM rates were 13.0% and 3.3%, respectively. Retrograde urethral dissection (odds ratio [OR] 2.73, P = 0.004) together with nerve sparing (OR 2.77, P = 0.003) and anterior apical tumor (OR 0.45, P = 0.017) were independently associated with immediate continence recovery. A multivariable model for 3-month continence recovery included anterior apical tumor (OR 0.28, P = 0.003) and puboprostatic open-collar technique (OR 3.42, P = 0.062). Immediate and 3-month continence recovery rates were 56.3% and 85.4%, respectively, in 103 patients undergoing both the puboprostatic open-collar technique and retrograde urethral dissection.

Conclusion

Novel anatomical apical dissection utilizing a puboprostatic open-collar technique may favorably impact on both apical surgical margin and continence recovery.

Visualization of Fluoroscopic Imaging in Orthopedic Surgery: Head-Mounted Display vs Conventional Monitor

Purpose. See-through head-mounted displays (HMDs) can be used to view fluoroscopic imaging during orthopedic surgical procedures. The goals of this study were to determine whether HMDs reduce procedure time, number of fluoroscopic images required, or number of head turns by the surgeon compared with standard monitors. Methods. Sixteen orthopedic surgery residents each performed fluoroscopy-guided drilling of 8 holes for placement of tibial nail distal interlocking screws in an anatomical model, with 4 holes drilled while using HMD and 4 holes drilled while using a standard monitor. Procedure time, number of fluoroscopic images needed, and number of head turns by the resident during the procedure were compared between the 2 modalities. Statistical significance was set at P < .05. Results. Mean (SD) procedure time did not differ significantly between attempts using the standard monitor (55 [37] seconds) vs the HMD (56 [31] seconds) (P = .73). Neither did mean number of fluoroscopic images differ significantly between attempts using the standard monitor vs the HMD (9 [5] images for each) (P = .84). Residents turned their heads significantly more times when using the standard monitor (9 [5] times) vs the HMD (1 [2] times) (P < .001). Conclusions. Head-mounted displays lessened the need for residents to turn their heads away from the surgical field while drilling holes for tibial nail distal interlocking screws in an anatomical model; however, there was no difference in terms of procedure time or number of fluoroscopic images needed using the HMD compared with the standard monitor.

Laparoscopy with augmented reality adaptations

One of the most promising applications of Optical See-Through Augmented Reality is minimally laparoscopic surgery, which currently suffers from problems such as surgeon discomfort and fatigue caused by looking at a display positioned outside the surgeon's visual field, made worse by the length of the procedure. This fatigue is especially felt on the surgeon's neck, as it is strained from adopting unnatural postures in order to visualise the laparoscopic video feed. Throughout this paper, we will present work in Augmented Reality, as well as developments in surgery and Augmented Reality applied to both surgery in general and laparoscopy in particular to address these issues. We applied user and task analysis methods to learn about practices performed in the operating room by observing surgeons in their working environment in order to understand, in detail, how they performed their tasks and achieved their intended goals. Drawing on observations and analysis of video recordings of laparoscopic surgeries, we identified relevant constraints and design requirements. Besides proposals to approach the ergonomic issues, we present a design and implementation of a multimodal interface to enhance the laparoscopic procedure. Our method makes it more comfortable for surgeons by allowing them to keep the laparoscopic video in their viewing area regardless of neck posture. Also, our interface makes it possible to access patient imaging data without interrupting the operation. It also makes it possible to communicate with team members through a pointing reticle. We evaluated how surgeons perceived the implemented prototype, in terms of usefulness and usability, via a think-aloud protocol to conduct qualitative evaluation sessions which we describe in detail in this paper. In addition to checking the advantages of the prototype as compared to traditional laparoscopic settings, we also conducted a System Usability Scale questionnaire for measuring its usability, and a NASA Task Load Index questionnaire to rate perceived workload and to assess the prototype effectiveness. Our results show that surgeons consider that our prototype can improve surgeon-to-surgeon communication using head pose as a means of pointing. Also, surgeons believe that our approach can afford a more comfortable posture throughout the surgery and enhance hand-eye coordination, as physicians no longer need to twist their necks to look at screens placed outside the field of operation.

Augmented reality guided osteotomy in hallux Valgus correction

Background

An optimal osteotomy angle avoids shortening of the first metatarsal bone after hallux valgus surgery and therefore reduces the risk of transfer-metatarsalgia. The purpose of the present ex-vivo study was to investigate whether augmented reality (AR) would improve accuracy of the distal osteotomy during hallux valgus surgery.

Methods

Distal osteotomies of the first metatarsals were performed on a foot model by two surgeons with different levels of surgical experience each with (AR, n = 15 × 2) or without (controls, n = 15 × 2) overlay of a hologram depicting an angle of osteotomy perpendicular to the second metatarsal. Subsequently, the deviation of the osteotomy angle in the transverse plane was analyzed.

Results

Overall, AR decreased the extent of deviation and the AR guided osteotomies were more accurate (4.9 ± 4.2°) compared to the freehand cuts (6.7 ± 6.1°) by tendency (p = 0.2). However, while the inexperienced surgeon performed more accurate osteotomies with AR with a mean angle of 6.4 ± 3.5° compared to freehand 10.5 ± 5.5° (p = 0.02), no significant difference was noticed for the experienced surgeon with an osteotomy angle of around 3° in both cases.

Conclusion

This pilot-study suggests that AR guided osteotomies can potentially improve accuracy during hallux valgus correction, particularly for less experienced surgeons.

Clinically node-positive micropapillary bladder cancer in a young female desiring to spare functional bladder and fertility

A 38-year-old female developing pelvic lymph node recurrence (cN2) of bladder cancer was referred to our hospital. Eighteen months earlier, she had received complete transurethral resection of bladder tumor and pathological diagnosis of muscle-invasive urothelial carcinoma with micropapillary variant had been made. She had declined radical cystectomy or chemoradiation because of a strong desire to spare the bladder and fertility. She received induction gemcitabine/cisplatin therapy. After three cycles, she achieved a clinically complete response. Then, she underwent transumbilical gasless retroperitoneoscopic super-extended pelvic lymph node dissection up to the inferior mesenteric artery trunk using the three-dimensional head-mounted display system. Pathologically, one of 42 resected lymph nodes was positive for metastasis. She is alive with no evidence of disease 5 years after lymph node dissection, retaining functional bladder and normal ovarian functions.

Does a 3D laparoscopic approach improve surgical outcome of mininvasive right colectomy? A retrospective case-control study

Laparoscopy has gained wide acceptance due its benefits for patients. However, advanced laparoscopic procedures are still challenging. One critical issue is lack of stereoscopic vision. Despite its diffusion, the totally laparoscopic approach for right hemicolectomy (TLRC) is still debated due to its difficulty, particularly for fashioning of the ileocolic anastomosis. The aim of this multicenter study is to investigate whether 3D vision offers any advantages on surgical performance over 2D vision during TLRC. All data of consecutive patients who underwent elective TLRC for cancer at three Italian surgical centers with either 2D or 3D technology from January 2013 to December 2018 were retrieved from a computer-maintained database. A case-matched analysis using the Mantel-Haenszel method was performed. After matching, a total of 106 patients were analyzed with 53 patients in each group. Mean operative time was significantly longer for 2D-TLRC than for 3D-TLRC (153.2 ± 52.4 vs. 131 ± 51 min, p = 0.029) and a statistically significant difference in anastomosing time (p = 0.032, 19.2 ± 5.9 min vs. 21.7 ± 6.2 min for 3D and 2D group, respectively) was also recorded. No difference in the median number of harvested nodes (23 ± 11 vs. 21 ± 7 for 3D and 2D group, respectively; p = 0.48) was found. Neither intraoperative complications nor conversions occurred in the two groups. In conclusion, 3D vision appears to improve the performance of a TLRC by reducing operative time and making intracorporeal anastomosis easier. Prospective randomized studies are required to determine the real beneficial effects.

Comparison of a 3D head-mounted display (HMS-3000MT) and 3D passive polarizing display with 2D technique for first laparoscopic inguinal hernia repair by novice surgeons

BACKGROUND

Three-dimensional (3D) laparoscopy improves the surgical skills of novice surgeons and positively affects the learning curve in experimental settings. This study aimed to investigate the effect of a 3D passive polarizing display (3DPPD) and a novel 3D head-mounted display (3DHMD; HMS-3000MT) on the performance of the first laparoscopic inguinal hernia repair by novices and compare both systems with standard high-definition 2D (HD2D) laparoscopy.

METHODS

Patients with symptomatic inguinal hernia underwent transabdominal preperitoneal (TAPP) approach hernia repair using 3DHMD, 3DPPD, or a conventional HD2D laparoscopic system. All surgeries were performed for the first time by three laparoscopically novice surgeons. Operative performance was compared in terms of the time taken for mesh placement and peritoneal suturing under standardized conditions. Additionally, visual perception parameters and adverse effects were assessed.

RESULTS

The use of both 3D techniques shortened the time required for mesh placement and peritoneal suturing compared with the conventional HD2D approach. Generally, 3D laparoscopy was superior to HD2D laparoscopy in terms of visual perception parameters such as depth perception, sharpness, ghosting, and contrast. However, compared with the use of HD2D laparoscopy, the use of 3DHMD significantly impaired a surgeon's comfort, with the greatest impairment caused by ear discomfort, headaches, and facial and physical discomforts.

CONCLUSIONS

The 3DHMD and 3DPPD systems showed clear improvement in first hernia repair laparoscopy by novice surgeons in terms of surgical performance, as well as visual perception; however, the 3DHMD system was not superior to the 3DPPD system. The reduction in training time for new surgeons is obviously advantageous. In this respect, the 3D equipment may be a worthwhile investment.

Head-Mounted Display Use in Surgery: A Systematic Review

Purpose. We analyzed the literature to determine (1) the surgically relevant applications for which head-mounted display (HMD) use is reported; (2) the types of HMD most commonly reported; and (3) the surgical specialties in which HMD use is reported. Methods. The PubMed, Embase, Cochrane Library, and Web of Science databases were searched through August 27, 2017, for publications describing HMD use during surgically relevant applications. We identified 120 relevant English-language, non-opinion publications for inclusion. HMD types were categorized as “heads-up” (nontransparent HMD display and direct visualization of the real environment), “see-through” (visualization of the HMD display overlaid on the real environment), or “non–see-through” (visualization of only the nontransparent HMD display). Results. HMDs were used for image guidance and augmented reality (70 publications), data display (63 publications), communication (34 publications), and education/training (18 publications). See-through HMDs were described in 55 publications, heads-up HMDs in 41 publications, and non–see-through HMDs in 27 publications. Google Glass, a see-through HMD, was the most frequently used model, reported in 32 publications. The specialties with the highest frequency of published HMD use were urology (20 publications), neurosurgery (17 publications), and unspecified surgical specialty (20 publications). Conclusion. Image guidance and augmented reality were the most commonly reported applications for which HMDs were used. See-through HMDs were the most commonly reported type used in surgically relevant applications. Urology and neurosurgery were the specialties with greatest published HMD use.

Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: a simulation-based feasibility study

Background

The advent of Virtual Reality technologies presents new opportunities for enhancing current surgical practice. Studies suggest that current techniques in endoscopic surgery are prone to disturbance of a surgeon’s visual-motor axis, influencing performance, ergonomics and iatrogenic injury rates. The Microsoft^® HoloLens is a novel head-mounted display that has not been explored within surgical innovation research. This study aims to evaluate the HoloLens as a potential alternative to conventional monitors in endoscopic surgery.

Materials and methods

This prospective, observational and comparative study recruited 72 participants consisting of novices (n = 28), intermediate-level (n = 24) and experts (n = 20). Participants performed ureteroscopy, within an inflatable operating environment, using a validated training model and the HoloLens mixed-reality device as a monitor. Novices also completed the assigned task using conventional monitors; whilst the experienced groups did not, due to their extensive familiarity. Outcome measures were procedural completion time and performance evaluation (OSATS) score. A final evaluation survey was distributed amongst all participants.

Results

The HoloLens facilitated improved outcomes for procedural times (absolute difference, − 73 s; 95% CI − 115 to − 30; P = 0.0011) and OSAT scores (absolute difference, 4.1 points; 95% CI 2.9–5.3; P < 0.0001) compared to conventional monitors. Feedback evaluation demonstrated 97% of participants agreed or strongly agreed that the HoloLens will have a role in surgical education (mean rating, 4.6 of 5; 95% CI 4.5–4.8). Furthermore, 95% of participants agreed or strongly agreed that the HoloLens is feasible to introduce clinically and will have a role within surgery (mean rating, 4.4 of 5; 95% CI 4.2–4.5).

Conclusion

This study demonstrates that the device facilitated improved outcomes of performance in novices and was widely accepted as a surgical visual aid by all groups. The HoloLens represents a feasible alternative to the conventional setup, possibly by aligning the surgeon’s visual-motor axis.

See-Through Type 3D Head-Mounted Display-Based Surgical Microscope System for Microsurgery: A Feasibility Study

Background

The surgical microscope is used primarily for microsurgeries, which are more complicated than other surgical procedures and require delicate tasks for a long time. Therefore, during these surgical procedures, surgeons experience back and neck pain. To solve this problem, new technology, such as wearable displays, is required to help surgeons maintain comfortable postures and enjoy advanced functionality during microsurgery.

Objective

The objective of this study was to develop a surgical microscope system that would work with wearable devices. It would include a head-mounted display (HMD) that can offer 3D surgical images and allow a flexible and comfortable posture instead of fixed eyepieces of surgical microscope and can also provide peripheral visual field with its optical see-through function.

Methods

We designed and fabricated a surgical microscope system that incorporates a see-through type 3D HMD, and we developed an image processing software to provide better image quality. The usability of the proposed system was confirmed with preclinical examination. Seven ENT (ear, nose, and throat) surgical specialists and 8 residents performed a mock surgery—axillary lymph node dissection on a rat. They alternated between looking through the eyepieces of the surgical microscope and viewing a 3D HMD screen connected to the surgical microscope. We examined the success of the surgery and asked the specialists and residents to grade eye fatigue on a scale of 0 (none) to 6 (severe) and posture discomfort on a scale of 1 (none) to 5 (severe). Furthermore, a statistical comparison was performed using 2-tailed paired t test, and P=.00083 was considered significant.

Results

Although 3D HMD case showed a slightly better result regarding visual discomfort (P=.097), the average eye fatigue was not significantly different between eyepiece and 3D HMD cases (P=.79). However, the average posture discomfort, especially in neck and shoulder, was lower with 3D HMD display use than with eyepiece use (P=.00083).

Conclusions

We developed a see-through type 3D HMD–based surgical microscope system and showed through preclinical testing that the system could help reduce posture discomfort. The proposed system, with its advanced functions, could be a promising new technique for microsurgery.

The use of 3D laparoscopic imaging systems in surgery: EAES consensus development conference 2018

BACKGROUND

The use of 3D laparoscopic systems is expanding. The European Association of Endoscopic Surgery (EAES) initiated a consensus development conference with the aim of creating evidence-based statements and recommendations for the surgical community.

METHODS

Systematic reviews of the PubMed and Embase libraries were performed to identify evidence on potential benefits of 3D on clinical practice and patient outcomes. Statements and recommendations were prepared and unanimously agreed by an international surgical and engineering expert panel which were presented and voted at the EAES annual congress, London, May 2018.

RESULTS

9967 abstracts were screened with 138 articles included. 18 statements and two recommendations were generated and approved. 3D significantly shortened operative time (mean difference 11 min (8% [95% CI 20.29-1.72], I² 96%)). A significant reduction in complications was observed when 3D systems were used (RR 0.75, [95 CI% 0.60-0.94], I² 0%) particularly for cases involving laparoscopic suturing (RR 0.57 [95% CI 0.35-0.90], I² 0%). In 69 box trainer or simulator studies, 64% concluded trainees were significant faster and 62% performed fewer errors when using 3D.

CONCLUSION

We recommend the use of 3D vision in laparoscopy to reduce the operative time (grade of recommendation: low). Future robust clinical research is required to specifically investigate the potential benefit of 3D laparoscopy system on complication rates (grade of recommendation: high).

The use of 3D laparoscopic imaging systems in surgery: EAES consensus development conference 2018

BACKGROUND

The use of 3D laparoscopic systems is expanding. The European Association of Endoscopic Surgery (EAES) initiated a consensus development conference with the aim of creating evidence-based statements and recommendations for the surgical community.

METHODS

Systematic reviews of the PubMed and Embase libraries were performed to identify evidence on potential benefits of 3D on clinical practice and patient outcomes. Statements and recommendations were prepared and unanimously agreed by an international surgical and engineering expert panel which were presented and voted at the EAES annual congress, London, May 2018.

RESULTS

9967 abstracts were screened with 138 articles included. 18 statements and two recommendations were generated and approved. 3D significantly shortened operative time (mean difference 11 min (8% [95% CI 20.29-1.72], I² 96%)). A significant reduction in complications was observed when 3D systems were used (RR 0.75, [95 CI% 0.60-0.94], I² 0%) particularly for cases involving laparoscopic suturing (RR 0.57 [95% CI 0.35-0.90], I² 0%). In 69 box trainer or simulator studies, 64% concluded trainees were significant faster and 62% performed fewer errors when using 3D.

CONCLUSION

We recommend the use of 3D vision in laparoscopy to reduce the operative time (grade of recommendation: low). Future robust clinical research is required to specifically investigate the potential benefit of 3D laparoscopy system on complication rates (grade of recommendation: high).

Randomized study of the influence of two-dimensional versus three-dimensional imaging using a novel 3D head-mounted display (HMS-3000MT) on performance of laparoscopic inguinal hernia repair

BACKGROUND

3D laparoscopy has proven to be superior to the 2D approach in experimental settings. The aim of the present study was to investigate the influence of 3D laparoscopy using a novel head-mounted display on the performance of defined steps within a laparoscopic inguinal hernia repair. This effect was investigated both in laparoscopically advanced surgeons and in beginners.

METHODS

Patients suffering from symptomatic inguinal hernia were randomly assigned to laparoscopic hernia repair using either a head-mounted 3D display or a conventional 2D laparoscopic approach. Operative performance of both groups was compared in terms of the time taken for mesh placement and for peritoneal suturing. Additionally, quality of imaging and physical discomfort were assessed.

RESULTS

The use of a head-mounted 3D display was able to shorten the time required for placement of the mesh as well as that for peritoneal suturing, both for experienced and novice surgeons when compared to the conventional 2D approach. 3D laparoscopy was significantly superior to 2D laparoscopy in terms of depth perception, image sharpness and image contrast. Additionally, increased impairment caused by ghosting effects could not be detected in 3D laparoscopy. Evaluation of image quality was independent of experience in laparoscopic surgery. However, use of a head-mounted 3D display resulted in a significant impairment of surgeon's comfort when compared to 2D laparoscopy. Thereby, the greatest impairment was caused by ear discomfort.

CONCLUSIONS

This is the first study examining the effect of a head-mounted 3D system on the performance of laparoscopy in a randomized controlled trial (RCT) showing a clear advantage of this system in surgical performance as well as in depth perception and image quality.

What Are the Advantages of 3D Cameras in Gynaecological Laparoscopy?

Introduction Minimally invasive operative techniques are being used increasingly in gynaecological surgery. The expansion of the laparoscopic operation spectrum is in part the result of improved imaging. This study investigates the practical advantages of using 3D cameras in routine surgical practice. Materials and Methods Two different 3-dimensional camera systems were compared with a 2-dimensional HD system; the operating surgeon's experiences were documented immediately postoperatively using a questionnaire. Results Significant advantages were reported for suturing and cutting of anatomical structures when using the 3D compared to 2D camera systems. There was only a slight advantage for coagulating. The use of 3D cameras significantly improved the general operative visibility and in particular the representation of spacial depth compared to 2-dimensional images. There was not a significant advantage for image width. Depiction of adhesions and retroperitoneal neural structures was significantly improved by the stereoscopic cameras, though this did not apply to blood vessels, ureter, uterus or ovaries. Conclusion 3-dimensional cameras were particularly advantageous for the depiction of fine anatomical structures due to improved spacial depth representation compared to 2D systems. 3D cameras provide the operating surgeon with a monitor image that more closely resembles actual anatomy, thus simplifying laparoscopic procedures.

Retroperitoneal Teratoma in an Adult: A Potential Pitfall in the Differential Diagnosis of Adrenal Myelolipoma

We report a 32-year-old female case of a right adrenal gland mass detected on CT scan at medical checkup. CT and MRI showed a mass of 5.1 cm made of fat and calcification in the right adrenal gland, leading to the clinical diagnosis of adrenal myelolipoma. Because of its relatively large size and the patient's desire, the patient underwent gasless single-port retroperitoneoscopic adrenalectomy using the RoboSurgeon system. Histopathological examination revealed that the cystic tumor is composed of keratinized epidermis, mature fat, nerve, cartilage, bone, and sebaceous glands compressing the normal adrenal gland, leading to the diagnosis of retroperitoneal mature cystic teratoma. The patient remains free of recurrence 29 months after surgery. Retroperitoneal teratoma is relatively rare but clinically important because of high possibility of malignancy. In a case of an adrenal mass difficult to clinically distinguish retroperitoneal teratoma from adrenal myelolipoma, surgical resection via a minimally invasive approach would be the best therapeutic option.

Live volumetric (4D) visualization and guidance of in vivo human ophthalmic surgery with intraoperative optical coherence tomography

Minimally-invasive microsurgery has resulted in improved outcomes for patients. However, operating through a microscope limits depth perception and fixes the visual perspective, which result in a steep learning curve to achieve microsurgical proficiency. We introduce a surgical imaging system employing four-dimensional (live volumetric imaging through time) microscope-integrated optical coherence tomography (4D MIOCT) capable of imaging at up to 10 volumes per second to visualize human microsurgery. A custom stereoscopic heads-up display provides real-time interactive volumetric feedback to the surgeon. We report that 4D MIOCT enhanced suturing accuracy and control of instrument positioning in mock surgical trials involving 17 ophthalmic surgeons. Additionally, 4D MIOCT imaging was performed in 48 human eye surgeries and was demonstrated to successfully visualize the pathology of interest in concordance with preoperative diagnosis in 93% of retinal surgeries and the surgical site of interest in 100% of anterior segment surgeries. In vivo 4D MIOCT imaging revealed sub-surface pathologic structures and instrument-induced lesions that were invisible through the operating microscope during standard surgical maneuvers. In select cases, 4D MIOCT guidance was necessary to resolve such lesions and prevent post-operative complications. Our novel surgical visualization platform achieves surgeon-interactive 4D visualization of live surgery which could expand the surgeon’s capabilities.

Novel microscope-integrated stereoscopic heads-up display for intrasurgical optical coherence tomography

Intra-operative optical coherence tomography (OCT) requires a display technology which allows surgeons to visualize OCT data without disrupting surgery. Previous research and commercial intrasurgical OCT systems have integrated heads-up display (HUD) systems into surgical microscopes to provide monoscopic viewing of OCT data through one microscope ocular. To take full advantage of our previously reported real-time volumetric microscope-integrated OCT (4D MIOCT) system, we describe a stereoscopic HUD which projects a stereo pair of OCT volume renderings into both oculars simultaneously. The stereoscopic HUD uses a novel optical design employing spatial multiplexing to project dual OCT volume renderings utilizing a single micro-display. The optical performance of the surgical microscope with the HUD was quantitatively characterized and the addition of the HUD was found not to substantially effect the resolution, field of view, or pincushion distortion of the operating microscope. In a pilot depth perception subject study, five ophthalmic surgeons completed a pre-set dexterity task with 50.0% (SD = 37.3%) higher success rate and in 35.0% (SD = 24.8%) less time on average with stereoscopic OCT vision compared to monoscopic OCT vision. Preliminary experience using the HUD in 40 vitreo-retinal human surgeries by five ophthalmic surgeons is reported, in which all surgeons reported that the HUD did not alter their normal view of surgery and that live surgical maneuvers were readily visible in displayed stereoscopic OCT volumes.

A novel interactive educational system in the operating room--the IE system

BACKGROUND

The shortage of surgeon is one of the serious problems in Japan. To solve the problem, various efforts have been undertaken to improve surgical education and training. However, appropriate teaching methods in the operating room have not been well established. The aim of this study is to assess the utility of a novel interactive educational (IE) system for surgical education on urologic surgeries in the operating room.

METHODS

A total of 20 Japanese medical students were educated on urologic surgery using the IE system in the operating room. The IE system consists of two parts. The first is three-dimensional (3D) magnified vision of the operative field using a 3D head-mounted display and a 3D endoscope. The second is interactive educative communication between medical students and surgeons using a small-sized wireless communication device. The satisfaction level with the IE system and the physical burden on medical students was examined via questionnaire.

RESULTS

All students utilized the IE system in urologic surgery and responded to the survey. Most students were satisfied with the IE system. They also felt more welcomed by the surgeon when using the IE system than when not using it. No major unpleasant symptoms were observed but five students (25 %) experienced mild eye fatigue as a result of viewing the medical images.

CONCLUSIONS

The IE system has the potential to motivate students to become interested in surgery and could be an efficient method of surgical education in the operating room.

Binocular Goggle Augmented Imaging and Navigation System provides real-time fluorescence image guidance for tumor resection and sentinel lymph node mapping

The inability to identify microscopic tumors and assess surgical margins in real-time during oncologic surgery leads to incomplete tumor removal, increases the chances of tumor recurrence, and necessitates costly repeat surgery. To overcome these challenges, we have developed a wearable goggle augmented imaging and navigation system (GAINS) that can provide accurate intraoperative visualization of tumors and sentinel lymph nodes in real-time without disrupting normal surgical workflow. GAINS projects both near-infrared fluorescence from tumors and the natural color images of tissue onto a head-mounted display without latency. Aided by tumor-targeted contrast agents, the system detected tumors in subcutaneous and metastatic mouse models with high accuracy (sensitivity = 100%, specificity = 98% ± 5% standard deviation). Human pilot studies in breast cancer and melanoma patients using a near-infrared dye show that the GAINS detected sentinel lymph nodes with 100% sensitivity. Clinical use of the GAINS to guide tumor resection and sentinel lymph node mapping promises to improve surgical outcomes, reduce rates of repeat surgery, and improve the accuracy of cancer staging.

Three-dimensional versus two-dimensional vision in laparoscopy: a systematic review

BACKGROUND

Laparoscopic surgery is widely used, and results in accelerated patient recovery time and hospital stay were compared with laparotomy. However, laparoscopic surgery is more challenging compared with open surgery, in part because surgeons must operate in a three-dimensional (3D) space through a two-dimensional (2D) projection on a monitor, which results in loss of depth perception. To counter this problem, 3D imaging for laparoscopy was developed. A systematic review of the literature was performed to assess the effect of 3D laparoscopy.

METHODS

A systematic search of the literature was conducted to identify randomized controlled trials that compared 3D with 2D laparoscopy. The search was accomplished in accordance with the PRISMA guidelines using the PubMed, EMBASE, and The Cochrane Library electronic databases. No language or year of publication restrictions was applied. Data extracted were cohort size and characteristics, skill trained or operation performed, instrument used, outcome measures, and conclusions. Two independent authors performed the search and data extraction.

RESULTS

Three hundred and forty articles were screened for eligibility, and 31 RCTs were included in the review. Three trials were carried out in a clinical setting, and 28 trials used a simulated setting. Time was used as an outcome measure in all of the trials, and number of errors was used in 19 out of 31 trials. Twenty-two out of 31 trials (71%) showed a reduction in performance time, and 12 out of 19 (63%) showed a significant reduction in error when using 3D compared to 2D.

CONCLUSIONS

Overall, 3D laparoscopy appears to improve speed and reduce the number of performance errors when compared to 2D laparoscopy. Most studies to date assessed 3D laparoscopy in simulated settings, and the impact of 3D laparoscopy on clinical outcomes has yet to be examined.

Patient's Self-monitoring of Transurethral Surgical Images Using a Head-mounted Display

We present an application of head-mounted display (HMD) to patient's self-monitoring of transurethral resection of bladder tumor (TURB). Six patients wore the HMD as an imaging monitor to view the operation in real-time during their TURB. Following the operation, the patients completed a questionnaire that evaluates understanding of the state of their disease and satisfaction with the HMD. As a result, monitoring the operation in real time through the HMD helped to increase patients' understanding of the state of their disease and satisfaction. For selected patients, the use of HMD could help to increase the patient's understanding of their disease.

A novel approach to patient self-monitoring of sonographic examinations using a head-mounted display

OBJECTIVES

Patients' use of a head-mounted display during their sonographic examinations could provide them with information about their diseases in real time and might help improve "patient-centered care." We conducted this prospective study to evaluate the feasibility of a modern head-mounted display for patient self-monitoring of sonographic examinations.

METHODS

In November and December 2013, 58 patients were enrolled. Patients wore a head-mounted display (HMZ-T2; Sony Corporation, Tokyo, Japan) during their sonographic examinations and watched their own images in real time. After the sonographic examinations, the patients completed a questionnaire, in which they evaluated the utility of the head-mounted display, their understanding of their diseases, their satisfaction with using the head-mounted display, and any adverse events. Until November 26, 2013, patients' names were requested on the questionnaire; after that date, the questionnaire was changed to be anonymous.

RESULTS

Of the 58 patients, 56 (97%) elected to participate in this study. The head-mounted display was reported to have good image quality by 42 patients (75%) and good wearability by 39 (70%). Thirty-six patients (64%) reported they had deepened their understanding of their diseases. There were no major complications, and only 2 patients (4%) had mild eye fatigue. There was no significant association between questionnaire results and patient characteristics. None of the questionnaire results changed significantly after the questionnaire was made anonymous.

CONCLUSIONS

The use of a modern head-mounted display by patients during sonographic examinations provided good image quality with acceptable wearability. It could deepen their understanding of their diseases and help develop patient-centered care.

A pneumatic laparoscope holder controlled by head movement

BACKGROUND

In traditional laparoscopic surgery, the laparoscope is handled by a camera assistant according to verbal instructions from the surgeon. Thus there is a strong need for a laparoscope holder who intuitively provides the appropriate view with excellent stability.

METHODS

A pneumatically driven robotic arm was developed to hold and manipulate a laparoscope. The robotic arm is operated by the user's head movement, which is measured with gyroscopes attached to the operator's head and body.

RESULTS

We confirmed experimentally that head tracking can be performed accurately using the proposed method. The experimental results indicated that the robotic camera holder has sufficient dynamic characteristics to quickly follow the operator's head movement.

CONCLUSIONS

A laparoscope holder control system has been developed. In this system, a laparoscope is held by a pneumatically driven robotic arm that is controlled to follow the operator's head movement. The experimental results prove the effectiveness of the system. Copyright © 2014 John Wiley & Sons, Ltd.

Head-Mounted Display for a Personal Integrated Image Monitoring System: Ureteral Stent Placement

The personal head-mounted display (HMD) has emerged as a novel image monitoring system. We present here the application of a high-definition organic electroluminescent binocular HMD in ureteral stent placement. Our HMD system displayed multiple forms of information such as integrated, sharp, high-contrast images using a four-split screen or a picture-in-picture technique both seamlessly and synchronously. When both the operator and the assistant wore an HMD, they could continuously and simultaneously monitor the cystoscopic and fluoroscopic images in an ergonomically natural position. Furthermore, each participant was able to modulate the displayed images depending on the procedure. In all five cases, both the operator and the assistant successfully used this system with no unfavorable event. No participants experienced any HMD wear-related adverse effects. We therefore believe this HMD system might be potentially beneficial during ureteral stent placement procedures. Furthermore, it is compact, easily introduced and affordable.

A three-dimensional head-mounted display system (RoboSurgeon system) for gasless laparoendoscopic single-port partial cystectomy

We developed a new three-dimensional (3D) head-mounted display (HMD) system (RoboSurgeon system) that combines a high-definition 3D organic electroluminescent HMD with a high-definition 3D endoscope and applies it to minimally invasive surgery. This system presents the surgeon with a higher quality of magnified 3D imagery in front of the eyes, regardless of head position. We report 5 cases of RoboSurgeon gasless laparoendoscopic single-port partial cystectomy, which is carried out as part of our selective bladder-sparing protocol, with a technique utilizing both an intravesical and extravesical approach. While carrying out the surgery, the system provides the surgeon with both excellent 3D imagery of the operative field and clear imagery of the cystoscopy. All procedures were safely completed and there were no complications except for a case of postoperative lymphorrhea. Our experience shows that the 3D HMD system might facilitate maneuverability and safety in various minimally invasive procedures.

Clinical application of a modern high-definition head-mounted display in sonography

Because of the remarkably improved image quality and wearability of modern head-mounted displays, a monitoring system using a head-mounted display rather than a fixed-site monitor for sonographic scanning has the potential to improve the diagnostic performance and lessen the examiner's physical burden during a sonographic examination. In a preclinical setting, 2 head-mounted displays, the HMZ-T2 (Sony Corporation, Tokyo, Japan) and the Wrap1200 (Vuzix Corporation, Rochester, NY), were found to be applicable to sonography. In a clinical setting, the feasibility of the HMZ-T2 was shown by its good image quality and acceptable wearability. This modern device is appropriate for clinical use in sonography.

Integrated image monitoring system using head-mounted display for gasless single-port clampless partial nephrectomy

A novel head-mounted display (HMD) offers a higher quality of endoscopic imagery in front of the eyes regardless of head position. We present an application of the HMD system as a personal integrated multi-image monitoring system in gasless single-port clampless partial nephrectomy (PN). Our HMD system displayed multiple forms of information as integrated, sharp, high-contrast images both seamlessly and synchronously using a four-split screen. The surgeon wearing an HMD display could continuously and simultaneously monitor the endoscopic, three-dimensional (3D) video and intraoperative ultrasound images. In addition, the operator can rotate the 3D video image using fingertip movements on the finger tracking system. All two clampless partial nephrectomies were safely completed within the operative time, blood loss was within usual limits and there were no complications. The integrated image HMD system might facilitate maneuverability and safety in minimally invasive clampless PN.

Integrated image navigation system using head-mounted display in "RoboSurgeon" endoscopic radical prostatectomy

The safety and efficacy of minimally invasive surgery relies on visual information. We aimed to develop an integrated image navigation system (RoboSurgeon System) that combines head-mounted displays (HMDs) with multiple image modalities, and assessed its feasibility in 5 prostate cancer patients who underwent gasless single-port endoscopic radical prostatectomy. A robotically manipulated transrectal ultrasound (TRUS) system was used. In all cases, preoperative magnetic resonance (MR) images and intraoperative real-time images of an endoscope, TRUS, and HMD-mounted camera were integrated and displayed synchronously on each HMD in a four-split screen mode during the entire process. The TRUS helped identify the boundary with the adjacent structures endoscopically in reference to MR images. There were no negative incidents in intraoperative or postoperative courses. Integrated image navigation using HMDs as individualized monitors is feasible in the natural ergonomic position and may be beneficial to identify correct dissection planes. The efficacy of the RoboSurgeon System deserves further evaluation.

A head-mounted display-based personal integrated-image monitoring system for transurethral resection of the prostate

The head-mounted display (HMD) is a new image monitoring system. We developed the Personal Integrated-image Monitoring System (PIM System) using the HMD (HMZ-T2, Sony Corporation, Tokyo, Japan) in combination with video splitters and multiplexers as a surgical guide system for transurethral resection of the prostate (TURP). The imaging information obtained from the cystoscope, the transurethral ultrasonography (TRUS), the video camera attached to the HMD, and the patient's vital signs monitor were split and integrated by the PIM System and a composite image was displayed by the HMD using a four-split screen technique. Wearing the HMD, the lead surgeon and the assistant could simultaneously and continuously monitor the same information displayed by the HMD in an ergonomically efficient posture. Each participant could independently rearrange the images comprising the composite image depending on the engaging step. Two benign prostatic hyperplasia (BPH) patients underwent TURP performed by surgeons guided with this system. In both cases, the TURP procedure was successfully performed, and their postoperative clinical courses had no remarkable unfavorable events. During the procedure, none of the participants experienced any HMD-wear related adverse effects or reported any discomfort.

Gastric Contraction Imaging System Using a 3-D Endoscope

This paper presents a gastric contraction imaging system for assessment of gastric motility using a 3-D endoscope. Gastrointestinal diseases are mainly based on morphological abnormalities. However, gastrointestinal symptoms are sometimes apparent without visible abnormalities. One of the major factors for these diseases is abnormal gastrointestinal motility. For assessment of gastric motility, a gastric motility imaging system is needed. To assess the dynamic motility of the stomach, the proposed system measures 3-D gastric contractions derived from a 3-D profile of the stomach wall obtained with a developed 3-D endoscope. After obtaining contraction waves, their frequency, amplitude, and speed of propagation can be calculated using a Gaussian function. The proposed system was evaluated for 3-D measurements of several objects with known geometries. The results showed that the surface profiles could be obtained with an error of [Formula: see text] of the distance between two different points on images. Subsequently, we evaluated the validity of a prototype system using a wave simulated model. In the experiment, the amplitude and position of waves could be measured with 1-mm accuracy. The present results suggest that the proposed system can measure the speed and amplitude of contractions. This system has low invasiveness and can assess the motility of the stomach wall directly in a 3-D manner. Our method can be used for examination of gastric morphological and functional abnormalities.