Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery

Training the Next Generation of Transplant Surgeons With a 3-Dimensional Trainer: A Pilot Study

Background

In the United States, no published guidelines promote exposure to technical variants (ie, living donor or split liver) during transplant fellowship. Simulation with hands-on liver models may improve training in transplantation. This pilot study addressed 3 overall goals (material and model creation tools, recruitment rates and assessment of workload, and protocol adherence).

Methods

A patient-specific hands-on liver model was constructed from clinical imaging, and it needed to be resilient and realistic. Multiple types of materials were tested between January 2020 and August 2022. Participants were recruited stepwise. A left lateral segmentectomy simulation was conducted between August 2022 and December 2022 to assess protocol adherence.

Results

Digital anatomy 3-dimensional printing was considered the best option for the hands-on liver model. The recruitment rate was 100% and 47% for junior attendings and surgical residents, respectively. Ten participants were included and completed all the required surveys. Seven (70%) and 6 (60%) participants "agreed" that the overall quality of the model and the material were acceptable for surgical simulation. Five participants (50%) "agreed" that the training improved their surgical skills. Nine participants (90%) "strongly agreed" that similar sessions should be included in surgical training programs.

Conclusions

Three-dimensional hands-on liver models have the advantage of tactile feedback and were rated favorably as a potential training tool. Study enrollment for further studies is possible with the support of leadership. Rigorous multicenter designs should be developed to measure the actual impact of 3-dimensional hands-on liver models on surgical training.

A contemporary view on vascular resections and reconstruction during hepatectomies

Oncological hepatic surgery carries the possibility to perform vascular reconstructions for advanced tumours with vessel invasion since surgery often represents the only potentially curative approach for these tumours. An extended review was conducted in an attempt to understand and clarify the latest trends in hepatectomies with vascular resections. We searched bibliographic databases including PubMed, Scopus, references from bibliographies and Cochrane Library. Information and outcomes from worldwide clinical trials were collected from qualified institutions performing hepatectomies with vascular resection and reconstruction. Careful patient selection and thorough preoperative imaging remain crucial for correct and safe surgical planning. A literature analysis shows that vascular resections carry different indications in different diseases. Despite significant advances made in imaging techniques and technical skills, reports of hepatectomies with vascular resections are still associated with high postoperative morbidity and mortality. The trend of complex liver resection with vascular resection is constantly on the increase, but more profound knowledge as well as further trials are required. Recent technological developments in multiple fields could surely provide novel approaches and enhance a new era of digital imaging and intelligent hepatic surgery.

Synchronization of a Virtual Reality Scenario to Uterine Contractions for Labor Pain Management: Development Study and Randomized Controlled Trial

Background: Labor is described as one of the most painful events women can experience through their lives, and labor pain shows unique features and rhythmic fluctuations. Purpose: The present study aims to evaluate virtual reality (VR) analgesic interventions for active labor with biofeedback-based VR technologies synchronized to uterine activity. Materials and Methods: We developed a VR system modeled on uterine contractions by connecting it to cardiotocographic equipment. We conducted a randomized controlled trial on a sample of 74 cases and 80 controls during active labor. Results: Results of the study showed a significant reduction of pain scores compared with both preintervention scores and to control group scores; a significant reduction of anxiety levels both compared with preintervention assessment and to control group and significant reduction in fear of labor experience compared with controls. Conclusion: VR may be considered as an effective nonpharmacological analgesic technique for the treatment of pain and anxiety and fear of childbirth experience during labor. The developed system could improve personalization of care, modulating the multisensory stimulation tailored to labor progression. Further studies are needed to compare the synchronized VR system to uterine activity and unsynchronized VR interventions.

SpecReFlow: an algorithm for specular reflection restoration using flow-guided video completion

Purpose

Specular reflections (SRs) are highlight artifacts commonly found in endoscopy videos that can severely disrupt a surgeon's observation and judgment. Despite numerous attempts to restore SR, existing methods are inefficient and time consuming and can lead to false clinical interpretations. Therefore, we propose the first complete deep-learning solution, SpecReFlow, to detect and restore SR regions from endoscopy video with spatial and temporal coherence.

Approach

SpecReFlow consists of three stages: (1) an image preprocessing stage to enhance contrast, (2) a detection stage to indicate where the SR region is present, and (3) a restoration stage in which we replace SR pixels with an accurate underlying tissue structure. Our restoration approach uses optical flow to seamlessly propagate color and structure from other frames of the endoscopy video.

Results

Comprehensive quantitative and qualitative tests for each stage reveal that our SpecReFlow solution performs better than previous detection and restoration methods. Our detection stage achieves a Dice score of 82.8% and a sensitivity of 94.6%, and our restoration stage successfully incorporates temporal information with spatial information for more accurate restorations than existing techniques.

Conclusions

SpecReFlow is a first-of-its-kind solution that combines temporal and spatial information for effective detection and restoration of SR regions, surpassing previous methods relying on single-frame spatial information. Future work will look to optimizing SpecReFlow for real-time applications. SpecReFlow is a software-only solution for restoring image content lost due to SR, making it readily deployable in existing clinical settings to improve endoscopy video quality for accurate diagnosis and treatment.

Three-dimensional Liver Model Application for Liver Transplantation

BACKGROUND

Children are removed from the liver transplant waitlist because of death or progressive illness. Size mismatch accounts for 30% of organ refusal. This study aimed to demonstrate that 3-dimensional (3D) technology is a feasible and accurate adjunct to organ allocation and living donor selection process.

METHODS

This prospective multicenter study included pediatric liver transplant candidates and living donors from January 2020 to February 2023. Patient-specific, 3D-printed liver models were used for anatomic planning, real-time evaluation during organ procurement, and surgical navigation. The primary outcome was to determine model accuracy. The secondary outcome was to determine the impact of outcomes in living donor hepatectomy. Study groups were analyzed using propensity score matching with a retrospective cohort.

RESULTS

Twenty-eight recipients were included. The median percentage error was -0.6% for 3D models and had the highest correlation to the actual liver explant (Pearson's R = 0.96, P < 0.001) compared with other volume calculation methods. Patient and graft survival were comparable. From 41 living donors, the median percentage error of the allograft was 12.4%. The donor-matched study group had lower central line utilization (21.4% versus 75%, P = 0.045), shorter length of stay (4 versus 7 d, P = 0.003), and lower mean comprehensive complication index (3 versus 21, P = 0.014).

CONCLUSIONS

Three-dimensional volume is highly correlated with actual liver explant volume and may vary across different allografts for living donation. The addition of 3D-printed liver models during the transplant evaluation and organ procurement process is a feasible and safe adjunct to the perioperative decision-making process.

A Systematic Review of Immersive Virtual Reality for Nontechnical Skills Training in Surgery

OBJECTIVE

Immersive virtual reality (IVR) can be utilized to provide low cost and easily accessible simulation on all aspects of surgical education. In addition to technical skills training in surgery, IVR simulation has been utilized for nontechnical skills training in domains such as clinical decision-making and pre-operative planning. This systematic review examines the current literature on the effectiveness of IVR for nontechnical skill acquisition in surgical education.

DESIGN

A literature search was performed using MEDLINE, EMBASE, and Web of Science for primary studies published between January 1, 1995 and February 9, 2022. Four reviewers screened titles, abstracts, full texts, extracted data, and analyzed included studies to answer 5 key questions: How is IVR being utilized in nontechnical skills surgical education? What is the methodological quality of studies? What technologies are being utilized? What metrics are reported? What are the findings of these studies?

RESULTS

The literature search yielded 2340 citations, with 12 articles included for qualitative synthesis. Of included articles, 33% focused on clinical decision-making and 67% on anatomy/pre-operative planning. Motion sickness was a recorded metric in 25% of studies, with an aggregate incidence of 13% (11/87). An application score was reported in 33% and time to completion in 16.7%. A commercially developed application was utilized in 25%, while 75% employed a noncommercial application. The Oculus Rift was used in 41.7% of studies, HTC Vive in 25%, Samsung Gear in 16.7% of studies, Google Daydream in 8%, and 1 study did not report. The mean Medical Education Research Quality Instrument (MERSQI) score was 10.3 ± 2.3 (out of 18). In all studies researching clinical decision-making, participants preferred IVR to conventional teaching methods and in a nonrandomized control study it was found to be more effective. Averaged across all studies, mean scores were 4.33 for enjoyment, 4.16 for utility, 4.11 for usability, and 3.73 for immersion on a 5-point Likert scale.

CONCLUSIONS

The IVR nontechnical skills applications for surgical education are designed for clinical decision-making or anatomy/pre-operative planning. These applications are primarily noncommercially produced and rely upon a diverse array of HMDs for content delivery, suggesting that development is primarily coming from within academia and still without clarity on optimal utilization of the technology. Excitingly, users find these applications to be immersive, enjoyable, usable, and of utility in learning. Although a few studies suggest that IVR is additive or superior to conventional teaching or imaging methods, the data is mixed and derived from studies with weak design. Motion sickness with IVR remains a complication of IVR use needing further study to determine the cause and means of mitigation.

Evaluation of a Virtual Reality-Based Open Educational Resource Software

OBJECTIVES

Virtual reality (VR) teaching methods have potential to support medical students acquire increasing amounts of knowledge. EVENT (Easy VR EducatioN Tool) is an open educational resource software for immersive VR environments, which is designed for use without programming skills. In this work, EVENT was used in a medical student VR course on pancreatic cancer.

METHODS

Medical students were invited to participate in the course. Before and after VR simulation, participants completed a multiple-choice knowledge assessment, with a maximum score of 10, and a VR experience questionnaire. The primary endpoint compared pre- and post-VR simulation test scores. Secondary endpoints included usability and factors that could affect learning growth and test results.

RESULTS

Data from 117 of the 135 participating students was available for analysis. Student test scores improved by an average of 3.4 points (95% CI 3.1-3.7, P < 0.001) after VR course. The secondary endpoints of gender, age, prior knowledge regarding the medical subject, professional training completed in the medical field, video game play, three-dimensional imagination skills, or cyber-sickness had no major impact on test scores or final ranking (top or bottom 25%). The 27 students whose post-VR simulation test scores ranked in the top 25% had no prior experience with VR. The average System Usability Scale score was 86.1, which corresponds to an excellent outcome for user-friendliness. Questionnaire responses post-VR simulation show students (81.2% [95/117]) interest in more VR options in medical school.

CONCLUSIONS

We present a freely available software that allows for the development of VR teaching lessons without programming skills.

Full-sized realistic 3D printed models of liver and tumour anatomy: a useful tool for the clinical medicine education of beginning trainees

BACKGROUND

Simulation-based medical education (SBME) and three-dimensional printed (3DP) models are increasingly used in continuing medical education and clinical training. However, our understanding of their role and value in improving trainees' understanding of the anatomical and surgical procedures associated with liver surgery remains limited. Furthermore, gender bias is also a potential factor in the evaluation of medical education. Therefore, the aim of this study was to evaluate the educational benefits trainees receive from the use of novel 3DP liver models while considering trainees' experience and gender.

METHODS

Full-sized 3DP liver models were developed and printed using transparent material based on anonymous CT scans. We used printed 3D models and conventional 2D CT scans of the liver to investigate thirty trainees with various levels of experience and different genders in the context of both small group teaching and formative assessment. We adopted a mixed methods approach involving both questionnaires and focus groups to collect the views of different trainees and monitors to assess trainees' educational benefits and perceptions after progressing through different training programs. We used Objective Structured Clinical Examination (OSCE) and Likert scales to support thematic analysis of the responses to the questionnaires by trainees and monitors, respectively. Descriptive analyses were conducted using SPSS statistical software version 21.0.

RESULTS

Overall, a 3DP model of the liver is of great significance for improving trainees' understanding of surgical procedures and cooperation during operation. After viewing the personalized full-sized 3DP liver model, all trainees at the various levels exhibited significant improvements in their understanding of the key points of surgery (p < 0.05), especially regarding the planned surgical procedure and key details of the surgical procedures. More importantly, the trainees exhibited higher levels of satisfaction and self-confidence during the operation regardless of gender. However, with regard to gender, the results showed that the improvement of male trainees after training with the 3DP liver model was more significant than that of female trainees in understanding and cooperation during the surgical procedure, while no such trend was found with regard to their understanding of the base knowledge.

CONCLUSION

Trainees and monitors agreed that the use of 3DP liver models was acceptable. The improvement of the learning effect for practical skills and theoretical understanding after training with the 3DP liver models was significant. This study also indicated that training with personalized 3DP liver models can improve all trainees' presurgical understanding of liver tumours and surgery and males show more advantage in understanding and cooperation during the surgical procedure as compared to females. Full-sized realistic 3DP models of the liver are an effective auxiliary teaching tool for SBME teaching in Chinese continuing medical education.

Role of preoperative 3D rendering for minimally invasive parenchyma sparing liver resections

BACKGROUND

3D rendering (3DR) represents a promising approach to plan surgical strategies. The study aimed to compare the results of minimally invasive liver resections (MILS) in patients with 3DR versus conventional 2D CT-scan.

METHODS

We performed 118 3DR for various indications; the patients underwent a preoperative tri-phasic CT-scan and rendered with Synapse3D® Software. Fifty-six patients undergoing MILS with pre-operative 3DR were compared to a similar cohort of 127 patients undergoing conventional pre-operative 2D CT-scan using the propensity score matching (PSM) analysis.

RESULTS

The 3DR mandated pre-operative surgical plan variations in 33.9% cases, contraindicated surgery in 12.7%, providing a new surgical indication in 5.9% previously excluded cases. PSM identified 39 patients in both groups with comparable results in terms of conversion rates, blood loss, blood transfusions, parenchymal R1-margins, grade ≥3 Clavien-Dindo complications, 90-days mortality, and hospital stay respectively in 3DR and conventional 2D. Operative time was significantly increased in the 3DR group (402 vs. 347 min, p = 0.020). Vascular R1 resections were 25.6% vs 7.7% (p = 0.068), while the conversion rate was 0% vs 10.2% (p = 0.058), respectively, for 3DR group vs conventional 2D.

CONCLUSION

3DR may help in surgical planning increasing resectability rate while reducing conversion rates, allowing the precise identification of anatomical landmarks in minimally invasive parenchyma-preserving liver resections.

Patient-Specific 3D Printed Soft Models for Liver Surgical Planning and Hands-On Training

Background: Pre-surgical simulation-based training with three-dimensional (3D) models has been intensively developed in complex surgeries in recent years. This is also the case in liver surgery, although with fewer reported examples. The simulation-based training with 3D models represents an alternative to current surgical simulation methods based on animal or ex vivo models or virtual reality (VR), showing reported advantages, which makes the development of realistic 3D-printed models an option. This work presents an innovative, low-cost approach for producing patient-specific 3D anatomical models for hands-on simulation and training. Methods: The article reports three paediatric cases presenting complex liver tumours that were transferred to a major paediatric referral centre for treatment: hepatoblastoma, hepatic hamartoma and biliary tract rhabdomyosarcoma. The complete process of the additively manufactured liver tumour simulators is described, and the different steps for the correct development of each case are explained: (1) medical image acquisition; (2) segmentation; (3) 3D printing; (4) quality control/validation; and (5) cost. A digital workflow for liver cancer surgical planning is proposed. Results: Three hepatic surgeries were planned, with 3D simulators built using 3D printing and silicone moulding techniques. The 3D physical models showed highly accurate replications of the actual condition. Additionally, they proved to be more cost-effective in comparison with other models. Conclusions: It is demonstrated that it is possible to manufacture accurate and cost-effective 3D-printed soft surgical planning simulators for treating liver cancer. The 3D models allowed for proper pre-surgical planning and simulation training in the three cases reported, making it a valuable aid for surgeons.

Laparoscopic Microwave Ablation: Which Technologies Improve the Results

Liver resection is the best treatment for hepatocellular carcinoma (HCC) when resectable. Unfortunately, many patients with HCC cannot undergo liver resection. Percutaneous thermoablation represents a valid alternative for inoperable neoplasms and for small HCCs, but it is not always possible to accomplish it. In cases where the percutaneous approach is not feasible (not a visible lesion or in hazardous locations), laparoscopic thermoablation may be indicated. HCC diagnosis is commonly obtained from imaging modalities, such as CT and MRI, However, the interpretation of radiological images, which have a two-dimensional appearance, during the surgical procedure and in particular during laparoscopy, can be very difficult in many cases for the surgeon who has to treat the tumor in a three-dimensional environment. In recent years, more technologies have helped surgeons to improve the results after ablative treatments. The three-dimensional reconstruction of the radiological images has allowed the surgeon to assess the exact position of the tumor both before the surgery (virtual reality) and during the surgery with immersive techniques (augmented reality). Furthermore, indocyanine green (ICG) fluorescence imaging seems to be a valid tool to enhance the precision of laparoscopic thermoablation. Finally, the association with laparoscopic ultrasound with contrast media could improve the localization and characteristics of tumor lesions. This article describes the use of hepatic three-dimensional modeling, ICG fluorescence imaging and laparoscopic ultrasound examination, convenient for improving the preoperative surgical preparation for personalized laparoscopic approach.

Virtual reality digital surgical planning for jaw reconstruction: a usability study

BACKGROUND

Digital surgical planning (DSP) has revolutionized the preparation and execution of the management of complex head and neck pathologies. The addition of virtual reality (VR) allows the surgeon to have a three-dimensional experience with six degrees of freedom for visualizing and manipulating objects. This pilot study describes the participants experience with the first head and neck reconstructive VR-DSP platform.

METHODS

An original VR-DSP platform has been developed for planning the ablation and reconstruction of head and neck pathologies. A prospective trial utilizing this platform involving reconstructive surgeons was performed. Participants conducted a simulated VR-DSP planning session, pre- and post-questionnaire as well as audio recordings allowing for qualitative analysis.

RESULTS

Thirteen consultant reconstructive surgeons representing three surgical backgrounds with varied experience were recruited. The majority of surgeons had no previous experience with VR. Based on the system usability score, the VR-DSP platform was found to have above average usability. The qualitative analysis demonstrated the majority had a positive experience. Participants identified some perceived barriers to implementing the VR-DSP platform.

CONCLUSIONS

Virtual reality-digital surgical planning is usable and acceptable to reconstructive surgeons. Surgeons were able to perform the steps in an efficient time despite limited experience. The addition of VR offers additional benefits to current VSP platforms. Based on the results of this pilot study, it is likely that VR-DSP will be of benefit to the reconstructive surgeon.

Effectiveness of an immersive virtual reality environment on curricular training for complex cognitive skills in liver surgery: a multicentric crossover randomized trial

BACKGROUND

Virtual reality (VR) is increasingly used in surgical education, but evidence of its benefits in complex cognitive training compared to conventional 3-dimensional (3D) visualization methods is lacking. The objective of this study is to assess the impact of 3D liver models rendered visible by VR or desktop interfaces (DIs) on residents' performance in clinical decision-making.

METHOD

From September 2020 to April 2021, a single-blinded, crossover randomized educational intervention trial was conducted at two university hospitals in Belgium and Italy. A proficiency-based stepwise curriculum for preoperative liver surgery planning was developed for general surgery residents. After completing the training, residents were randomized in one of two assessment sequences to evaluate ten real clinical scenarios.

RESULTS

Among the 50 participants, 46 (23 juniors/23 seniors) completed the training and were randomized. Forty residents (86.96%) achieved proficiency in decision-making. The accuracy of virtual surgical planning using VR was higher than that using DI in both groups A (8.43 ± 1.03 vs 6.86 ± 1.79, p < 0.001) and B (8.08 ± 0.9 vs 6.52 ± 1.37, p < 0.001).

CONCLUSION

Proficiency-based curricular training for liver surgery planning successfully resulted in the acquisition of complex cognitive skills. VR was superior to DI visualization of 3D models in decision-making.

CLINICALTRIALS

GOV ID

NCT04959630.

Leberchirurgie 4.0 - OP-Planung, Volumetrie, Navigation und Virtuelle Realität

Durch die Optimierung der konservativen Behandlung, die Verbesserung der bildgebenden Verfahren und die Weiterentwicklung der Operationstechniken haben sich das operative Spektrum sowie der Maßstab für die Resektabilität in Bezug auf die Leberchirurgie in den letzten Jahrzehnten deutlich verändert.

Dank zahlreicher technischer Entwicklungen, insbesondere der 3-dimensionalen Segmentierung, kann heutzutage die präoperative Planung und die Orientierung während der Operation selbst, vor allem bei komplexen Eingriffen, unter Berücksichtigung der patientenspezifischen Anatomie erleichtert werden.

Neue Technologien wie 3-D-Druck, virtuelle und augmentierte Realität bieten zusätzliche Darstellungsmöglichkeiten für die individuelle Anatomie. Verschiedene intraoperative Navigationsmöglichkeiten sollen die präoperative Planung im Operationssaal verfügbar machen, um so die Patientensicherheit zu erhöhen.

Dieser Übersichtsartikel soll einen Überblick über den gegenwärtigen Stand der verfügbaren Technologien sowie einen Ausblick in den Operationssaal der Zukunft geben.

Three-dimensional visualization and virtual reality simulation role in hepatic surgery: Further research warranted

Artificial intelligence (AI) is the study of algorithms that enable machines to analyze and execute cognitive activities including problem solving, object and word recognition, reduce the inevitable errors to improve the diagnostic accuracy, and decision-making. Hepatobiliary procedures are technically complex and the use of AI in perioperative management can improve patient outcomes as discussed below. Three-dimensional (3D) reconstruction of images obtained via ultrasound, computed tomography scan or magnetic resonance imaging, can help surgeons better visualize the surgical sites with added depth perception. Pre-operative 3D planning is associated with lesser operative time and intraoperative complications. Also, a more accurate assessment is noted, which leads to fewer operative complications. Images can be converted into physical models with 3D printing technology, which can be of educational value to students and trainees. 3D images can be combined to provide 3D visualization, which is used for preoperative navigation, allowing for more precise localization of tumors and vessels. Nevertheless, AI enables surgeons to provide better, personalized care for each patient.

Influence of Three-Dimensional Visual Reconstruction Technology Combined with Virtual Surgical Planning of CTA Images on Precise Resection of Liver Cancer in Hepatobiliary Surgery

Hepatobiliary malignancies, such as hepatocellular carcinoma (HCC) and biliary tract cancers, namely, gallbladder carcinoma and cholangiocarcinoma, are linked to a high rate of morbidity and mortality, depending on the phase of the disease. The intricate hepatobiliary anatomy and the need for accurate peroperative management, especially in patients with advanced liver disease, make these tumors difficult to treat. Surgical resection is a notable therapy for hepatobiliary cancers. Unnecessary or excessive liver excision influences patient rehabilitation, normal liver function, and postoperative complications. Hepatobiliary operations must therefore include accurate liver removal. The present advancements in imaging technology are aimed at improving the diagnostic efficacy of liver injury even more. Three-dimensional visual reconstruction is becoming more important in the diagnosis as well as treatment of a variety of disorders. In this paper, we proposed a novel three-dimensional visual reconstruction technology using enhanced nonuniform rational basis spline (ENURBS) combined with virtual surgical planning of Computed Tomography Angiography (CTA) images for precise liver cancer resection. The purpose of this project is to rebuild 2D CTA scan images of liver cancer into a 3D reconstructed model for efficient visualization and diagnosis of liver cancer and to prepare an effective preoperative surgical plan for precise liver excision based on a 3D recreated liver model. This method's performance is compared to that of 2D planning in terms of accuracy and time taken to complete the plan. It is concluded that our proposed technique outperforms the planning technique based on 2D images.

Patient-individualized resection planning in liver surgery using 3D print and virtual reality (i-LiVR)-a study protocol for a prospective randomized controlled trial

Background

A multitude of different diseases—benign and malign—can require surgery of the liver. The liver is an especially challenging organ for resection planning due to its unique and interindividually variable anatomy. This demands a high amount of mental imagination from the surgeon in order to plan accordingly - a skill, which takes years of training to acquire and which is difficult to teach. Since the volume of the functional remnant liver is of great importance, parenchyma sparing resections are favoured. 3D reconstructions of computed tomography imaging enable a more precise understanding of anatomy and facilitate resection planning. The modality of presentation of these 3D models ranges from 2D monitors to 3D prints and virtual reality applications.

Methods

The presented trial compares three different modes of demonstration of a 3D reconstruction of CT scans of the liver, which are 3D print, a demonstration on a regular computer screen or using a head-mounted virtual reality headset, with the current gold standard of viewing the CT scan on a computer screen. The group size was calculated with n=25 each. Patients with major liver resections in a laparoscopic or open fashion are eligible for inclusion. Main endpoint is the comparison of the quotient between planned resection volume and actual resection volume between these groups. Secondary endpoints include usability for the surgical team as well as patient specifics and perioperative outcome measures and teaching issues.

Discussion

The described study will give insight in systematic planning of liver resections and the comparison of different demonstration modalities of 3D reconstruction of preoperative CT scans and the preference of technology. Especially teaching of these demanding operations is underrepresented in prior investigations.

Trial registration

Prospective trials registration at the German Clinical Trials register with the registration number DRKS00027865. Registration Date: January 24, 2022.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06347-0.

Essential updates 2020/2021: Current topics of simulation and navigation in hepatectomy

With the development of three-dimensional (3D) simulation software, preoperative simulation technology is almost completely established. The remaining issue is how to recognize anatomy three-dimensionally. Extended reality is a newly developed technology with several merits for surgical application: no requirement for a sterilized display monitor, better spatial awareness, and the ability to share 3D images among all surgeons. Various technology or devices for intraoperative navigation have also been developed to support the safety and certainty of liver surgery. Consensus recommendations regarding indocyanine green fluorescence were determined in 2021. Extended reality has also been applied to intraoperative navigation, and artificial intelligence (AI) is one of the topics of real-time navigation. AI might overcome the problem of liver deformity with automatic registration. Including the issues described above, this article focuses on recent advances in simulation and navigation in liver surgery from 2020 to 2021.

Virtual Reality for Surgical Planning – Evaluation Based on Two Liver Tumor Resections

Purpose

For complex cases, preoperative surgical planning is a standard procedure to ensure patient safety and keep the surgery time to a minimum. Based on the available information, such as MRI or CT images, and prior anatomical knowledge the surgeons create their own mental 3D model of the organ of interest. This is challenging, requires years of training and an inherent uncertainty remains even for experienced surgeons.

Goal

Virtual reality (VR) is by nature excellent in showing spatial relationships through its stereoscopic displays. Therefore, it is well suited to be used to support the understanding of individual anatomy of patient-specific 3D organ models generated from MRI or CT data. Utilizing this potential, we developed a VR surgical planning tool that provides a 3D view of the medical data for better spatial understanding and natural interaction with the data in 3D space. Following a user-centered design process, in this first user study, we focus on usability, usefulness, and target audience feedback. Thereby, we also investigate the individual impact the tool and the 3D presentation of the organ have on the understanding of the 3D structures for the surgical team.

Methods

We employed the VR prototype for surgical planning using a standard VR setup to two real cases of patients with liver tumors who were scheduled for surgery at a University Hospital for Visceral Surgery. Surgeons (N = 4) used the VR prototype before the surgery to plan the procedure in addition to their regular planning process. We used semi-structured interviews before and after the surgery to explore the benefits and pitfalls of VR surgical planning.

Results

The participants used on average 14.3 min (SD = 3.59) to plan the cases in VR. The reported usability was good. Results from the interviews and observations suggest that planning in VR can be very beneficial for surgeons. They reported an improved spatial understanding of the individual anatomical structures and better identification of anatomical variants. Additionally, as the surgeons mentioned an improved recall of the information and better identification of surgical relevant structures, the VR tool has the potential to improve the surgery and patient safety.

New Approach to Accelerated Image Annotation by Leveraging Virtual Reality and Cloud Computing

Three-dimensional imaging is at the core of medical imaging and is becoming a standard in biological research. As a result, there is an increasing need to visualize, analyze and interact with data in a natural three-dimensional context. By combining stereoscopy and motion tracking, commercial virtual reality (VR) headsets provide a solution to this critical visualization challenge by allowing users to view volumetric image stacks in a highly intuitive fashion. While optimizing the visualization and interaction process in VR remains an active topic, one of the most pressing issue is how to utilize VR for annotation and analysis of data. Annotating data is often a required step for training machine learning algorithms. For example, enhancing the ability to annotate complex three-dimensional data in biological research as newly acquired data may come in limited quantities. Similarly, medical data annotation is often time-consuming and requires expert knowledge to identify structures of interest correctly. Moreover, simultaneous data analysis and visualization in VR is computationally demanding. Here, we introduce a new procedure to visualize, interact, annotate and analyze data by combining VR with cloud computing. VR is leveraged to provide natural interactions with volumetric representations of experimental imaging data. In parallel, cloud computing performs costly computations to accelerate the data annotation with minimal input required from the user. We demonstrate multiple proof-of-concept applications of our approach on volumetric fluorescent microscopy images of mouse neurons and tumor or organ annotations in medical images.

Virtual reality and 3D printing improve preoperative visualization of 3D liver reconstructions-results from a preclinical comparison of presentation modalities and user's preference

Background

Preoperative three-dimensional (3D) reconstructions for liver surgery planning have been shown to be effective in reduction of blood loss and operation time. However, the role of the ‘presentation modality’ is not well investigated. We present the first study to compare 3D PDFs, 3D printed models (PR) and virtual reality (VR) 3D models with regard to anatomical orientation and personal preferences in a high volume liver surgery center.

Methods

Thirty participants, 10 medical students, 10 residents, 5 fellows and 5 hepatopancreatobiliary (HPB) experts, assigned the tumor-bearing segments of 20 different patient’s individual liver reconstructions. Liver models were presented in a random order in all modalities. Time needed to specify the tumor location was recorded. In addition, a score was calculated factoring in correct, wrong and missing segment assignments. Furthermore, standardized test/questionnaires for spatial thinking and seeing, vegetative side effects and usability were completed.

Results

Participants named significantly more correct segments in VR (P=0.040) or PR (P=0.036) compared to PDF. Tumor assignment was significantly shorter with 3D PR models compared to 3D PDF (P<0.001) or VR application (P<0.001). Regardless of the modality, HPB experts were significantly faster (24±8 vs. 35±11 sec; P=0.014) and more often correct (0.87±0.12 vs. 0.83±0.15; P<0.001) than medical students. Test results for spatial thinking and seeing had no influence on time but on correctness of tumor assignment. Regarding usability and user experience the VR application achieved the highest scores without causing significant vegetative symptoms and was also the most preferred method (n=22, 73.3%) because of the multiple functions like scaling and change of transparency. Ninety percent (n=27) stated that this application can positively influence the operation planning.

Conclusions

3D PR models and 3D VR models enable a better and partially faster anatomical orientation than reconstructions presented as 3D PDFs. User’s preferred the VR application over the PR models and PDF. A prospective trial is needed to evaluate the different presentation modalities regarding intra- and postoperative outcomes.

Investigating the utility of VR for spatial understanding in surgical planning: evaluation of head-mounted to desktop display

Recent technological advances have made Virtual Reality (VR) attractive in both research and real world applications such as training, rehabilitation, and gaming. Although these other fields benefited from VR technology, it remains unclear whether VR contributes to better spatial understanding and training in the context of surgical planning. In this study, we evaluated the use of VR by comparing the recall of spatial information in two learning conditions: a head-mounted display (HMD) and a desktop screen (DT). Specifically, we explored (a) a scene understanding and then (b) a direction estimation task using two 3D models (i.e., a liver and a pyramid). In the scene understanding task, participants had to navigate the rendered the 3D models by means of rotation, zoom and transparency in order to substantially identify the spatial relationships among its internal objects. In the subsequent direction estimation task, participants had to point at a previously identified target object, i.e., internal sphere, on a materialized 3D-printed version of the model using a tracked pointing tool. Results showed that the learning condition (HMD or DT) did not influence participants' memory and confidence ratings of the models. In contrast, the model type, that is, whether the model to be recalled was a liver or a pyramid significantly affected participants' memory about the internal structure of the model. Furthermore, localizing the internal position of the target sphere was also unaffected by participants' previous experience of the model via HMD or DT. Overall, results provide novel insights on the use of VR in a surgical planning scenario and have paramount implications in medical learning by shedding light on the mental model we make to recall spatial structures.