Efficient EndoNeRF reconstruction and its application for data-driven surgical simulation

PURPOSE

The healthcare industry has a growing need for realistic modeling and efficient simulation of surgical scenes. With effective models of deformable surgical scenes, clinicians are able to conduct surgical planning and surgery training on scenarios close to real-world cases. However, a significant challenge in achieving such a goal is the scarcity of high-quality soft tissue models with accurate shapes and textures. To address this gap, we present a data-driven framework that leverages emerging neural radiance field technology to enable high-quality surgical reconstruction and explore its application for surgical simulations.

METHOD

We first focus on developing a fast NeRF-based surgical scene 3D reconstruction approach that achieves state-of-the-art performance. This method can significantly outperform traditional 3D reconstruction methods, which have failed to capture large deformations and produce fine-grained shapes and textures. We then propose an automated creation pipeline of interactive surgical simulation environments through a closed mesh extraction algorithm.

RESULTS

Our experiments have validated the superior performance and efficiency of our proposed approach in surgical scene 3D reconstruction. We further utilize our reconstructed soft tissues to conduct FEM and MPM simulations, showcasing the practical application of our method in data-driven surgical simulations.

CONCLUSION

We have proposed a novel NeRF-based reconstruction framework with an emphasis on simulation purposes. Our reconstruction framework facilitates the efficient creation of high-quality surgical soft tissue 3D models. With multiple soft tissue simulations demonstrated, we show that our work has the potential to benefit downstream clinical tasks, such as surgical education.

Exploring the use of driving simulation to improve robotic surgery simulator training: an observational case-control study

The correlation between driving skills and the ability to perform robotic surgery have not yet been discussed. Therefore, this study aimed to investigate the impact of driving skills on learning robotic surgery using a driving simulator and a robotic simulator. Sixty robot- and simulator-naïve participants were recruited: 30 with a driver's license and 30 without a driver's license. All participants completed a test on the driving simulator and learned four tasks using a robotic surgical simulator (dV-Trainer). On the driving simulator, the lap time in the driver's license group (D-Group) was significantly lower than that in the non-driver's license group (ND-Group) [217.93 ± 42.79 s vs. 271.24 ± 46.63 s, P < 0.001]. The average number of tires off track in the D-Group was lower than that in the ND-Group (0.13 ± 0.35 vs. 0.57 ± 0.63, P = 0.002). The baseline score of the D-Group on the robotic simulator was higher than that of the ND-Group (467.53 ± 107.62 vs. 385.53 ± 136.30, P = 0.022). In the Pick-and-Place-Clutching, Peg-Board-2, and Thread-the-Rings-1 tasks, the learning curve of the D-Group was steeper than that of the ND-Group. However, no significant difference was observed in the Match-Board-2 task. According to the lap time ranking, participants in the top tertile had a steeper learning curve than those in the bottom tertile, especially for the Pick-and-Place-Clutching and Peg-Board-2 tasks (P < 0.05). Significant differences were also found in the baseline and final stages of the Thread-the-Rings-1 task and in the initial stage of the Match-Board-2 task (P < 0.05). Students with a driver's license or better performance in racing games had more success in learning robotic surgery. Driving simulators may promote robotic surgery training.

Mandibular resection and defect reconstruction guided by a contour registration-based augmented reality system: A preclinical trial

The aim of this study was to verify the feasibility and accuracy of a contour registration-based augmented reality (AR) system in jaw surgery. An AR system was developed to display the interaction between virtual planning and images of the surgical site in real time. Several trials were performed with the guidance of the AR system and the surgical guide. The postoperative cone beam CT (CBCT) data were matched with the preoperatively planned data to evaluate the accuracy of the system by comparing the deviations in distance and angle. All procedures were performed successfully. In nine model trials, distance and angular deviations for the mandible, reconstructed fibula, and fixation screws were 1.62 ± 0.38 mm, 1.86 ± 0.43 mm, 1.67 ± 0.70 mm, and 3.68 ± 0.71°, 5.48 ± 2.06°, 7.50 ± 1.39°, respectively. In twelve animal trials, results of the AR system were compared with the surgical guide. Distance deviations for the bilateral condylar outer poles were 0.93 ± 0.63 mm and 0.81 ± 0.30 mm, respectively (p = 0.68). Distance deviations for the bilateral mandibular posterior angles were 2.01 ± 2.49 mm and 2.89 ± 1.83 mm, respectively (p = 0.50). Distance and angular deviations for the mandible were 1.41 ± 0.61 mm, 1.21 ± 0.18 mm (p = 0.45), and 6.81 ± 2.21°, 6.11 ± 2.93° (p = 0.65), respectively. Distance and angular deviations for the reconstructed tibiofibular bones were 0.88 ± 0.22 mm, 0.84 ± 0.18 mm (p = 0.70), and 6.47 ± 3.03°, 6.90 ± 4.01° (p = 0.84), respectively. This study proposed a contour registration-based AR system to assist surgeons in intuitively observing the surgical plan intraoperatively. The trial results indicated that this system had similar accuracy to the surgical guide.

Infrared thoracoscopic extended lobectomy (right upper lobe and S6) with intravenous indocyanine green administration using preoperative simulation to ensure safe surgical margin

Although it is crucial to ensure a sufficient surgical margin for a malignant neoplasm, we sometimes struggle to achieve this goal using a minimally invasive approach because it is difficult to palpate the tumor adequately via the small skin incision. To overcome this issue, we adopted a preoperative simulation method for a patient undergoing a right upper lobe and a posterior segmentectomy of the lower lobe (extended lobectomy) and obtained successful results. The discrepancy between the virtual and the actual surgical margins was 5 mm.

A deformation model of pulsating brain tissue for neurosurgery simulation

BACKGROUND AND OBJECTIVES

For neurological simulation, an accurate deformation model of brain tissue is of key importance for faithful visual feedback. Existing models, however, do not take into account intracranial pulsation, which degrades significantly the realism of visual feedback.

METHODS

In this paper, a finite element model incorporating intracranial pressure is proposed for simulating brain tissue deformation with pulsation. An implicit Euler method is developed to calculate the deformation of brain tissue. A circuit model of intracranial pressure dynamics is established based on cerebral blood and cerebrospinal fluid circulations. The intracranial pulsation of pressure is introduced into the deformation model, so that the simulated brain tissues pulsate with a rhythm in accord with the changes of intracranial pressure, which resembles real-life neurosurgery.

RESULTS AND CONCLUSIONS

The experimental implementation of the proposed deformation model and the calculation method shows that it provides realistic simulation of brain tissue pulsation and real-time performance is achieved on an ordinary computer for certain procedures of neurosurgery.

GPU-friendly data structures for real time simulation

Simulators for virtual surgery training need to perform complex calculations very quickly to provide realistic haptic and visual interactions with a user. The complexity is further increased by the addition of cuts to virtual organs, such as would be needed for performing tumor resection. A common method for achieving large performance improvements is to make use of the graphics hardware (GPU) available on most general-use computers. Programming GPUs requires data structures that are more rigid than on conventional processors (CPU), making that data more difficult to update. We propose a new method for structuring graph data, which is commonly used for physically based simulation of soft tissue during surgery, and deformable objects in general. Our method aligns all nodes of the graph in memory, independently from the number of edges they contain, allowing for local modifications that do not affect the rest of the structure. Our method also groups memory transfers so as to avoid updating the entire graph every time a small cut is introduced in a simulated organ. We implemented our data structure as part of a simulator based on a meshless method. Our tests show that the new GPU implementation, making use of the new graph structure, achieves a 10 times improvement in computation times compared to the previous CPU implementation. The grouping of data transfers into batches allows for a 80-90% reduction in the amount of data transferred for each graph update, but accounts only for a small improvement in performance. The data structure itself is simple to implement and allows simulating increasingly complex models that can be cut at interactive rates.

Parallelized collision detection with applications in virtual bone machining

BACKGROUND AND OBJECTIVES

Virtual reality surgery simulators have been proved effective for training in several surgical disciplines. Nevertheless, this technology is presently underutilized in orthopaedics, especially for bone machining procedures, due to the limited realism in haptic simulation of bone interactions. Collision detection is an integral part of surgery simulators and its accuracy and computational efficiency play a determinant role on the fidelity of simulations. To address this, the primary objective of this study was to develop a new algorithm that enables faster and more accurate collision detection within 1 ms (required for stable haptic rendering) in order to facilitate the improvement of the realism of virtual bone machining procedures.

METHODS

The core of the developed algorithm is constituted by voxmap point shell method according to which tool and osseous tissue geometries were sampled by points and voxels, respectively. The algorithm projects tool sampling points into the voxmap coordinates and compute an intersection condition for each point-voxel pair. This step is massively parallelized using Graphical Processing Units and it is further accelerated by an early culling of the unnecessary threads as instructed by the rapid estimation of the possible intersection volume. A contiguous array was used for implicit definition of voxmap in order to guarantee a fast access to voxels and thereby enable efficient material removal. A sparse representation of tool points was employed for efficient memory reductions. The effectiveness of the algorithm was evaluated at various bone sampling resolutions and was compared with prior relevant implementations.

RESULTS

The results obtained with an average hardware configuration have indicated that the developed algorithm is capable to reliably maintain < 1 ms running time in severe tool-bone collisions, both sampled at 1024³ resolutions. The results also showed the algorithm running time has a low sensitivity to bone sampling resolution. The comparisons performed suggested that the proposed approach is significantly faster than comparable methods while relying on lower or similar memory requirements.

CONCLUSIONS

The algorithm proposed through this study enables a higher numerical efficiency and is capable to significantly enlarge the maximum resolution that can be used by high fidelity/high realism haptic simulators targeting surgical orthopaedic procedures.

Using rapid cycle deliberate practice to improve primary and secondary survey in pediatric trauma

BACKGROUND

Optimal performance of the primary and secondary survey is the foundation of Advance Trauma Life Support care. Despite its importance, not all primary surveys completed at level 1 pediatric trauma centers are performed according to established guidelines (Gala et al., Pediatr Emerg Care 32:756-762, 2016, Carter et al., Resuscitation 84:66-71, 2013). We hypothesize that rapid cycle deliberate practice (RCDP) will improve surgical residents' confidence in performing the primary and secondary survey.

METHODS

We developed a curriculum to teach surgical interns the principles of performing the primary and secondary survey using RCDP. Surveys distributed after each session assessed the impact of the curriculum on learner confidence and perception that this curriculum would benefit patient care. Questions were scored on a 5-point Likert scale. Sixteen surgical interns participated during intern orientation and 100% of the participants completed the post curriculum survey.

RESULTS

Thirteen (81%) of participants agreed or strongly agreed that the simulation would impact future performance in the pediatric trauma bay. The curriculum also significantly improved the confidence of our learners to perform trauma surveys (p < 0.001).

CONCLUSION

This curriculum improves the confidence of junior surgical residents in learning the primary and secondary survey. Most learners enjoyed the session and felt that the curriculum would positively impact their performance.

The Development of an Online Standalone Cognitive Hazard Training for Laparoscopic Cholecystectomy: A Feasibility Study

INTRODUCTION

In the UK, surgical training is under pressure due to reductions in training time and training opportunities, which pose patient safety risks. Cognitive, nontechnical, training has been suggested as a possible solution inspired by the identified benefits in aviation industry. A recent review article highlighted the need for such training despite its high cost and the need for expert trainers.

AIM

This study aimed to design and test the feasibility of an online standalone module to address the current gap in cognitive surgical training.

METHOD

An online standalone, Cognitive Hazard Training module for laparoscopic cholecystectomy was created. It combined multiple choice questions, extended matching items, and single-line free text questions. It contained relevant sketch images and real life hazards video clips, highlighting potential mistakes to enhance: Safety knowledge, reduce bias, and improve self-limitation awareness. Two experts were invited to validate the prototype before testing its feasibility in one English Deanery training environment.

RESULTS

In total 93 candidates signed up to review the training. However only 47 (50%) later participated and 33 completed the Module. Those included 3 juniors, 20 higher trainees, and 10 consultants. Candidates' answers were quantitatively analysed. Qualitative feedback was also collected from 27 candidates, via semi-structured interviews. The overall feedback from the feasibility study was positive. Results supported this online resource value in enhancing knowledge and awareness. Interview data also suggested the module's potential to change trainees' practice by being more cautious and adhering to the safety steps of dissection.

DISCUSSION

This new training module overcomes some of the previously reported problems in surgical cognitive training. It is a stand-alone online resource with low running cost and does not require expert trainers. The feasibility study supported the aim to enhance hazard awareness and create an attitude shift towards adherence to safety steps during the procedure.

Endosuture trainer box simulator as a tool for training and teaching in bariatric laparoscopic surgery

Background

Video surgery requires acquisition of psychomotor skills that are different from those required for open surgery. The aim of this study was to assess the EndoSuture Trainer Box Simulator (ESTBS), a new bariatric laparoscopic skills simulator, as a tool for surgical education, comparing it with a standard laparoscopic trainer (SLT).

Methods

A randomized prospective crossover study was designed to compare ESTBS versus SLT as a tool for training bariatric laparoscopic skills. Participants were assigned to perform a task simulating Nissen fundoplication operation. All subjects evaluated the simulators concerning to their performance on simulating laparoscopic procedures by the use of a questionnaire comparing: triangulation, resistance and resilience, spatial perception (stereotaxy), ergonomics and positioning, inverted movements, visibility, design, technical and technological resources for training and education. The overall score was defined as the median value obtained. A total of 37 participants were enrolled in the study, including 29 experienced surgeons (78.37%) and 08 surgical residents (21.63%).

Results

A superior performance was observed with ESTBS as compared to SLT upon 7 of the 10 items evaluated in the questionnaire. Additionally, the overall score of ESTBS (median of 4, very good) was significantly higher (P < 0.0001) than that of SLT (median of 3, good). For the items, triangulation, resistance and resilience, ergonomics, design, training, technology and teaching, the evaluation for the ESTBS was very good and excellent, which was significantly higher than obtained by SLT. Also, ESTBS was cheaper.

Conclusions

The ESTBS was shown to present excellent technical and technological performances and appears to constitute a useful cost-effective promising instrument for teaching and training bariatric surgical laparoscopic skills.

3D-printed model improves clinical assessment of surgeons on anatomy

Performing surgical procedures often requires a surgeon to develop a skill to create 3-dimensional (3D) mental model on patient's anatomy. Question remains whether the touching on the 3D printed model can facilitate learning of patient anatomy than viewing the rendered virtual on-screen model. The printed and the virtual 3D model were developed from CT films taken from a 4-year-old girl, who had dysplasia of the hip in the left hip. Eleven subjects were called to report measures on six key anatomical features on the hips. The reporting time and the accuracy were compared between the two models, along with the gaze characteristics of subjects while inspecting the models. The variables were analysed using a 2 × 2 within subject ANOVA to examine the difference between viewing the models (on-screen vs. printed-out) and the side of the hip (right vs. left). Interacting with the printed 3D model required shorter times and yielded more accurate visual judgments than viewing the virtual models on most of the anatomical features. Subjects performed a fewer number of fixations but with a longer mean fixation duration when interacting the printed than inspecting the virtual on-screen 3D model. Results confirmed the value of the printed 3D model on improving the clinical judgement on patient anatomy. Confidence in collecting information from the physical world and the cross-model sensor integration may explain why participants performed better with the printed model compared to the virtual model.

Application of 3D-printing technology in the treatment of humeral intercondylar fractures

PURPOSE OF THE STUDY

This study was aimed to compare conventional surgery and surgery assisted by 3D-printing technology in the treatment of humeral intercondylar fractures. In addition, we also investigated the effect of 3D-printing technology on the communication between doctors and patients.

MATERIAL AND METHODS

A total of 91 patients with humeral intercondylar fracture were enrolled in the study from March 2013 to August 2015. They were divided into two groups: 43 cases of 3D-printing group, 48 cases of conventional group. The individual models were used to simulate the surgical procedures and carry out the surgery according to plan. Operation duration, blood loss volume, fluoroscopy times and time to fracture union were recorded. The final functional outcomes, including the motion of the elbow, MEPS and DASH were also evaluated. Besides, we made a simple questionnaire to verify the effectiveness of the 3D-printed model for both doctors and patients.

RESULTS

The operation duration, blood loss volume and fluoroscopy times for 3D-printing group was 76.6±7.9minutes, 231.1±18.1mL and 5.3±1.9 times, and for conventional group was 92.0±10.5minutes, 278.6±23.0mL and 8.7±2.7 times respectively. There was statistically significant difference between the conventional group and 3D-printing group (p<0.05). However, No significant difference was noted in the final functional outcomes between the two groups. Furthermore, the questionnaire showed that both doctors and patients exhibited high scores of overall satisfaction with the use of a 3D-printing model.

DISCUSSIONS

This study suggested the clinical feasibility of 3D-printing technology in treatment of humeral intercondylar fractures.

LEVEL OF EVIDENCE

Level II prospective randomized study.

Partial hepatectomy hemodynamics changes: Experimental data explained by closed-loop lumped modeling

The liver function may be degraded after partial liver ablation surgery. Adverse liver hemodynamics have been shown to be associated to liver failure. The link between these hemodynamics changes and ablation size is however poorly understood. This article proposes to explain with a closed-loop lumped model the hemodynamics changes observed during twelve surgeries in pigs. The portal venous tree is modeled with a pressure-dependent variable resistor. The variables measured, before liver ablation, are used to tune the model parameters. Then, the liver partial ablation is simulated with the model and the simulated pressures and flows are compared with post-operative measurements. Fluid infusion and blood losses occur during the surgery. The closed-loop model presented accounts for these blood volume changes. Moreover, the impact of blood volume changes and the liver lobe mass estimations on the simulated variables is studied. The typical increase of portal pressure, increase of liver pressure loss, slight decrease of portal flow and major decrease in arterial flow are quantitatively captured by the model for a 75% hepatectomy. It appears that the 75% decrease in hepatic arterial flow can be explained by the resistance increase induced by the surgery, and that no hepatic arterial buffer response (HABR) mechanism is needed to account for this change. The different post-operative states, observed in experiments, are reproduced with the proposed model. Thus, an explanation for inter-subjects post-operative variability is proposed. The presented framework can easily be adapted to other species circulations and to different pathologies for clinical hepatic applications.