Graphic and haptic modelling of the oesophagus for VR-based medical simulation

Characterization of the layer, direction and time-dependent mechanical behaviour of the human oesophagus and the effects of formalin preservation

The mechanical characterization of the oesophagus is essential for applications such as medical device design, surgical simulations and tissue engineering, as well as for investigating the organ's pathophysiology. However, the material response of the oesophagus has not been established ex vivo in regard to the more complex aspects of its mechanical behaviour using fresh, human tissue: as of yet, in the literature, only the hyperelastic response of the intact wall has been studied. Therefore, in this study, the layer-dependent, anisotropic, visco-hyperelastic behaviour of the human oesophagus was investigated through various mechanical tests. For this, cyclic tests, with increasing stretch levels, were conducted on the layers of the human oesophagus in the longitudinal and circumferential directions and at two different strain rates. Additionally, stress-relaxation tests on the oesophageal layers were carried out in both directions. Overall, the results show discrete properties in each layer and direction, highlighting the importance of treating the oesophagus as a multi-layered composite material with direction-dependent behaviour. Previously, the authors conducted layer-dependent cyclic experimentation on formalin-embalmed human oesophagi. A comparison between the fresh and embalmed tissue response was carried out and revealed surprising similarities in terms of anisotropy, strain-rate dependency, stress-softening and hysteresis, with the main difference between the two preservation states being the magnitude of these properties. As formalin fixation is known to notably affect the formation of cross-links between the collagen of biological materials, the differences may reveal the influence of cross-links on the mechanical behaviour of soft tissues.

Surgical instrument detection and tracking technologies: Automating dataset labeling for surgical skill assessment

Surgical skills can be improved by continuous surgical training and feedback, thus reducing adverse outcomes while performing an intervention. With the advent of new technologies, researchers now have the tools to analyze surgical instrument motion to differentiate surgeons’ levels of technical skill. Surgical skills assessment is time-consuming and prone to subjective interpretation. The surgical instrument detection and tracking algorithm analyzes the image captured by the surgical robotic endoscope and extracts the movement and orientation information of a surgical instrument to provide surgical navigation. This information can be used to label raw surgical video datasets that are used to form an action space for surgical skill analysis. Instrument detection and tracking is a challenging problem in MIS, including robot-assisted surgeries, but vision-based approaches provide promising solutions with minimal hardware integration requirements. This study offers an overview of the developments of assessment systems for surgical intervention analysis. The purpose of this study is to identify the research gap and make a leap in developing technology to automate the incorporation of new surgical skills. A prime factor in automating the learning is to create datasets with minimal manual intervention from raw surgical videos. This review encapsulates the current trends in artificial intelligence (AI) based visual detection and tracking technologies for surgical instruments and their application for surgical skill assessment.

Cutting Simulation in Unity 3D Using Position Based Dynamics with Various Refinement Levels

Augmented and Virtual Reality-based surgical simulations have become some of the fastest-developing areas, due to the recent technological advances and changes, in surgical education. Cutting simulation is a crucial part of the virtual surgery simulation in which an incision operation is performed. It is a complex process that includes three main tasks: soft body simulation, collision detection and handling, and topological deformation of the soft body. In this paper, considering the content developer’s convenience, the deformable object simulation, using position-based dynamics (PBD), was applied in the Unity 3D environment. The proposed algorithm for fast collision detection and handling between the cutting tool and the deformable object uses a sweep surface. In case of incision, the algorithm updates the mesh topology by deleting intersected triangles, re-triangulation, and refinement. In the refinement part, the boundary edges threshold was used to match the resolution of new triangles to the existing mesh triangles. Additionally, current research is focused on triangle surface meshes, which help to reduce the computational costs of the topology modifications. It was found that the algorithm can successfully handle arbitrary cuts, keeping the framerate within interactive and, in some cases, in the real-time.

The current and possible future role of 3D modelling within oesophagogastric surgery: a scoping review

BACKGROUND

3D reconstruction technology could revolutionise medicine. Within surgery, 3D reconstruction has a growing role in operative planning and procedures, surgical education and training as well as patient engagement. Whilst virtual and 3D printed models are already used in many surgical specialities, oesophagogastric surgery has been slow in their adoption. Therefore, the authors undertook a scoping review to clarify the current and future roles of 3D modelling in oesophagogastric surgery, highlighting gaps in the literature and implications for future research.

METHODS

A scoping review protocol was developed using a comprehensive search strategy based on internationally accepted guidelines and tailored for key databases (MEDLINE, Embase, Elsevier Scopus and ISI Web of Science). This is available through the Open Science Framework (osf.io/ta789) and was published in a peer-reviewed journal. Included studies underwent screening and full text review before inclusion. A thematic analysis was performed using pre-determined overarching themes: (i) surgical training and education, (ii) patient education and engagement, and (iii) operative planning and surgical practice. Where applicable, subthemes were generated.

RESULTS

A total of 56 papers were included. Most research was low-grade with 88% (n = 49) of publications at or below level III evidence. No randomised control trials or systematic reviews were found. Most literature (86%, n = 48) explored 3D reconstruction within operative planning. These were divided into subthemes of pre-operative (77%, n = 43) and intra-operative guidance (9%, n = 5). Few papers reported on surgical training and education (14%, n = 8), and were evenly subcategorised into virtual reality simulation (7%, n = 4) and anatomical teaching (7%, n = 4). No studies utilising 3D modelling for patient engagement and education were found.

CONCLUSION

The use of 3D reconstruction is in its infancy in oesophagogastric surgery. The quality of evidence is low and key themes, such as patient engagement and education, remain unexplored. Without high quality research evaluating the application and benefits of 3D modelling, oesophagogastric surgery may be left behind.

Electrically Elicited Force Response Characteristics of Forearm Extensor Muscles for Electrical Muscle Stimulation-Based Haptic Rendering

A haptic interface based on electrical muscle stimulation (EMS) has huge potential in terms of usability and applicability compared with conventional haptic interfaces. This study analyzed the force response characteristics of forearm extensor muscles for EMS-based haptic rendering. We introduced a simplified mathematical model of the force response, which has been developed in the field of rehabilitation, and experimentally validated its feasibility for haptic applications. Two important features of the force response, namely the peak force and response time, with respect to the frequency and amplitude of the electrical stimulation were identified by investigating the experimental force response of the forearm extensor muscles. An exponential function was proposed to estimate the peak force with respect to the frequency and amplitude, and it was verified by comparing with the measured peak force. The response time characteristics were also examined with respect to the frequency and amplitude. A frequency-dependent tendency, i.e., an increase in response time with increasing frequency, was observed, whereas there was no correlation with the amplitude. The analysis of the force response characteristics with the application of the proposed force response model may help enhance the fidelity of EMS-based haptic rendering.

Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature

With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.

Graphic and haptic simulation for transvaginal cholecystectomy training in NOTES

BACKGROUND

Natural Orifice Transluminal Endoscopic Surgery (NOTES) provides an emerging surgical technique which usually needs a long learning curve for surgeons. Virtual reality (VR) medical simulators with vision and haptic feedback can usually offer an efficient and cost-effective alternative without risk to the traditional training approaches. Under this motivation, we developed the first virtual reality simulator for transvaginal cholecystectomy in NOTES (VTEST™).

METHODS

This VR-based surgical simulator aims to simulate the hybrid NOTES of cholecystectomy. We use a 6DOF haptic device and a tracking sensor to construct the core hardware component of simulator. For software, an innovative approach based on the inner-spheres is presented to deform the organs in real time. To handle the frequent collision between soft tissue and surgical instruments, an adaptive collision detection method based on GPU is designed and implemented. To give a realistic visual performance of gallbladder fat tissue removal by cautery hook, a multi-layer hexahedral model is presented to simulate the electric dissection of fat tissue.

RESULTS

From the experimental results, trainees can operate in real time with high degree of stability and fidelity. A preliminary study was also performed to evaluate the realism and the usefulness of this hybrid NOTES simulator.

CONCLUSIONS

This prototyped simulation system has been verified by surgeons through a pilot study. Some items of its visual performance and the utility were rated fairly high by the participants during testing. It exhibits the potential to improve the surgical skills of trainee and effectively shorten their learning curve.

Haptic Augmentation Utilizing the Reaction Force of a Base Object

A haptic device is one of the interfaces promising as a human-computer interaction tool that provides users with information in a virtual reality environment. The proposed haptic augmentation presents the force response of an object by combining the force generated from a haptic device against the force response generated from the base object, which has material properties similar to those of the target object. In this paper, the concept of a haptic augmentation technique is described and the prototype of a haptic augmentation system is developed. Frequency characteristics and experiments by human participants show that the proposed method has better performance than a traditional device-only method.

Simple display system of mechanical properties of cells and their dispersion

The mechanical properties of cells are unique indicators of their states and functions. Though, it is difficult to recognize the degrees of mechanical properties, due to small size of the cell and broad distribution of the mechanical properties. Here, we developed a simple virtual reality system for presenting the mechanical properties of cells and their dispersion using a haptic device and a PC. This system simulates atomic force microscopy (AFM) nanoindentation experiments for floating cells in virtual environments. An operator can virtually position the AFM spherical probe over a round cell with the haptic handle on the PC monitor and feel the force interaction. The Young's modulus of mesenchymal stem cells and HEK293 cells in the floating state was measured by AFM. The distribution of the Young's modulus of these cells was broad, and the distribution complied with a log-normal pattern. To represent the mechanical properties together with the cell variance, we used log-normal distribution-dependent random number determined by the mode and variance values of the Young's modulus of these cells. The represented Young's modulus was determined for each touching event of the probe surface and the cell object, and the haptic device-generating force was calculated using a Hertz model corresponding to the indentation depth and the fixed Young's modulus value. Using this system, we can feel the mechanical properties and their dispersion in each cell type in real time. This system will help us not only recognize the degrees of mechanical properties of diverse cells but also share them with others.

Graphic and haptic simulation system for virtual laparoscopic rectum surgery

BACKGROUND

Medical simulators with vision and haptic feedback techniques offer a cost-effective and efficient alternative to the traditional medical trainings. They have been used to train doctors in many specialties of medicine, allowing tasks to be practised in a safe and repetitive manner. This paper describes a virtual-reality (VR) system which will help to influence surgeons' learning curves in the technically challenging field of laparoscopic surgery of the rectum.

METHODS

Data from MRI of the rectum and real operation videos are used to construct the virtual models. A haptic force filter based on radial basis functions is designed to offer realistic and smooth force feedback. To handle collision detection efficiently, a hybrid model is presented to compute the deformation of intestines. Finally, a real-time cutting technique based on mesh is employed to represent the incision operation.

RESULTS

Despite numerous research efforts, fast and realistic solutions of soft tissues with large deformation, such as intestines, prove extremely challenging. This paper introduces our latest contribution to this endeavour. With this system, the user can haptically operate with the virtual rectum and simultaneously watch the soft tissue deformation.

CONCLUSIONS

Our system has been tested by colorectal surgeons who believe that the simulated tactile and visual feedbacks are realistic. It could replace the traditional training process and effectively transfer surgical skills to novices.