Design and development of a personalized virtual reality-based training system for vascular intervention surgery

BACKGROUND AND OBJECTIVES

Virtual training has emerged as an exceptionally effective approach for training healthcare practitioners in the field of vascular intervention surgery. By providing a simulated environment and blood vessel model that enables repeated practice, virtual training facilitates the acquisition of surgical skills in a safe and efficient manner for trainees. However, the current state of research in this area is characterized by limitations in the fidelity of blood vessel and guidewire models, which restricts the effectiveness of training. Additionally, existing approaches lack the necessary real-time responsiveness and precision, while the blood vessel models suffer from incompleteness and a lack of scientific rigor.

METHODS

To address these challenges, this paper integrates position-based dynamics (PBD) and its extensions, shape matching, and Cosserat elastic rods. By combining these approaches within a unified particle framework, accurate and realistic deformation simulation of personalized blood vessel and guidewire models is achieved, thereby enhancing the training experience. Furthermore, a multi-level progressive continuous collision detection method, leveraging spatial hashing, is proposed to improve the accuracy and efficiency of collision detection.

RESULTS

Our proposed blood vessel model demonstrated acceptable performance with the reduced deformation simulation response times of 7 ms, improving the real-time capability at least of 43.75 %. Experimental validation confirmed that the guidewire model proposed in this paper can dynamically adjust the density of its elastic rods to alter the degree of bending and torsion. It also exhibited a deformation process comparable to that of real guidewires, with an average response time of 6 ms. In the interaction of blood vessel and guidewire models, the simulator blood vessel model used for coronary vascular intervention training exhibited an average response time of 15.42 ms, with a frame rate of approximately 64 FPS.

CONCLUSIONS

The method presented in this paper achieves deformation simulation of both vascular and guidewire models, demonstrating sufficient real-time performance and accuracy. The interaction efficiency between vascular and guidewire models is enhanced through the unified simulation framework and collision detection. Furthermore, it can be integrated with virtual training scenarios within the system, making it suitable for developing more advanced vascular interventional surgery training systems.

Fast and precise collision detection for detailed and complex physiological structures

BACKGROUND AND OBJECTIVES

Virtual reality has been proved indispensable in computer-assisted surgery, especially for surgical planning, and simulation systems. Collision detection is an essential part of surgery simulators and its accuracy and computational efficiency play a decisive role in the fidelity of simulations. Nevertheless, current collision detection methods in surgical simulation and planning struggle to meet precise requirements, especially for detailed and complex physiological structures. To address this, the primary objective of this study was to develop a new algorithm that enables fast and precise collision detection to facilitate the improvement of the realism of virtual reality surgical procedures.

METHODS

The method consists of two main parts, bounding spheres formation and two-level collision detection. A specified surface subdivision method is devised to reduce the radius of basic bounding spheres formed by circumcenters of underlying triangles. The spheres are then clustered and adjusted to obtain a compact personalized hierarchy whose position is updated in real time during surgical simulation, followed by two-level collision detection. Triangular facets with collision potential through interaction between hierarchies and then accurate results are obtained by means of precise detection phase. The effectiveness of the algorithm was evaluated in various models and surgical scenarios and was compared with prior relevant implementations.

RESULTS

Results on multiple models demonstrated that the method can generate a personalized hierarchy with fewer and smaller bounding spheres for tight wrapping. Simulation experiments proved that the proposed approach is significantly superior to comparable methods under the premise of error-free detection, even for severe model-model collision.

CONCLUSIONS

The algorithm proposed through this study enables higher numerical efficiency and detection accuracy, which is capable of significantly enlarging the fidelity/realism of haptic simulators and surgical planning methods.

Bone collision detection method for robot assisted fracture reduction based on vibration excitation

BACKGROUND AND OBJECTIVE

In the process of robotic fracture reduction, there is a risk of unintended collision of broken bones, which is not conducive to ensuring the safety of the reduction system. In order to solve this problem, this paper proposed a vibration-based collision detection method for fracture reduction process.

METHODS

Based on the two degree-of-freedom vibration response model, the factors affecting the respond of the vibration, including the excitation voltage, the clamping length at the proximal and distal ends, the mass and tensile force of the soft tissue, were obtained. The effects of these factors on the vibration transfer performance of broken bones and soft tissue were investigated by single factor experiments.

RESULTS

The results showed that, in terms of peak value, the increase of excitation voltage would make the vibration amplitude increase linearly, and the increase of soft tissue mass and tension increased the vibration transmission capacity of soft tissue in the frequency range of 500-1000 Hz. In terms of peak frequency, the clamping length at the distal end had the greatest influence, which reached 74 Hz, followed by 45 Hz at the proximal end. While the influence of other factors was little. According to single factor experiments, the excitation frequency in the verification experiments was determined as 677 Hz. Under the vibration interference with the acceleration amplitude of 1.2 G, this method achieved correct detection.

CONCLUSION

This research developed a broken bone collision detection method based on vibration excitation. The method can correctly detect the collision of broken bones with strong anti-interference ability. It is of great significance to improve the safety of fracture reduction process.

A mathematical framework for the analysis and comparison of contact detection methods for ellipses and ellipsoids

The purpose of this research is to provide a framework for the analysis and comparison of contact detection algorithms for pairs of ellipses and ellipsoids. This work focuses primarily on the category of algorithms that are the most computationally efficient and can produce estimates of the separation and the penetration distance between ellipses and ellipsoids. Specifically, only analytic representations of the ellipses and ellipsoids are considered and contact detection for moving pairs of ellipsoids is not treated. The first contribution is a mathematical framework for the study of these algorithms, most notably with existence and uniqueness proofs for classes of contact detection algorithms, formal descriptions of the asymptotics of pairs of ellipses in close contact (or overlap), and a global analysis of constraints on the normals. The framework highlights the key role played by the different definitions of contact found in the literature, independent of the numerical strategies deployed to estimate the separation/penetration distance. Specifically, it is shown that all the studied algorithms can be expressed as minimization problems, with or without non-binding constraints on the normal(s) at the contact point(s), and that the constraints can be used to identify the global minima among the critical points in the minimization problem. Another contribution of this research, based on the mathematical framework introduced, is a better classification of the known algorithms. These algorithms are compared on established test problems, and their strengths and weaknesses are highlighted and explained in terms of their classification. Furthermore, this research provides comparisons in speed and stability between the most efficient algorithms in each category over a large sample size of test problems. Among the other contributions, this research describes inexpensive but effective initial estimates of the contact to be used in iterative algorithms. Finally, the usefulness of the new framework is illustrated with the introduction of a fast algorithm combining some new and old ideas.

Control of aperture closure during reach-to-grasp movements in immersive haptic-free virtual reality

Virtual reality (VR) has garnered much interest as a training environment for motor skill acquisition, including for neurological rehabilitation of upper extremities. While the focus has been on gross upper limb motion, VR applications that involve reaching for, and interacting with, virtual objects are growing. The absence of true haptics in VR when it comes to hand-object interactions raises a fundamentally important question: can haptic-free immersive virtual environments (hf-VEs) support naturalistic coordination of reach-to-grasp movements? This issue has been grossly understudied, and yet is of significant importance in the development and application of VR across a number of sectors. In a previous study (Furmanek et al., J Neuroeng Rehabil 16:78, 2019), we reported that reach-to-grasp movements are similarly coordinated in both the physical environment (PE) and hf-VE. The most noteworthy difference was that the closure phase-which begins at maximum aperture and lasts through the end of the movement-was longer in hf-VE than in PE, suggesting that different control laws might govern the initiation of closure between the two environments. To do so, we reanalyzed data from Furmanek et al. (J Neuroeng Rehabil 16:78, 2019), in which the participants reached to grasp three differently sized physical objects, and matching 3D virtual object renderings, placed at three different locations. Our analysis revealed two key findings pertaining to the initiation of closure in PE and hf-VE. First, the respective control laws governing the initiation of aperture closure in PE and hf-VE both included state estimates of transport velocity and acceleration, supporting a general unified control policy for implementing reach-to-grasp across physical and virtual environments. Second, the aperture was less informative to the control law in hf-VE. We suggest that the latter was likely because transport velocity at closure onset and aperture at closure onset were less independent in hf-VE than in PE, ultimately resulting in an aperture at closure onset having a weaker influence on the initiation of closure. In this way, the excess time and muscular effort needed to actively bring the fingers to a stop at the interface of a virtual object was factored into the control law governing the initiation of closure in hf-VE. Critically, this control law remained applicable, albeit with different weights in hf-VE, despite the absence of terminal haptic feedback and potential perceptual differences.

Deformation modeling based on mechanical properties of liver tissue for virtuanormal vectors of trianglesl surgical simulation

PURPOSE

In this paper, a method for rapidly constructing a virtual surgical simulation system is proposed. A deformation model based on the mechanical properties of the liver and a rapid collision detection between the surgical micro-instruments and the liver tissue are included in this method. The purpose of this work is to improve the accuracy and real time of particle model deformation interaction in virtual surgery system.

METHODS

Firstly, a finite element model is established based on the constitutive model parameters of liver tissue. According to the simulation results, a mathematical model of node displacement is established. Secondly, the virtual liver is established based on the fast model reconstruction method, and the virtual manipulator is controlled by Geomagic Touch manipulator. Based on the hybrid bounding box, a rapid collision detection process between the instrument and liver is realized and the proposed deformation method is used to simulate the deformation of liver tissue.

RESULTS

The simulation and experiment results show that the proposed deformation model can achieve high deformation interaction accuracy. The collision detection algorithm based on the hybrid bounding boxes can realize the collision between the liver and the instrument, and the established virtual surgical simulation system can simulate the liver tissue deformation in the case of small loading displacement.

CONCLUSIONS

The effectiveness of the collision detection algorithm and deformation model was verified by an established virtual surgery simulation system. The proposed rapid construction method of virtual surgical simulation is feasible.

An optimal 𝒪(N) scheme for simulations of colliding, particle-laden flows on unstructured grids

The cost of tracking Lagrangian particles in a domain discretized on an unstructured grid can become prohibitively expensive as the number of particles or elements grows. A major part of the cost in these calculations is spent on locating the element that hosts a particle and detecting binary collisions, with the latter traditionally requiring 𝒪 ( N 2 ) operations, N being the number of particles. This paper introduces an optimal search box strategy to significantly reduce the cost of these two operations, ensuring a nearly 𝒪 ( N ) scaling of the cost of collision detection for large-scale simulations. The particle localization strategy is constructed by obtaining an a priori estimate for the optimal number of search boxes as a function of the number of elements, particles, and time steps. The introduced method is generic, as it must be tuned only once for a given implementation and element type. The optimal number of search boxes for collision detection, although complex in form, can be reasonably approximated as the number of particles. The optimality of our method is shown using three drastically varying geometries.

Parallelized collision detection with applications in virtual bone machining

BACKGROUND AND OBJECTIVES

Virtual reality surgery simulators have been proved effective for training in several surgical disciplines. Nevertheless, this technology is presently underutilized in orthopaedics, especially for bone machining procedures, due to the limited realism in haptic simulation of bone interactions. Collision detection is an integral part of surgery simulators and its accuracy and computational efficiency play a determinant role on the fidelity of simulations. To address this, the primary objective of this study was to develop a new algorithm that enables faster and more accurate collision detection within 1 ms (required for stable haptic rendering) in order to facilitate the improvement of the realism of virtual bone machining procedures.

METHODS

The core of the developed algorithm is constituted by voxmap point shell method according to which tool and osseous tissue geometries were sampled by points and voxels, respectively. The algorithm projects tool sampling points into the voxmap coordinates and compute an intersection condition for each point-voxel pair. This step is massively parallelized using Graphical Processing Units and it is further accelerated by an early culling of the unnecessary threads as instructed by the rapid estimation of the possible intersection volume. A contiguous array was used for implicit definition of voxmap in order to guarantee a fast access to voxels and thereby enable efficient material removal. A sparse representation of tool points was employed for efficient memory reductions. The effectiveness of the algorithm was evaluated at various bone sampling resolutions and was compared with prior relevant implementations.

RESULTS

The results obtained with an average hardware configuration have indicated that the developed algorithm is capable to reliably maintain < 1 ms running time in severe tool-bone collisions, both sampled at 1024³ resolutions. The results also showed the algorithm running time has a low sensitivity to bone sampling resolution. The comparisons performed suggested that the proposed approach is significantly faster than comparable methods while relying on lower or similar memory requirements.

CONCLUSIONS

The algorithm proposed through this study enables a higher numerical efficiency and is capable to significantly enlarge the maximum resolution that can be used by high fidelity/high realism haptic simulators targeting surgical orthopaedic procedures.

Coordination of reach-to-grasp in physical and haptic-free virtual environments

Background

Virtual reality (VR) offers unprecedented opportunity as a scientific tool to study visuomotor interactions, training, and rehabilitation applications. However, it remains unclear if haptic-free hand-object interactions in a virtual environment (VE) may differ from those performed in the physical environment (PE). We therefore sought to establish if the coordination structure between the transport and grasp components remain similar whether a reach-to-grasp movement is performed in PE and VE.

Method

Reach-to-grasp kinematics were examined in 13 healthy right-handed young adults. Subjects were instructed to reach-to-grasp-to-lift three differently sized rectangular objects located at three different distances from the starting position. Object size and location were matched between the two environments. Contact with the virtual objects was based on a custom collision detection algorithm. Differences between the environments were evaluated by comparing movement kinematics of the transport and grasp components.

Results

Correlation coefficients, and the slope of the regression lines, between the reach and grasp components were similar for the two environments. Likewise, the kinematic profiles of the transport velocity and grasp aperture were strongly correlated across the two environments. A rmANOVA further identified some similarities and differences in the movement kinematics between the two environments - most prominently that the closure phase of reach-to-grasp movement was prolonged when movements were performed in VE.

Conclusions

Reach-to-grasp movement patterns performed in a VE showed both similarities and specific differences compared to those performed in PE. Additionally, we demonstrate a novel approach for parsing the reach-to-grasp movement into three phases- initiation, shaping, closure- based on established kinematic variables, and demonstrate that the differences in performance between the environments are attributed to the closure phase. We discuss this in the context of how collision detection parameters may modify hand-object interactions in VE. Our study shows that haptic-free VE may be a useful platform to study reach-to-grasp movements, with potential implications for haptic-free VR in neurorehabilitation.

Electronic supplementary material

The online version of this article (10.1186/s12984-019-0525-9) contains supplementary material, which is available to authorized users.

Simulating unmanned aerial vehicle flight control and collision detection

An unmanned aerial vehicle (UAV) is a small, fast aircraft with many useful features. It is widely used in military reconnaissance, aerial photography, searches, and other fields; it also has very good practical-application and development prospects. Since the UAV’s flight orientation is easily changeable, its orientation and flight path are difficult to control, leading to its high damage rate. Therefore, UAV flight-control technology has become the focus of attention. This study focuses on simulating a UAV’s flight and orientation control, and detecting collisions between a UAV and objects in a complex virtual environment. The proportional-integral-derivative control algorithm is used to control the orientation and position of the UAV in a virtual environment. A version of the bounding-box method that combines a grid with a k-dimensional tree is adopted in this paper, to improve the system performance and accelerate the collision-detection process. This provides a practical method for future studies on UAV flight position and orientation control, collision detection, etc.

Three-dimensional shape and velocity changes affect responses of a locust visual interneuron to approaching objects

Adaptive collision avoidance behaviours require accurate detection of complex spatiotemporal properties of an object approaching in an animal's natural, three-dimensional environment. Within the locust, the lobula giant movement detector and its postsynaptic partner, the descending contralateral movement detector (DCMD), respond robustly to images that emulate an approaching two-dimensional object and exhibit firing rate modulation correlated with changes in object trajectory. It is not known how this pathway responds to visual expansion of a three-dimensional object or an approaching object that changes velocity, both of which represent natural stimuli. We compared DCMD responses with images that emulate the approach of a sphere with those elicited by a two-dimensional disc. A sphere evoked later peak firing and decreased sensitivity to the ratio of the half size of the object to the approach velocity, resulting in an increased threshold subtense angle required to generate peak firing. We also presented locusts with an approaching sphere that decreased or increased in velocity. A velocity decrease resulted in transition-associated peak firing followed by a firing rate increase that resembled the response to a constant, slower velocity. A velocity increase resulted in an earlier increase in the firing rate that was more pronounced with an earlier transition. These results further demonstrate that this pathway can provide motor circuits for behaviour with salient information about complex stimulus dynamics.

Three-dimensional shape and velocity changes affect responses of a locust visual interneuron to approaching objects

Adaptive collision avoidance behaviours require accurate detection of complex spatiotemporal properties of an object approaching in an animal's natural, three-dimensional environment. Within the locust, the lobula giant movement detector and its postsynaptic partner, the descending contralateral movement detector (DCMD), respond robustly to images that emulate an approaching two-dimensional object and exhibit firing rate modulation correlated with changes in object trajectory. It is not known how this pathway responds to visual expansion of a three-dimensional object or an approaching object that changes velocity, both of which represent natural stimuli. We compared DCMD responses with images that emulate the approach of a sphere with those elicited by a two-dimensional disc. A sphere evoked later peak firing and decreased sensitivity to the ratio of the half size of the object to the approach velocity, resulting in an increased threshold subtense angle required to generate peak firing. We also presented locusts with an approaching sphere that decreased or increased in velocity. A velocity decrease resulted in transition-associated peak firing followed by a firing rate increase that resembled the response to a constant, slower velocity. A velocity increase resulted in an earlier increase in the firing rate that was more pronounced with an earlier transition. These results further demonstrate that this pathway can provide motor circuits for behaviour with salient information about complex stimulus dynamics.

Virtual interactive suturing for the Fundamentals of Laparoscopic Surgery (FLS)

BACKGROUND

Suturing with intracorporeal knot-tying is one of the five tasks of the Fundamentals of Laparoscopic Surgery (FLS), which is a pre-requisite for board certification in general surgery. This task involves placing a short suture through two marks in a penrose drain and then tying a double-throw knot followed by two single-throw knots using two needle graspers operated by both hands. A virtual basic laparoscopic skill trainer (VBLaST©) is being developed to represent the virtual versions of the FLS tasks, including automated, real time performance measurement and feedback. In this paper, we present the development of a VBLaST suturing simulator (VBLaST-SS©). Developing such a simulator involves solving multiple challenges associated with fast collision detection, response and force feedback.

METHODS

In this paper, we present a novel projection-intersection based knot detection method, which can identify the validity of different types of knots at haptic update rates. A simple and robust edge-edge based collision detection algorithm is introduced to support interactive knot tying and needle insertion operations. A bimanual hardware interface integrates actual surgical instruments with haptic devices enabling not only interactive rendering of force feedback but also realistic sensation of needle grasping, which realizes an immersive surgical suturing environment.

RESULTS

Experiments on performing the FLS intracorporeal suturing task show that the simulator is able to run on a standard personal computer at interactive rates.

CONCLUSIONS

VBLaST-SS© is a computer-based interactive virtual simulation system for FLS intracorporeal knot-tying suturing task that can provide real-time objective assessment for the user's performance.