A Smart Haptic Hand-Held Device for Neurosurgical Microdissection

A Sensorised Surgical Glove to Analyze Forces During Neurosurgery

BACKGROUND

Measuring intraoperative forces in real time can provide feedback mechanisms to improve patient safety and surgical training. Previous force monitoring has been achieved through the development of specialized and adapted instruments or use designs that are incompatible with neurosurgical workflow.

OBJECTIVE

To design a universal sensorised surgical glove to detect intraoperative forces, applicable to any surgical procedure, and any surgical instrument in either hand.

METHODS

We created a sensorised surgical glove that was calibrated across 0 to 10 N. A laboratory experiment demonstrated that the sensorised glove was able to determine instrument-tissue forces. Six expert and 6 novice neurosurgeons completed a validated grape dissection task 20 times consecutively wearing the sensorised glove. The primary outcome was median and maximum force (N).

RESULTS

The sensorised glove was able to determine instrument-tissue forces reliably. The average force applied by experts (2.14 N) was significantly lower than the average force exerted by novices (7.15 N) ( P = .002). The maximum force applied by experts (6.32 N) was also significantly lower than the maximum force exerted by novices (9.80 N) ( P = .004). The sensorised surgical glove's introduction to operative workflow was feasible and did not impede on task performance.

CONCLUSION

We demonstrate a novel and scalable technique to detect forces during neurosurgery. Force analysis can provide real-time data to optimize intraoperative tissue forces, reduce the risk of tissue injury, and provide objective metrics for training and assessment.

Tool-Tissue Forces in Hemangioblastoma Surgery

OBJECTIVE

Surgical resection of intracranial hemangioblastoma poses technical challenges that may be difficult to impart to trainees. Here, we introduce knowledge of tool-tissue forces in Newton (N), observed during hemangioblastoma surgery.

METHODS

Seven surgeons (2 groups: trainees and mentor), with mentor (n = 1) and trainees (n = 6, PGY 1-6 including clinical fellowship), participated in 6 intracranial hemangioblastoma surgeries. Using sensorized bipolar forceps, we evaluated tool-tissue force profiles of 5 predetermined surgical tasks: 1) dissection, 2) coagulation, 3) retracting, 4) pulling, and 5) manipulating. Force profile for each trial included force duration, average, maximum, minimum, range, standard deviation (SD), and correlation coefficient. Force errors including unsuccessful trial bleeding or incomplete were compared between surgeons and with successful trials.

RESULTS

Force data from 718 trials were collected. The mean (standard deviation) of force used in all surgical tasks and across all surgical levels was 0.20 ± 0.17 N. The forces exerted by trainee surgeons were significantly lower than those of the mentor (0.15 vs. 0.24; P < 0.0001). A total of 18 (4.5%) trials were unsuccessful, 4 of them being unsuccessful trial-bleeding and the rest, unsuccessful trial-incomplete. The force in unsuccessful trial-bleeding was higher than successful trials (0.3 [0.09] vs. 0.17 [0.11]; P = 0.0401). Toward the end of surgery, higher force was observed (0.17 vs. 0.20; P < 0.0001).

CONCLUSIONS

The quantification of tool-tissue forces during hemangioblastoma surgery with feedback to the surgeon, could well enhance surgical training and allow avoidance of bleeding associated with high force error.

An Atlas for the Inkjet Printing of Large-Area Tactile Sensors

This review aims to discuss the inkjet printing technique as a fabrication method for the development of large-area tactile sensors. The paper focuses on the manufacturing techniques and various system-level sensor design aspects related to the inkjet manufacturing processes. The goal is to assess how printed electronics simplify the fabrication process of tactile sensors with respect to conventional fabrication methods and how these contribute to overcoming the difficulties arising in the development of tactile sensors for real robot applications. To this aim, a comparative analysis among different inkjet printing technologies and processes is performed, including a quantitative analysis of the design parameters, such as the costs, processing times, sensor layout, and general system-level constraints. The goal of the survey is to provide a complete map of the state of the art of inkjet printing, focusing on the most effective topics for the implementation of large-area tactile sensors and a view of the most relevant open problems that should be addressed to improve the effectiveness of these processes.

Tool-tissue forces in surgery: A systematic review

Background

Excessive tool-tissue interaction forces often result in tissue damage and intraoperative complications, while insufficient forces prevent the completion of the task. This review sought to explore the tool-tissue interaction forces exerted by instruments during surgery across different specialities, tissues, manoeuvres and experience levels.

Materials & methods

A PRISMA-guided systematic review was carried out using Embase, Medline and Web of Science databases.

Results

Of 462 articles screened, 45 studies discussing surgical tool-tissue forces were included. The studies were categorized into 9 different specialities with the mean of average forces lowest for ophthalmology (0.04N) and highest for orthopaedic surgery (210N). Nervous tissue required the least amount of force to manipulate (mean of average: 0.4N), whilst connective tissue (including bone) required the most (mean of average: 45.8). For manoeuvres, drilling recorded the highest forces (mean of average: 14N), whilst sharp dissection recorded the lowest (mean of average: 0.03N). When comparing differences in the mean of average forces between groups, novices exerted 22.7% more force than experts, and presence of a feedback mechanism (e.g. audio) reduced exerted forces by 47.9%.

Conclusions

The measurement of tool-tissue forces is a novel but rapidly expanding field. The range of forces applied varies according to surgical speciality, tissue, manoeuvre, operator experience and feedback provided. Knowledge of the safe range of surgical forces will improve surgical safety whilst maintaining effectiveness. Measuring forces during surgery may provide an objective metric for training and assessment. Development of smart instruments, robotics and integrated feedback systems will facilitate this.

•

This review explores tool-tissue forces during surgery, a new and expanding field.

•

Forces were lowest in ophthalmology (0.04N) and highest in orthopaedics (210N).

•

Forces were lowest during sharp dissection (0.03N) and highest when drilling (14N).

•

Being an expert (vs. novice) and having feedback mechanisms (e.g. haptic) reduced exerted forces.

•

Development of force metrics will facilitate training, assessment & novel technology.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

A review of haptic feedback in tele-operated robotic surgery

During traditional surgery, the surgeons' hands are in direct contact with organs, and surgeons rely on the sense of touch to perform surgery. In teleoperated robotic systems, all physical connections between the surgeon and both the robot and patient, are absent. The surgeon must estimate the force exerted on organs, based only on visual deformation of tissues he is pulling, pushing, gripping, or suturing. It is hard to imagine how to operate with no haptic sensations, and it is surprising that commercially available robots didn't include until now any Haptic Feedback, despite reports about tissue injury, and inability to perform complex manipulation. The sense of touch must be created by stimuli sensed by the surgeon. Haptic sensors are required to collect and send haptic information, and display them on the operator's side, creating telepresence, known as transparency. Multiple ways have been developed to improve transparency through force feedback and tactile feedback. However, this interferes with the stability of the closed-loop controlling interactions between master, robot and remote environment. Cutaneous feedback is more stable and less transparent; force feedback is more transparent and less stable. Thus, multimodal platforms of haptic feedback would try to find the best trade-off between both modalities.

A hand-held device with 3-DOF haptic feedback mechanism for microsurgery

BACKGROUND

The hand-held devices have the advantages of being compact and easily integrated into surgical workflow especially for microsurgery. However, one of the technical challenges of hand-held device in microsurgery is the lack of force sensing and feedback.

METHODS

This paper presents a hand-held haptic device that converts imperceptible forces into human-perceptible tactile signals. It combines a force sensing tip based on fiber Bragg grating (FBG) sensors that can detect three-degrees-of-freedom (3-DOF) forces, and a haptic feedback mechanism to indicate the magnitude and direction of the forces.

RESULTS

Experimental results demonstrate the ability to measure transverse force at the level of millinewton. User trials have been performed to validate the performance of the haptic feedback. During the phantom experiment, the transverse force was reduced with the feedback mechanism.

CONCLUSIONS

The proposed device provides important insights into the design of hand-held device incorporating real-time force sensing and haptic feedback for microsurgery.

Nonparametric bootstrap technique for calibrating surgical SmartForceps: theory and application

Knowledge of forces, exerted on the brain tissue during the performance of neurosurgical tasks, is critical for quality assurance, case rehearsal, and training purposes. Quantifying the interaction forces has been made possible by developing SmartForceps, a bipolar forceps retrofitted by a set of strain gauges. The forces are estimated using voltages read from strain gauges. We therefore need to quantify the force-voltage relationship to estimate the interaction forces during microsurgery. This problem has been addressed in the literature by following the physical and deterministic properties of the force-sensing strain gauges without obtaining the precision associated with each estimate. In this paper, we employ a probabilistic methodology by using a nonparametric Bootstrap approach to obtain both point and interval estimates of the applied forces at the tool tips, while the precision associated with each estimate is provided. To show proof-of-concept, the Bootstrap technique is employed to estimate unknown forces, and construct necessary confidence intervals using observed voltages in data sets that are measured from the performance of surgical tasks on a cadaveric brain. Results indicate that the Bootstrap technique is capable of estimating tool-tissue interaction forces with acceptable level of accuracy compared to the linear regression technique under the normality assumption.

Motorized Micro-Forceps with Active Motion Guidance based on Common-Path SSOCT for Epiretinal Membranectomy

In this study, we built and tested a handheld motion-guided micro-forceps system using common-path swept source optical coherence tomography (CP-SSOCT) for highly accurate depth controlled epiretinal membranectomy. A touch sensor and two motors were used in the forceps design to minimize the inherent motion artifact while squeezing the tool handle to actuate the tool and grasp, and to independently control the depth of the tool-tip. A smart motion monitoring and a guiding algorithm were devised to provide precise and intuitive freehand control. We compared the involuntary tool-tip motion occurring while grasping with a standard manual micro-forceps and our touch sensor activated micro-forceps. The results showed that our touch-sensor-based and motor-actuated tool can significantly attenuate the motion artifact during grasping (119.81 μm with our device versus 330.73 μm with the standard micro-forceps). By activating the CP-SSOCT based depth locking feature, the erroneous tool-tip motion can be further reduced down to 5.11μm. We evaluated the performance of our device in comparison to the standard instrument in terms of the elapsed time, the number of grasping attempts, and the maximum depth of damage created on the substrate surface while trying to pick up small pieces of fibers (Ø 125 μm) from a soft polymer surface. The results indicate that all metrics were significantly improved when using our device; of note, the average elapsed time, the number of grasping attempts, and the maximum depth of damage were reduced by 25%, 31%, and 75%, respectively.

In-Situ Force Augmentation Improves Surface Contact and Force Control

Surgeons routinely perform surgery with noisy, sub-threshold, or obscured visual and haptic feedback, either due to the necessary surgical approach, or because the systems on which they are operating are exceedingly delicate. Technological solutions incorporating haptic feedback augmentation have been proposed to address these difficulties, but the consequences for motor control have not been directly investigated and quantified. In this paper, we present two isometric force generation tasks performed with a hand-held robotic tool that provides in-situ augmentation of force sensation. An initial study indicated that magnification helps the operator maintain a desired supra-threshold target force in the absence of visual feedback. We further found that such force magnification reduces the mean and standard deviation of applied forces, and reduces the magnitude of power in the 4 to 7 Hz band corresponding to tremor. Specific benefits to stability, voluntary control, and tremor were observed in the pull direction, which has been previously identified as more dexterous compared to push.

The force pyramid: a spatial analysis of force application during virtual reality brain tumor resection

OBJECTIVE

Virtual reality simulators allow development of novel methods to analyze neurosurgical performance. The concept of a force pyramid is introduced as a Tier 3 metric with the ability to provide visual and spatial analysis of 3D force application by any instrument used during simulated tumor resection. This study was designed to answer 3 questions: 1) Do study groups have distinct force pyramids? 2) Do handedness and ergonomics influence force pyramid structure? 3) Are force pyramids dependent on the visual and haptic characteristics of simulated tumors?

METHODS

Using a virtual reality simulator, NeuroVR (formerly NeuroTouch), ultrasonic aspirator force application was continually assessed during resection of simulated brain tumors by neurosurgeons, residents, and medical students. The participants performed simulated resections of 18 simulated brain tumors with different visual and haptic characteristics. The raw data, namely, coordinates of the instrument tip as well as contact force values, were collected by the simulator. To provide a visual and qualitative spatial analysis of forces, the authors created a graph, called a force pyramid, representing force sum along the z-coordinate for different xy coordinates of the tool tip.

RESULTS

Sixteen neurosurgeons, 15 residents, and 84 medical students participated in the study. Neurosurgeon, resident and medical student groups displayed easily distinguishable 3D “force pyramid fingerprints.” Neurosurgeons had the lowest force pyramids, indicating application of the lowest forces, followed by resident and medical student groups. Handedness, ergonomics, and visual and haptic tumor characteristics resulted in distinct well-defined 3D force pyramid patterns.

CONCLUSIONS

Force pyramid fingerprints provide 3D spatial assessment displays of instrument force application during simulated tumor resection. Neurosurgeon force utilization and ergonomic data form a basis for understanding and modulating resident force application and improving patient safety during tumor resection.

A Force-Visualized Silicone Retractor Attachable to Surgical Suction Pipes

This paper presents a force-visually-observable silicone retractor, which is an extension of a previously developed system that had the same functions of retracting, suction, and force sensing. These features provide not only high usability by reducing the number of tool changes, but also a safe choice of retracting by visualized force information. Suction is achieved by attaching the retractor to a suction pipe. The retractor has a deformable sensing component including a hole filled with a liquid. The hole is connected to an outer tube, and the liquid level displaced in proportion to the extent of deformation resulting from the retracting load. The liquid level is capable to be observed around the surgeon’s fingertips, which enhances the usability. The new hybrid structure of soft sensing and hard retracting allows the miniaturization of the retractor as well as a resolution of less than 0.05 N and a range of 0.1–0.7 N. The overall structure is made of silicone, which has the advantages of disposability, low cost, and easy sterilization/disinfection. This system was validated by conducting experiments.

Development and preliminary results of bimanual smart micro-surgical system using a ball-lens coupled OCT distance sensor

Bimanual surgery enhances surgical effectiveness and is required to successfully accomplish complex microsurgical tasks. The essential advantage is the ability to simultaneously grasp tissue with one hand to provide counter traction or exposure, while dissecting with the other. Towards enhancing the precision and safety of bimanual microsurgery we present a bimanual SMART micro-surgical system for a preliminary ex-vivo study. To the best of our knowledge, this is the first demonstration of a handheld bimanual microsurgical system. The essential components include a ball-lens coupled common-path swept source optical coherence tomography sensor. This system effectively suppresses asynchronous hand tremor using two PZT motors in feedback control loop and efficiently assists ambidextrous tasks. It allows precise bimanual dissection of biological tissues with a reduction in operating time as compared to the same tasks performed with conventional one-handed approaches.

A "Smart" Force-Limiting Instrument for Microsurgery: Laboratory and In Vivo Validation

Residents are required to learn a multitude of skills during their microsurgical training. One such skill is the judicious application of force when handling delicate tissue. An instrument has been developed that indicates to the surgeon when a force threshold has been exceeded by providing vibrotactile feedback. The objective of this study was to validate the use of this “smart” force-limiting instrument for microsurgery. A laboratory and an in vivo experiment were performed to evaluate the force-limiting instrument. In the laboratory experiment, twelve novice surgeons were randomly allocated to use either the force-limiting instrument or a standard instrument. Surgeons were then asked to perform microsurgical dissection in a model. In the in vivo experiment, an intermediate surgeon performed microsurgical dissection in a stepwise fashion, alternating every 30 seconds between use of the force-limiting instrument and a standard instrument. The primary outcomes were the forces exerted and the OSATS scores. In the laboratory experiment, the maximal forces exerted by novices using the force-limiting instrument were significantly less than using a standard instrument, and were comparable to intermediate and expert surgeons (0.637N versus 4.576N; p = 0.007). In the in vivo experiment, the maximal forces exerted with the force-limiting instrument were also significantly less than with a standard instrument (0.441N versus 0.742N; p <0.001). Notably, use of the force-limiting instrument did not significantly impede the surgical workflow as measured by the OSATS score (p >0.1). In conclusion, the development and use of this force-limiting instrument in a clinical setting may improve patient safety.

Force-Sensing Silicone Retractor for Attachment to Surgical Suction Pipes

This paper presents a novel force-sensing silicone retractor that can be attached to a surgical suction pipe to improve the usability of the suction and retraction functions during neurosurgery. The retractor enables simultaneous utilization of three functions: suction, retraction, and retraction-force sensing. The retractor also reduces the number of tool changes and ensures safe retraction through visualization of the magnitude of the retraction force. The proposed force-sensing system is based on a force visualization mechanism through which the force is displayed in the form of motion of a colored pole. This enables surgeons to estimate the retraction force. When a fiberscope or camera is present, the retractor enables measurement of the retraction force with a resolution of 0.05 N. The retractor has advantages of being disposable, inexpensive, and easy to sterilize or disinfect. Finite element analysis and experiments demonstrate the validity of the proposed force-sensing system.