Design and comparison of computationally efficient uniaxial stress-strain models of the lung parenchyma for real-time applications

Real-time clinical applications such as robotic lung surgery, tumor localization, atelectasis diagnosis, tumor motion prediction for radiation therapy of lung cancer, or surgery training are in need of biomechanical models of lungs, not necessarily highly accurate, but with good computational properties. These properties can include one or several of the following: low computation time, low memory resource requirement, a low number of parameters, and ease of parameter identification in real-time. Among the numerous existing models of lung parenchyma, some may be well suited for real-time applications; however, they should be extensively assessed against both accuracy and computational efficiency criteria to make an informed choice depending on the requirements of the application. After demonstrating how to derive a real-time compliant force-indentation model from a unixial stress-strain model with rational expression, the core purpose of this paper is to propose such an evaluation of selected models in fitting human lung parenchyma experimental and synthetic data of uniaxial tension. Furthermore, new uniaxial stress-strain models are developed based on an empirical observation of the volumetric behavior of the lungs along with an emphasis on computational performance. These new proposed models are competitive with existing one in terms of computational efficiency and compliance with experimental and synthetic data. One of them reduces the prediction error by 2 compared to other investigated models while maintaining an excellent adjusted coefficient of determination between 0.999 and 1 across various datasets. It exhibits excellent real-time capabilities with an explicit rational expression, only 3 parameters and linear numerator and denominator in the parameters. It is computed with only 20 floating point operations (flops) while another proposed model even requires as few as 2 flops.

A Minimally Invasive Robotic Tissue Palpation Device

OBJECTIVE

Robot-assisted minimally invasive surgery remains limited by the absence of haptic feedback, which surgeons routinely rely on to assess tissue stiffness. This limitation hinders surgeons' ability to identify and treat abnormal tissues, such as tumors, during robotic surgery.

METHODS

To address this challenge, we developed a robotic tissue palpation device capable of rapidly and non-invasively quantifying the stiffness of soft tissues, allowing surgeons to make objective and data-driven decisions during minimally invasive procedures. We evaluated the effectiveness of our device by measuring the stiffness of phantoms as well as lung, heart, liver, and skin tissues obtained from both rats and swine.

RESULTS

Results demonstrated that our device can accurately determine tissue stiffness and identify tumor mimics. Specifically, in swine lung, we determined elastic modulus (E) values of 9.1 ± 2.3, 16.8 ± 1.8, and 26.0 ± 3.6 kPa under different internal pressure of the lungs (PIP) of 2, 25, and 45 cmH2O, respectively. Using our device, we successfully located a 2-cm tumor mimic embedded at a depth of 5 mm in the lung subpleural region. Additionally, we measured E values of 33.0 ± 5.4, 19.2 ± 2.2, 33.5 ± 8.2, and 22.6 ± 6.0 kPa for swine heart, liver, abdominal skin, and muscle, respectively, which closely matched existing literature data.

CONCLUSION/SIGNIFICANCE

Results suggest that our robotic palpation device can be utilized during surgery, either as a stand-alone or additional tool integrated into existing robotic surgical systems, to enhance treatment outcomes by enabling accurate intraoperative identification of abnormal tissue.

A roller scanning type prostate palpation sensor system using a cantilever beam type force measurement

Palpation has been used for centuries in medicine as a screening and diagnosis modality for a number of pathological conditions. However, this technique is subjective since palpation sensitivity depends on the skill and experience of a medical examiner. An objective approach to acquire quantitative tactile information is needed. A roller scanning type palpation sensor system consisting of an x-axis linear slider and a palpation probe for detecting lumps in a soft object such as the prostate tissue is proposed and fabricated in this study. By translating the slider and changing the palpation angle of the probe, a chrome steel ball which was the contact component indented and scanned the surface of soft silicone phantoms embedded with hard lumps. Lumps were detected by measuring reaction force waveform fluctuations. Fundamental characteristics of the proposed sensor system were validated by comparison with the ground truth load cell output and showed a strong linear correlation with r²=0.9985. On silicone samples with hard lumps, the system was able to detect lumps of various sizes embedded at the depth of 5mm. From the results, the proposed sensor system holds potential for tissue characterization.

A decade retrospective of medical robotics research from 2010 to 2020

[Figure: see text].

Miniaturized optical fiber probe for prostate cancer screening

Tissue elasticity is universally recognized as a diagnostic and prognostic biomarker for prostate cancer. As the first diagnostic test, the digital rectal examination is used since malignancy changes the prostate morphology and affects its mechanical properties. Currently, this examination is performed manually by the physician, with an unsatisfactory positive predictive value of 42%. A more objective and spatially selective technique is expected to provide a better prediction degree and understanding of the disease. To this aim, here we propose a miniaturized probe, based on optical fiber sensor technology, for mechanical characterization of the prostate with sub-millimeter resolution. Specifically, the optical system incorporates a customized Fiber Bragg Grating, judiciously integrated in a metallic cannula and moved by a robotic arm. The probe enables the local measurement of the force upon tissue indentation with a resolution of 0.97 mN. The system has been developed in such a way to be potentially used directly in vivo. Measurements performed on phantom tissues mimicking different stages of the prostatic carcinoma demonstrated the capability of our device to distinguish healthy from diseased zones of the prostate. The study on phantoms has been complemented with preliminary ex vivo experiments on real organs obtained from radical surgeries. Our findings lay the foundation for the development of advanced optical probes that, when integrated inside biopsy needle, are able to perform in vivo direct mechanical measurements with high sensitivity and spatial resolution, opening to new scenarios for early diagnosis and enhanced diagnostic accuracy of prostate cancer.

Self-Sensing Pneumatic Compressing Actuator

Using soft pneumatic actuator is a feasible solution in the complex unstructured environment, owing to their inherent compliance, light weight, and safety. However, due to the limitations of soft actuators' materials and structures, they fall short of motion accuracy and load capacity, or need large-size, bulky compressors. Meanwhile, in order to gain better control, it is essential for them to sense the environments as well. This leads to high-price sensors or a complicated manufacture technique. Here, a self-sensing vacuum soft actuation structure is proposed, aiming at acquiring good balance among precision, output force, and actuation pressure. The actuator mainly comprises a flexible membrane and a compression spring. When actuated, the flexible membrane outside the actuator compresses the internal spring skeleton, realizing large contractile motion in axial direction. Its built-in force sensor can indirectly measure the absolute displacement of the actuator with certain accuracy (about 5% F.S.). Besides, it does not require high actuation pressure to generate enough output force. The actuator is quite easy to manufacture with low cost, and there are a variety of materials to choose from. We established quasi-static models for actuators built of two different kinds of membrane materials, and tested their accuracy and output force. In addition, to break through the limits of vacuum actuation, a method of positive-negative pressure combined actuation has been proposed, which lowers the requirements for air source equipments, increases actuation pressure, and reduces potential safety threats at the same time. This kind of soft actuators can also effectively resist and detect impacts. The design of a two-finger dexterous robot hand and robot joint based on this soft actuator illustrates its broad application prospects in the fields of mobile robots, wearable devices, and human–robot interaction.

Reaction Force Mapping by 3-Axis Tactile Sensing With Arbitrary Angles for Tissue Hard-Inclusion Localization

Although robot-assisted diagnosis and minimally invasive surgery (MIS) brings distinct benefits, deficient multi-dimensional force feedback remains a noteworthy limitation and challenge in MIS. Aiming for a comprehensive high-fidelity perception of tissue-instrument interactions, we present a Fiber Bragg Grating (FBG)-based 3-axis tactile sensing for surface reaction force mapping, identification and localization of tissue hard-inclusion. The tactile sensing probe consists of five optical fibers inscribed with FBGs and a force-sensitive 3D printed deformable body. All fibers are suspended inside the deformable body in a parallel manner, leading to the direct compression or tension of each FBG. Such configuration can effectively avoid the chirping failure of FBG compared with the pasting FBG-based sensors. A linearized difference model is proposed to calibrate the 3-axis force detection and enhance the resistance to nonlinear interferences. Hard-inclusion identification experiments with varied hard-inclusion sizes and depths have been implemented through discrete palpation and dragging palpation modes. Results indicate that the probe can effectively identify the presence and location of these small hard-inclusions from the force mapping. Furthermore, lengthy vessels embedded in the phantom can be accurately identified through dragging palpation with an arbitrary contact angle. Another novelty of the probe is the reconstruction of the surface profile of a non-planar tissue, which further allows hard-inclusion identification and 3D localization. Ex-vivo tissue palpation on a porcine kidney further validates the effectiveness and feasibility of the probe to map surface reaction forces and localize the hard-inclusions intraoperatively.

A Fiber Bragg Grating Sensing Structure for the Design, Simulation and Stress Strain Monitoring of Human Puncture Surgery

In order to improve the precision and stability of puncture surgical operations to assist doctors in completing fine manipulation, a new of type puncturing needle sensor is proposed based on a fiber Bragg grating (FBG). Compared with the traditional puncture needle sensor, the new type of puncturing needle sensor is able to sense not only the axial force, but also the torque force during the puncture process. A spoke-type structure is designed near the needle tip. In order to eliminate the influence of temperature and realize temperature compensation, a reference fiber method using three FBGs is applied. FBG1 and the reference FBG2 are pasted on the upper and lower surfaces of the new-type elastic beam, and FBG3 is pasted into the groove on the surface of the new type of puncturing needle cylinder. The difference of Bragg wavelength between FBG1 and the reference FBG2 is calibrated with the torque force, while the difference between the Bragg wavelength of the FBG3 and the reference FBG2 is calibrated with the axial force. Through simulation and sensing tests, when the torque force calibration range is 10 mN·m, the torque average sensitivity is 22.8 pm/mN·m, and the determination coefficient R2 is 0.99992, with a hysteresis error YH and repetition error YR of 0.03%FS and 0.81%FS, respectively. When the axial force calibration rang is 5 N, the axial force average sensitivity is 0.089 nm/N, and the determination coefficient R2 is 0.9997, with hysteresis error YH and repetition error YR of 0.014%FS and 0.11%FS, respectively. The axial force resolution and torque resolution of the new type of puncturing needle sensor are 0.03 N and 0.8 mN·m, respectively. The experimental data and simulation analysis show that the proposed new type of puncturing needle sensor has good practicability and versatility.

Compensating Uncertainties in Force Sensing for Robotic-Assisted Palpation

Force sensing in robotic-assisted minimally invasive surgery (RMIS) is crucial for performing dedicated surgical procedures, such as bilateral teleoperation and palpation. Due to the bio-compatibility and sterilization requirements, a specially designed surgical tool/shaft is normally attached to the sensor while contacting the organ targets. Through this design, the measured force from the sensor usually contains uncertainties, such as noise, inertial force etc., and thus cannot reflect the actual interaction force with the tissue environment. Motivated to provide the authentic contact force between a robotic tool and soft tissue, we proposed a data-driven force compensation scheme without intricate modeling to reduce the effects of force measurement uncertainties. In this paper, a neural-network-based approach is utilized to automatically model the inertial force subject to noise during the robotic palpation procedure, then the exact contact force can be obtained through the force compensation method which cancels the noise and inertial force. Following this approach, the genuine interaction force during the palpation task can be achieved furthermore to improve the appraisal of the tumor surrounded by the soft tissue. Experiments are conducted with robotic-assisted palpation tasks on a silicone-based soft tissue phantom and the results verify the effectiveness of the suggested method.

Magnetic Levitation Haptic Augmentation for Virtual Tissue Stiffness Perception

Haptic-based tissue stiffness perception is essential for palpation training system, which can provide the surgeon haptic cues for improving the diagnostic abilities. However, current haptic devices, such as Geomagic Touch, fail to provide immersive and natural haptic interaction in virtual surgery due to the inherent mechanical friction, inertia, limited workspace and flawed haptic feedback. To tackle this issue, we design a novel magnetic levitation haptic device based on electromagnetic principles to augment the tissue stiffness perception in virtual environment. Users can naturally interact with the virtual tissue by tracking the motion of magnetic stylus using stereoscopic vision so that they can accurately sense the stiffness by the magnetic stylus, which moves in the magnetic field generated by our device. We propose the idea that the effective magnetic field (EMF) is closely related to the coil attitude for the first time. To fully harness the magnetic field and flexibly generate the specific magnetic field for obtaining required haptic perception, we adopt probability clouds to describe the requirement of interactive applications and put forward an algorithm to calculate the best coil attitude. Moreover, we design a control interface circuit and present a self-adaptive fuzzy proportion integration differentiation (PID) algorithm to precisely control the coil current. We evaluate our haptic device via a series of quantitative experiments which show the high consistency of the experimental and simulated magnetic flux density, the high accuracy (0.28 mm) of real-time 3D positioning and tracking of the magnetic stylus, the low power consumption of the adjustable coil configuration, and the tissue stiffness perception accuracy improvement by 2.38 percent with the self-adaptive fuzzy PID algorithm. We conduct a user study with 22 participants, and the results suggest most of the users can clearly and immersively perceive different tissue stiffness and easily detect the tissue abnormality. Experimental results demonstrate that our magnetic levitation haptic device can provide accurate tissue stiffness perception augmentation with natural and immersive haptic interaction.

Real-Time Vision-Based Stiffness Mapping †

This paper presents new findings concerning a hand-held stiffness probe for the medical diagnosis of abnormalities during palpation of soft-tissue. Palpation is recognized by the medical community as an essential and low-cost method to detect and diagnose disease in soft-tissue. However, differences are often subtle and clinicians need to train for many years before they can conduct a reliable diagnosis. The probe presented here fills this gap providing a means to easily obtain stiffness values of soft tissue during a palpation procedure. Our stiffness sensor is equipped with a multi degree of freedom (DoF) Aurora magnetic tracker, allowing us to track and record the 3D position of the probe whilst examining a tissue area, and generate a 3D stiffness map in real-time. The stiffness probe was integrated in a robotic arm and tested in an artificial environment representing a good model of soft tissue organs; the results show that the sensor can accurately measure and map the stiffness of a silicon phantom embedded with areas of varying stiffness.

Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model

Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation.

Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation

BACKGROUND

Haptic feedback has been proven to play a key role in enhancing the performance of teleoperated medical procedures. However, due to safety issues, commercially-available medical robots do not currently provide the clinician with haptic feedback.

METHODS

This work presents the experimental evaluation of a teleoperation system for robot-assisted medical procedures able to provide magnified haptic feedback to the clinician. Forces registered at the operating table are magnified and provided to the clinician through a 7-DoF haptic interface. The same interface is also used to control the motion of a 6-DoF slave robotic manipulator. The safety of the system is guaranteed by a time-domain passivity-based control algorithm.

RESULTS

Two experiments were carried out on stiffness discrimination (during palpation and needle insertion) and one experiment on needle guidance.

CONCLUSIONS

Our haptic-enabled teleoperation system improved the performance with respect to direct hand interaction of 80%, 306%, and 27% in stiffness discrimination through palpation, stiffness discrimination during needle insertion, and guidance, respectively.

Evaluation of Pseudo-Haptic Interactions with Soft Objects in Virtual Environments

This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness information through a visual interface. The method exploits a combination of two feedback techniques, namely visual feedback of soft surface deformation and control of the indenter avatar speed, to convey stiffness information of a simulated surface of a soft object in virtual environments. The proposed method was effective in distinguishing different sizes of virtual hard nodules integrated into the simulated soft bodies. To further improve the interactive experience, the approach was extended creating a multi-point pseudo-haptic feedback system. A comparison with regards to (a) nodule detection sensitivity and (b) elapsed time as performance indicators in hard nodule detection experiments to a tablet computer incorporating vibration feedback was conducted. The multi-point pseudo-haptic interaction is shown to be more time-efficient than the single-point pseudo-haptic interaction. It is noted that multi-point pseudo-haptic feedback performs similarly well when compared to a vibration-based feedback method based on both performance measures elapsed time and nodule detection sensitivity. This proves that the proposed method can be used to convey detailed haptic information for virtual environmental tasks, even subtle ones, using either a computer mouse or a pressure sensitive device as an input device. This pseudo-haptic feedback method provides an opportunity for low-cost simulation of objects with soft surfaces and hard inclusions, as, for example, occurring in ever more realistic video games with increasing emphasis on interaction with the physical environment and minimally invasive surgery in the form of soft tissue organs with embedded cancer nodules. Hence, the method can be used in many low-budget applications where haptic sensation is required, such as surgeon training or video games, either using desktop computers or portable devices, showing reasonably high fidelity in conveying stiffness perception to the user.

Frictional Compliant Haptic Contact and Deformation of Soft Objects

This paper is concerned with compliant haptic contact and deformation of soft objects. A human soft fingertip model is considered to act as the haptic interface and is brought into contact with and deforms a discrete surface. A nonlinear constitutive law is developed in predicting normal forces and, for the haptic display of surface texture, motions along the surface are also resisted at various rates by accounting for dynamic Lund-Grenoble (LuGre) frictional forces. For the soft fingertip to apply forces over an area larger than a point, normal and frictional forces are distributed around the soft fingertip contact location on the deforming surface. The distribution is realized based on a kernel smoothing function and by a nonlinear spring-damper net around the contact point. Experiments conducted demonstrate the accuracy and effectiveness of our approach in real-time haptic rendering of a kidney surface. The resistive (interaction) forces are applied at the user fingertip bone edge. A 3-DoF parallel robotic manipulator equipped with a constraint based controller is used for the implementation. By rendering forces both in lateral and normal directions, the designed haptic interface system allows the user to realistically feel both the geometrical and mechanical (nonlinear) properties of the deforming kidney.

Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue

This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback-based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user's haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks was compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5%, the positive predictive value by 4%, and decreased detection time by 48.7%. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed.

Wireless tissue palpation for intraoperative detection of lumps in the soft tissue

In an open surgery, identification of precise margins for curative tissue resection is performed by manual palpation. This is not the case for minimally invasive and robotic procedures, where tactile feedback is either distorted or not available. In this paper, we introduce the concept of intraoperative wireless tissue palpation. The wireless palpation probe (WPP) is a cylindrical device (15 mm in diameter, 60 mm in length) that can be deployed through a trocar incision and directly controlled by the surgeon to create a volumetric stiffness distribution map of the region of interest. This map can then be used to guide the tissue resection to minimize healthy tissue loss. The wireless operation prevents the need for a dedicated port and reduces the chance of instrument clashing in the operating field. The WPP is able to measure in real time the indentation pressure with a sensitivity of 34 Pa, the indentation depth with an accuracy of 0.68 mm, and the probe position with a maximum error of 11.3 mm in a tridimensional workspace. The WPP was assessed on the benchtop in detecting the local stiffness of two different silicone tissue simulators (elastic modulus ranging from 45 to 220 kPa), showing a maximum relative error below 5%. Then, in vivo trials were aimed to identify an agar-gel lump injected into a porcine liver and to assess the device usability within the frame of a laparoscopic procedure. The stiffness map created intraoperatively by the WPP was compared with a map generated ex vivo by a standard uniaxial material tester, showing less than 8% local stiffness error at the site of the lump.

Optical-based artificial palpation sensors for lesion characterization

Palpation techniques are widely used in medical procedures to detect the presence of lumps or tumors in the soft breast tissues. Since these procedures are very subjective and depend on the skills of the physician, it is imperative to perform detailed a scientific study in order to develop more efficient medical sensors to measure and generate palpation parameters. In this research, we propose an optical-based, artificial palpation sensor for lesion characterization. This has been developed using a multilayer polydimethylsiloxane optical waveguide. Light was generated at the critical angle to reflect totally within the flexible and transparent waveguide. When a waveguide was compressed by an external force, its contact area would deform and cause the light to scatter. The scattered light was captured by a high-resolution camera and saved as an image format. To test the performance of the proposed system, we used a realistic tissue phantom with embedded hard inclusions. The experimental results show that the proposed sensor can detect inclusions and provide the relative value of size, depth, and Young's modulus of an inclusion.

A stiffness probe based on force and vision sensing for soft tissue diagnosis

this paper introduces a novel approach of stiffness measurement based on force and vision sensing for tissue diagnosis. The developed probe is mainly composed of a force sensor and an image acquisition unit capable of obtaining contact area of probe-soft tissue interaction. By measuring the change of diameter of contact area during indentation test, the indentation depth can be determined. The stiffness of target soft tissue then can be evaluated by measuring indentation force and depth simultaneously. The probe can generalize a mechanical image to visualize the stiffness distribution for localization of abnormalities when sliding over soft tissue. The performance of the developed probe was validated by experiments on multiple materials including silicone phantoms and pork organs. The results show that the probe can perform stiffness measurement effectively when the probe indents or slides on the tissue surface.