An extended algorithm for autonomous grasping of soft tissues during robotic surgery

Landet: an efficient physics-informed deep learning approach for automatic detection of anatomical landmarks and measurement of spinopelvic alignment

PURPOSE

An efficient physics-informed deep learning approach for extracting spinopelvic measures from X-ray images is introduced and its performance is evaluated against manual annotations.

METHODS

Two datasets, comprising a total of 1470 images, were collected to evaluate the model's performance. We propose a novel method of detecting landmarks as objects, incorporating their relationships as constraints (LanDet). Using this approach, we trained our deep learning model to extract five spine and pelvis measures: Sacrum Slope (SS), Pelvic Tilt (PT), Pelvic Incidence (PI), Lumbar Lordosis (LL), and Sagittal Vertical Axis (SVA). The results were compared to manually labelled test dataset (GT) as well as measures annotated separately by three surgeons.

RESULTS

The LanDet model was evaluated on the two datasets separately and on an extended dataset combining both. The final accuracy for each measure is reported in terms of Mean Absolute Error (MAE), Standard Deviation (SD), and R Pearson correlation coefficient as follows: [ S S ∘ : 3.7 ( 2.7 ) , R = 0.89 ] ,[ P T ∘ : 1.3 ( 1.1 ) , R = 0.98 ] , [ P I ∘ : 4.2 ( 3.1 ) , R = 0.93 ] , [ L L ∘ : 5.1 ( 6.4 ) , R = 0.83 ] , [ S V A ( m m ) : 2.1 ( 1.9 ) , R = 0.96 ] . To assess model reliability and compare it against surgeons, the intraclass correlation coefficient (ICC) metric is used. The model demonstrated better consistency with surgeons with all values over 0.88 compared to what was previously reported in the literature.

CONCLUSION

The LanDet model exhibits competitive performance compared to existing literature. The effectiveness of the physics-informed constraint method, utilized in our landmark detection as object algorithm, is highlighted. Furthermore, we addressed the limitations of heatmap-based methods for anatomical landmark detection and tackled issues related to mis-identifying of similar or adjacent landmarks instead of intended landmark using this novel approach.

Three-Dimensional Collision Avoidance Method for Robot-Assisted Minimally Invasive Surgery

In the robot-assisted minimally invasive surgery, if a collision occurs, the robot system program could be damaged, and normal tissues could be injured. To avoid collisions during surgery, a 3-dimensional collision avoidance method is proposed in this paper. The proposed method is predicated on the design of 3 strategic vectors: the collision-with-instrument-avoidance (CI) vector, the collision-with-tissues-avoidance (CT) vector, and the constrained-control (CC) vector. The CI vector demarcates 3 specific directions to forestall collision among the surgical instruments. The CT vector, on the other hand, comprises 2 components tailored to prevent inadvertent contact between the robot-controlled instrument and nontarget tissues. Meanwhile, the CC vector is introduced to guide the endpoint of the robot-controlled instrument toward the desired position, ensuring precision in its movements, in alignment with the surgical goals. Simulation results verify the proposed collision avoidance method for robot-assisted minimally invasive surgery. The code and data are available at https://github.com/cynerelee/collision-avoidance.

Design and Fabrication of a Fiber Bragg Grating Shape Sensor for Shape Reconstruction of a Continuum Manipulator

Continuum dexterous manipulators (CDMs) are suitable for performing tasks in a constrained environment due to their high dexterity and maneuverability. Despite the inherent advantages of CDMs in minimally invasive surgery, real-time control of CDMs' shape during nonconstant curvature bending is still challenging. This study presents a novel approach for the design and fabrication of a large deflection fiber Bragg grating (FBG) shape sensor embedded within the lumens inside the walls of a CDM with a large instrument channel. The shape sensor consisted of two fibers, each with three FBG nodes. A shape-sensing model was introduced to reconstruct the centerline of the CDM based on FBG wavelengths. Different experiments, including shape sensor tests and CDM shape reconstruction tests, were conducted to assess the overall accuracy of the shape-sensing. The FBG sensor evaluation results revealed the linear curvature-wavelength relationship with the large curvature detection of 0.045 mm and a high wavelength shift of up to 5.50 nm at a 90° bending angle in both the bending directions. The CDM's shape reconstruction experiments in a free environment demonstrated the shape-tracking accuracy of 0.216 ± 0.126 mm for positive/negative deflections. Also, the CDM shape reconstruction error for three cases of bending with obstacles was observed to be 0.436 ± 0.370 mm for the proximal case, 0.485 ± 0.418 mm for the middle case, and 0.312 ± 0.261 mm for the distal case. This study indicates the adequate performance of the FBG sensor and the effectiveness of the model for tracking the shape of the large-deflection CDM with nonconstant-curvature bending for minimally invasive orthopedic applications.

A multi-module soft robotic arm with soft actuator for minimally invasive surgery

BACKGROUND

Compared to traditional rigid robotic arms, soft robotic arms are flexible, environmentally adaptable and biocompatible. Recently, most minimally invasive cardiac procedures still rely on traditional rigid surgical tools. However, rigid tools lack sufficient bending angles, which are high-risk in terms of contact with tissues and organs.

METHODS

A soft robotic arm with multiple degrees of freedom was designed to repair atrial septal defects in cardiac surgery. The developed multi-module soft robotic arm consists of four different units, including a bending unit, a turning unit, a stretching unit and gripper units. The three movement units can reach the specified position, and the gripper units can hold a surgical tool stably, such as a suture needle in cardiac surgery.

RESULTS

A cardiac surgery to repair an atrial septal defect has been completed, validating the reliability and functionality of the developed multi-module soft robotic arm.

CONCLUSIONS

The multi-module flexible soft robotic arm for minimally invasive surgery proposed in this paper can reach the designated surgical area during surgery to repair Atrial Septal Defects. Meanwhile, the design of the actuator of the robot arm was used a completely soft silicone material replacing the rigid material, which releases the contact trauma of the organs during the surgery.